2023-08-11T13:24:02.750 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-11T13:24:02.751 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898 -- /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/orig.config.yaml 2023-08-11T13:24:02.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:24:02.925 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898 branch: wip-rishabh-2023Aug1-b4 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: null first_in_suite: false job_id: '7365898' kernel: client: branch: testing flavor: default kdb: true sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Aug1-b4 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Aug1-b4 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Aug1-b4 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 7fda95956ac10132c9b74016ba832db907df09fa timestamp: 2023-08-10_20:16:46 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-11T13:24:02.925 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa; will attempt to use it 2023-08-11T13:24:02.926 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks 2023-08-11T13:24:02.926 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-11T13:24:02.929 INFO:teuthology.task.internal:Checking packages... 2023-08-11T13:24:02.969 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '92684807f016ce045ac66e37fdde6f8c47857bcc' 2023-08-11T13:24:02.970 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-11T13:24:02.970 INFO:teuthology.packaging:ref: None 2023-08-11T13:24:02.970 INFO:teuthology.packaging:tag: None 2023-08-11T13:24:02.971 INFO:teuthology.packaging:branch: wip-rishabh-2023Aug1-b4 2023-08-11T13:24:02.971 INFO:teuthology.packaging:sha1: 92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:24:02.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Aug1-b4 2023-08-11T13:24:03.109 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5437.g92684807 2023-08-11T13:24:03.110 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-11T13:24:03.121 INFO:teuthology.task.internal:no buildpackages task found 2023-08-11T13:24:03.121 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-11T13:24:03.136 INFO:teuthology.task.internal:Saving configuration 2023-08-11T13:24:03.178 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-11T13:24:03.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T13:24:03.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.764735', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T13:24:03.232 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.766683', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T13:24:03.254 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.768394', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-11T13:24:03.254 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-11T13:24:03.263 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-08-11T13:24:03.264 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-08-11T13:24:03.264 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-08-11T13:24:03.264 INFO:teuthology.run_tasks:Running task console_log... 2023-08-11T13:24:03.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2b5b1ed0d0>, signals=[15]) 2023-08-11T13:24:03.401 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-11T13:24:03.411 INFO:teuthology.task.internal:Opening connections... 2023-08-11T13:24:03.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T13:24:03.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:24:03.487 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2023-08-11T13:24:03.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:24:03.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2023-08-11T13:24:03.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:24:03.631 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-11T13:24:03.643 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-08-11T13:24:03.668 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-08-11T13:24:03.668 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-08-11T13:24:03.727 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T13:24:03.728 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T13:24:03.728 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel" 2023-08-11T13:24:03.728 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora" 2023-08-11T13:24:03.728 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.6" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:24:03.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:24:03.730 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:24:03.730 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T13:24:03.730 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T13:24:03.730 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T13:24:03.731 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-08-11T13:24:03.752 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2023-08-11T13:24:03.771 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2023-08-11T13:24:03.772 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2023-08-11T13:24:03.828 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T13:24:03.828 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T13:24:03.828 INFO:teuthology.orchestra.run.smithi103.stdout:ID="rhel" 2023-08-11T13:24:03.828 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="fedora" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8.6" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T13:24:03.829 INFO:teuthology.orchestra.run.smithi103.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T13:24:03.830 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T13:24:03.831 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2023-08-11T13:24:03.852 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2023-08-11T13:24:03.874 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2023-08-11T13:24:03.874 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-08-11T13:24:03.932 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2023-08-11T13:24:03.933 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)" 2023-08-11T13:24:03.933 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2023-08-11T13:24:03.933 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2023-08-11T13:24:03.933 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2023-08-11T13:24:03.934 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-11T13:24:03.935 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-11T13:24:03.936 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-08-11T13:24:03.953 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-11T13:24:03.969 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-11T13:24:03.979 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-11T13:24:03.980 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2023-08-11T13:24:03.983 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2023-08-11T13:24:03.987 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2023-08-11T13:24:04.005 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-11T13:24:04.014 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-11T13:24:04.014 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2023-08-11T13:24:04.042 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2023-08-11T13:24:04.044 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2023-08-11T13:24:04.072 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-11T13:24:04.154 INFO:teuthology.run_tasks:Running task kernel... 2023-08-11T13:24:04.173 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a7fb1265323db972dd333f71b9a53e9479f62e37'} 2023-08-11T13:24:04.174 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-08-11T13:24:04.174 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-08-11T13:24:04.175 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T13:24:04.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:24:04.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-08-11T13:24:04.380 DEBUG:teuthology.task.kernel:sha1 for client.0 is a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T13:24:04.381 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a7fb1265323db972dd333f71b9a53e9479f62e37"... 2023-08-11T13:24:04.381 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-08-11T13:24:04.402 INFO:teuthology.orchestra.run.smithi003.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-11T13:24:04.402 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T13:24:04.404 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-11T13:24:04.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum upgrade -y linux-firmware 2023-08-11T13:24:05.611 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-08-11T13:24:05.611 INFO:teuthology.orchestra.run.smithi003.stdout:Unable to read consumer identity 2023-08-11T13:24:07.032 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-08-11T13:24:07.690 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-08-11T13:24:11.424 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 285 kB/s | 24 kB 00:00 2023-08-11T13:24:12.244 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-08-11T13:24:12.246 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2023-08-11T13:24:12.246 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-08-11T13:24:12.369 INFO:teuthology.task.kernel:Downloading kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on client.0... 2023-08-11T13:24:12.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7fb1265323db972dd333f71b9a53e9479f62e37 2023-08-11T13:24:12.520 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/ 2023-08-11T13:24:12.521 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/ --input-file=- 2023-08-11T13:24:12.523 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2023-08-11T13:26:03.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11 13:26:03 URL:https://1.chacra.ceph.com/r/kernel/testing/a7fb1265323db972dd333f71b9a53e9479f62e37/centos/8/flavors/default/x86_64/kernel-6.5.0_rc4_ga7fb1265323d-1.x86_64.rpm [380807140/380807140] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-11T13:26:03.041 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2023-08-11 13:26:03-- 2023-08-11T13:26:03.042 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 1m 50s 2023-08-11T13:26:03.042 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 363M in 1m 50s (3.30 MB/s) 2023-08-11T13:26:03.043 INFO:teuthology.task.kernel:Installing kernel a7fb1265323db972dd333f71b9a53e9479f62e37 on client.0... 2023-08-11T13:26:03.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-11T13:26:04.705 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying... ######################################## 2023-08-11T13:26:04.767 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing... ######################################## 2023-08-11T13:26:04.854 INFO:teuthology.orchestra.run.smithi003.stdout:Updating / installing... 2023-08-11T13:26:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-6.5.0_rc4_ga7fb1265323d-1 ######################################## 2023-08-11T13:27:13.903 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-11T13:27:13.907 DEBUG:teuthology.orchestra.run.smithi003:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-11T13:27:13.966 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/config-6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/vmlinuz-6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch 2023-08-11T13:27:13.968 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86 2023-08-11T13:27:13.969 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto 2023-08-11T13:27:13.970 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-11T13:27:13.970 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-11T13:27:13.970 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-11T13:27:13.971 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-11T13:27:13.971 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-11T13:27:13.971 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-11T13:27:13.972 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-11T13:27:13.972 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-11T13:27:13.972 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-11T13:27:13.973 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-11T13:27:13.973 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-11T13:27:13.973 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-11T13:27:13.973 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-11T13:27:13.974 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-11T13:27:13.974 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-11T13:27:13.974 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-11T13:27:13.974 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-11T13:27:13.975 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-11T13:27:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-11T13:27:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-11T13:27:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel 2023-08-11T13:27:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu 2023-08-11T13:27:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce 2023-08-11T13:27:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-11T13:27:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm 2023-08-11T13:27:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-11T13:27:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-11T13:27:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/arch/x86/kvm/kvm.ko 2023-08-11T13:27:13.979 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto 2023-08-11T13:27:13.979 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ansi_cprng.ko 2023-08-11T13:27:13.979 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/anubis.ko 2023-08-11T13:27:13.980 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/arc4.ko 2023-08-11T13:27:13.980 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx 2023-08-11T13:27:13.980 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_memcpy.ko 2023-08-11T13:27:13.981 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_pq.ko 2023-08-11T13:27:13.981 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-11T13:27:13.981 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_tx.ko 2023-08-11T13:27:13.981 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/async_xor.ko 2023-08-11T13:27:13.982 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/async_tx/raid6test.ko 2023-08-11T13:27:13.982 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authenc.ko 2023-08-11T13:27:13.982 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/authencesn.ko 2023-08-11T13:27:13.982 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blake2b_generic.ko 2023-08-11T13:27:13.983 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_common.ko 2023-08-11T13:27:13.983 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/blowfish_generic.ko 2023-08-11T13:27:13.983 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/camellia_generic.ko 2023-08-11T13:27:13.984 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast5_generic.ko 2023-08-11T13:27:13.984 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast6_generic.ko 2023-08-11T13:27:13.984 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cast_common.ko 2023-08-11T13:27:13.985 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/ccm.ko 2023-08-11T13:27:13.985 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cmac.ko 2023-08-11T13:27:13.985 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crc32_generic.ko 2023-08-11T13:27:13.986 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_engine.ko 2023-08-11T13:27:13.986 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/crypto_user.ko 2023-08-11T13:27:13.987 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/cts.ko 2023-08-11T13:27:13.987 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/des_generic.ko 2023-08-11T13:27:13.987 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/dh_generic.ko 2023-08-11T13:27:13.988 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/echainiv.ko 2023-08-11T13:27:13.988 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/essiv.ko 2023-08-11T13:27:13.988 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/fcrypt.ko 2023-08-11T13:27:13.988 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/khazad.ko 2023-08-11T13:27:13.989 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4.ko 2023-08-11T13:27:13.989 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/lz4hc.ko 2023-08-11T13:27:13.989 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/md4.ko 2023-08-11T13:27:13.990 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/michael_mic.ko 2023-08-11T13:27:13.990 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcbc.ko 2023-08-11T13:27:13.990 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/pcrypt.ko 2023-08-11T13:27:13.990 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/rmd160.ko 2023-08-11T13:27:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/seed.ko 2023-08-11T13:27:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/serpent_generic.ko 2023-08-11T13:27:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tcrypt.ko 2023-08-11T13:27:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/tea.ko 2023-08-11T13:27:13.991 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_common.ko 2023-08-11T13:27:13.992 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/twofish_generic.ko 2023-08-11T13:27:13.992 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/vmac.ko 2023-08-11T13:27:13.993 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/wp512.ko 2023-08-11T13:27:13.993 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xcbc.ko 2023-08-11T13:27:13.994 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xor.ko 2023-08-11T13:27:13.994 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/crypto/xxhash_generic.ko 2023-08-11T13:27:13.994 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers 2023-08-11T13:27:13.994 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi 2023-08-11T13:27:13.994 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/acpi_pad.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/custom_method.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/ec_sys.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/platform_profile.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbs.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/sbshc.ko 2023-08-11T13:27:13.995 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/acpi/video.ko 2023-08-11T13:27:13.996 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata 2023-08-11T13:27:13.996 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/acard-ahci.ko 2023-08-11T13:27:13.996 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ahci_platform.ko 2023-08-11T13:27:13.996 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/ata_generic.ko 2023-08-11T13:27:13.996 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/libahci_platform.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_acpi.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ali.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_amd.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_artop.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atiixp.ko 2023-08-11T13:27:13.997 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_atp867x.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd640.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cmd64x.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_cypress.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_efar.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt366.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt37x.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-11T13:27:13.998 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it8213.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_it821x.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_jmicron.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_marvell.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_mpiix.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_netcell.ko 2023-08-11T13:27:13.999 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ninja32.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87410.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_ns87415.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_oldpiix.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_opti.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_optidma.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pcmcia.ko 2023-08-11T13:27:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_piccolo.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_rdc.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sch.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_serverworks.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sil680.ko 2023-08-11T13:27:14.001 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sis.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_sl82c105.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_triflex.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pata_via.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/pdc_adma.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_inic162x.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_mv.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_nv.ko 2023-08-11T13:27:14.002 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_promise.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_qstor.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sil24.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sis.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_svw.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_sx4.ko 2023-08-11T13:27:14.003 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_uli.ko 2023-08-11T13:27:14.004 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_via.ko 2023-08-11T13:27:14.004 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ata/sata_vsc.ko 2023-08-11T13:27:14.004 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay 2023-08-11T13:27:14.004 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/auxdisplay/ks0108.ko 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma 2023-08-11T13:27:14.005 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/bcma/bcma.ko 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/aoe/aoe.ko 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/brd.ko 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/drbd/drbd.ko 2023-08-11T13:27:14.006 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/floppy.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/loop.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/nbd.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/null_blk/null_blk.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/pktcdvd.ko 2023-08-11T13:27:14.007 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/rbd.ko 2023-08-11T13:27:14.008 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/virtio_blk.ko 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/xen-blkfront.ko 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/block/zram/zram.ko 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char 2023-08-11T13:27:14.009 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hangcheck-timer.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/via-rng.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-11T13:27:14.010 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/lp.ko 2023-08-11T13:27:14.011 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/mwave/mwave.ko 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/ppdev.ko 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tlclk.ko 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm.ko 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-11T13:27:14.012 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/uv_mmtimer.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/char/virtio_console.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-11T13:27:14.013 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp 2023-08-11T13:27:14.014 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/ccp/ccp.ko 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-11T13:27:14.015 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-aes.ko 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/padlock-sha.ko 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dca/dca.ko 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq 2023-08-11T13:27:14.016 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac 2023-08-11T13:27:14.017 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/amd64_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/e752x_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/edac_mce_amd.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3000_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i3200_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5100_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i5400_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7300_edac.ko 2023-08-11T13:27:14.018 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i7core_edac.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/i82975x_edac.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/ie31200_edac.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/sb_edac.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/edac/x38_edac.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/edd.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-11T13:27:14.019 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-ich.ko 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-appleir.ko 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-aureal.ko 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-axff.ko 2023-08-11T13:27:14.020 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-dr.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elecom.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-elo.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-emsff.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gaff.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gt683r.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-gyration.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-11T13:27:14.021 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-holtekff.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-hyperv.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-icade.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-keytouch.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-kye.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lcpower.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-led.ko 2023-08-11T13:27:14.022 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-lenovo.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-multitouch.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-ortek.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-penmount.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-petalynx.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-picolcd.ko 2023-08-11T13:27:14.023 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-pl.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-plantronics.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-primax.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-rmi.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-common.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-11T13:27:14.024 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-11T13:27:14.025 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-roccat.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-saitek.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-samsung.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sjoy.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sony.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-speedlink.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-steelseries.ko 2023-08-11T13:27:14.026 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-sunplus.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tivo.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-tmff.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-topseed.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-twinhan.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-uclogic.ko 2023-08-11T13:27:14.027 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-waltop.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-wiimote.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-xinmo.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zpff.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/hid-zydacron.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/uhid.ko 2023-08-11T13:27:14.028 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hid/wacom.ko 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_balloon.ko 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_utils.ko 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hv/hv_vmbus.ko 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru.ko 2023-08-11T13:27:14.029 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/abituguru3.ko 2023-08-11T13:27:14.030 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-11T13:27:14.030 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7414.ko 2023-08-11T13:27:14.031 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ad7418.ko 2023-08-11T13:27:14.032 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adc128d818.ko 2023-08-11T13:27:14.032 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1025.ko 2023-08-11T13:27:14.032 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1026.ko 2023-08-11T13:27:14.033 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1029.ko 2023-08-11T13:27:14.033 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm1031.ko 2023-08-11T13:27:14.033 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adm9240.ko 2023-08-11T13:27:14.034 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ads7828.ko 2023-08-11T13:27:14.034 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7410.ko 2023-08-11T13:27:14.034 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7411.ko 2023-08-11T13:27:14.034 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7462.ko 2023-08-11T13:27:14.035 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7470.ko 2023-08-11T13:27:14.035 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7475.ko 2023-08-11T13:27:14.035 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/adt7x10.ko 2023-08-11T13:27:14.036 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/amc6821.ko 2023-08-11T13:27:14.036 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/applesmc.ko 2023-08-11T13:27:14.036 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asb100.ko 2023-08-11T13:27:14.036 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asc7621.ko 2023-08-11T13:27:14.037 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-11T13:27:14.037 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/atxp1.ko 2023-08-11T13:27:14.037 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/coretemp.ko 2023-08-11T13:27:14.038 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-11T13:27:14.038 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/dme1737.ko 2023-08-11T13:27:14.038 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds1621.ko 2023-08-11T13:27:14.039 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ds620.ko 2023-08-11T13:27:14.039 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc1403.ko 2023-08-11T13:27:14.039 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/emc6w201.ko 2023-08-11T13:27:14.039 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71805f.ko 2023-08-11T13:27:14.040 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f71882fg.ko 2023-08-11T13:27:14.040 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/f75375s.ko 2023-08-11T13:27:14.040 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fam15h_power.ko 2023-08-11T13:27:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/fschmd.ko 2023-08-11T13:27:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g760a.ko 2023-08-11T13:27:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/g762.ko 2023-08-11T13:27:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl518sm.ko 2023-08-11T13:27:14.041 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/gl520sm.ko 2023-08-11T13:27:14.042 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-11T13:27:14.042 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5500_temp.ko 2023-08-11T13:27:14.042 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/i5k_amb.ko 2023-08-11T13:27:14.042 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmaem.ko 2023-08-11T13:27:14.043 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ibmpex.ko 2023-08-11T13:27:14.043 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina209.ko 2023-08-11T13:27:14.043 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ina2xx.ko 2023-08-11T13:27:14.043 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/it87.ko 2023-08-11T13:27:14.044 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k10temp.ko 2023-08-11T13:27:14.044 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/k8temp.ko 2023-08-11T13:27:14.044 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lineage-pem.ko 2023-08-11T13:27:14.045 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm63.ko 2023-08-11T13:27:14.045 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm73.ko 2023-08-11T13:27:14.045 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm75.ko 2023-08-11T13:27:14.045 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm77.ko 2023-08-11T13:27:14.046 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm78.ko 2023-08-11T13:27:14.046 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm80.ko 2023-08-11T13:27:14.046 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm83.ko 2023-08-11T13:27:14.047 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm85.ko 2023-08-11T13:27:14.047 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm87.ko 2023-08-11T13:27:14.047 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm90.ko 2023-08-11T13:27:14.047 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm92.ko 2023-08-11T13:27:14.048 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm93.ko 2023-08-11T13:27:14.048 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95234.ko 2023-08-11T13:27:14.048 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95241.ko 2023-08-11T13:27:14.048 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/lm95245.ko 2023-08-11T13:27:14.049 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc2945.ko 2023-08-11T13:27:14.049 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4151.ko 2023-08-11T13:27:14.049 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4215.ko 2023-08-11T13:27:14.050 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4222.ko 2023-08-11T13:27:14.050 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4245.ko 2023-08-11T13:27:14.050 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4260.ko 2023-08-11T13:27:14.050 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/ltc4261.ko 2023-08-11T13:27:14.051 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max16065.ko 2023-08-11T13:27:14.051 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1619.ko 2023-08-11T13:27:14.051 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max1668.ko 2023-08-11T13:27:14.051 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max197.ko 2023-08-11T13:27:14.052 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6639.ko 2023-08-11T13:27:14.052 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6650.ko 2023-08-11T13:27:14.052 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/max6697.ko 2023-08-11T13:27:14.053 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/mcp3021.ko 2023-08-11T13:27:14.053 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6683.ko 2023-08-11T13:27:14.053 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775-core.ko 2023-08-11T13:27:14.053 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct6775.ko 2023-08-11T13:27:14.053 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/nct7802.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87360.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pc87427.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pcf8591.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-11T13:27:14.054 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-11T13:27:14.055 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/powr1220.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5627.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch5636.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-11T13:27:14.056 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht15.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sht21.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/shtc1.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/sis5595.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47b397.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m1.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/smsc47m192.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/thmc50.ko 2023-08-11T13:27:14.057 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp102.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp103.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp401.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/tmp421.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via-cputemp.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/via686a.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt1211.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/vt8231.ko 2023-08-11T13:27:14.058 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627ehf.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83627hf.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83781d.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83791d.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83792d.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83793.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83795.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l785ts.ko 2023-08-11T13:27:14.059 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/hwmon/w83l786ng.ko 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-11T13:27:14.060 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-11T13:27:14.061 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-11T13:27:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-dev.ko 2023-08-11T13:27:14.063 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-mux.ko 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-smbus.ko 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/i2c/i2c-stub.ko 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-11T13:27:14.064 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_core.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4 2023-08-11T13:27:14.065 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-11T13:27:14.066 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp 2023-08-11T13:27:14.067 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/gameport.ko 2023-08-11T13:27:14.068 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/lightning.ko 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/gameport/ns558.ko 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/joydev.ko 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/appletouch.ko 2023-08-11T13:27:14.069 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/bcm5974.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/cyapatp.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/sermouse.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4 2023-08-11T13:27:14.070 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/altera_ps2.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/arc_ps2.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/serio/serio_raw.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/input/sparse-keymap.ko 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu 2023-08-11T13:27:14.071 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-blinkm.ko 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lm3530.ko 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-lp3944.ko 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/leds-ss4200.ko 2023-08-11T13:27:14.072 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-11T13:27:14.073 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/bcache/bcache.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-bio-prison.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache-smq.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-cache.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-crypt.ko 2023-08-11T13:27:14.074 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-delay.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-flakey.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-log-userspace.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-multipath.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-queue-length.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-raid.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-round-robin.ko 2023-08-11T13:27:14.075 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-service-time.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-switch.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-thin-pool.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/dm-verity.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/faulty.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/linear.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/multipath.ko 2023-08-11T13:27:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid0.ko 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid1.ko 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid10.ko 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/md/raid456.ko 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion 2023-08-11T13:27:14.077 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptbase.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptctl.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptfc.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptlan.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptsas.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptscsih.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/message/fusion/mptspi.ko 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd 2023-08-11T13:27:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_ich.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/lpc_sch.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/mfd-core.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/sm501.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/viperboard.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/vx855.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/mfd/wl1273-core.ko 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc 2023-08-11T13:27:14.079 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds9802als.ko 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/apds990x.ko 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/bh1770glc.ko 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/cb710/cb710.ko 2023-08-11T13:27:14.080 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/at24.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/eeprom/max6875.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/enclosure.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/hpilo.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29003.ko 2023-08-11T13:27:14.081 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/isl29020.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-me.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei-txe.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/mei/mei.ko 2023-08-11T13:27:14.082 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_7xx1.ko 2023-08-11T13:27:14.083 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tifm_core.ko 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/tsl2550.ko 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_balloon.ko 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/bonding/bonding.ko 2023-08-11T13:27:14.084 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/dummy.ko 2023-08-11T13:27:14.085 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/eql.ko 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-11T13:27:14.086 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390 2023-08-11T13:27:14.087 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-11T13:27:14.087 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-11T13:27:14.087 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-11T13:27:14.087 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-11T13:27:14.087 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-11T13:27:14.088 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-11T13:27:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom 2023-08-11T13:27:14.090 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-11T13:27:14.091 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-11T13:27:14.092 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco 2023-08-11T13:27:14.093 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-11T13:27:14.094 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/dnet.ko 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex 2023-08-11T13:27:14.095 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ethoc.ko 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/fealnx.ko 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-11T13:27:14.096 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-11T13:27:14.097 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-11T13:27:14.098 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/jme.ko 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-11T13:27:14.099 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom 2023-08-11T13:27:14.100 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-11T13:27:14.101 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-11T13:27:14.102 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-11T13:27:14.103 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan 2023-08-11T13:27:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-11T13:27:14.105 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-11T13:27:14.106 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-11T13:27:14.107 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/geneve.ko 2023-08-11T13:27:14.108 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ifb.ko 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan 2023-08-11T13:27:14.109 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvlan.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/macvtap.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-11T13:27:14.110 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/mii.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/net_failover.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/netconsole.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/nlmon.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy 2023-08-11T13:27:14.111 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/amd.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/bcm87xx.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/broadcom.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/cicada.ko 2023-08-11T13:27:14.112 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/davicom.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/dp83640.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/et1011c.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/icplus.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/lxt.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/marvell.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/micrel.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/national.ko 2023-08-11T13:27:14.113 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/phylink.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/qsemi.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/realtek.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/smsc.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/ste10Xp.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/phy/vitesse.ko 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp 2023-08-11T13:27:14.114 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_async.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppoe.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pppox.ko 2023-08-11T13:27:14.115 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/ppp/pptp.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slhc.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/slip/slip.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/sungem_phy.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tap.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team.ko 2023-08-11T13:27:14.116 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_random.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/tun.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/veth.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/virtio_net.ko 2023-08-11T13:27:14.117 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/vxlan/vxlan.ko 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/net/xen-netfront.ko 2023-08-11T13:27:14.118 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ntb/ntb.ko 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme-core.ko 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/nvme/host/nvme.ko 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport.ko 2023-08-11T13:27:14.119 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_cs.ko 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_pc.ko 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/parport/parport_serial.ko 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-11T13:27:14.120 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pci/xen-pcifront.ko 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/i82092.ko 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/pd6729.ko 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel 2023-08-11T13:27:14.121 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/acerhdf.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-11T13:27:14.122 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/hdaps.ko 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel 2023-08-11T13:27:14.123 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/intel_ips.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-11T13:27:14.124 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/platform/x86/wmi.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-11T13:27:14.125 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/clients/pps_parport.ko 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/pps/pps_core.ko 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp.ko 2023-08-11T13:27:14.126 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ptp/ptp_kvm.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-11T13:27:14.127 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-em3027.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-11T13:27:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-max6900.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-11T13:27:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-11T13:27:14.130 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/rtc/rtc-x1205.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-9xxx.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-sas.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/3w-xxxx.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/BusLogic.ko 2023-08-11T13:27:14.131 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/a100u2w.ko 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/advansys.ko 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-11T13:27:14.132 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/am53c974.ko 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/atp870u.ko 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bfa/bfa.ko 2023-08-11T13:27:14.133 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ch.ko 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-11T13:27:14.134 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dc395x.ko 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler 2023-08-11T13:27:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/dmx3191d.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/esp_scsi.ko 2023-08-11T13:27:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/fnic/fnic.ko 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hpsa.ko 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hptiop.ko 2023-08-11T13:27:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/hv_storvsc.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/initio.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ipr.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ips.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/isci/isci.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-11T13:27:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libfc/libfc.ko 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi.ko 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/libsas/libsas.ko 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-11T13:27:14.139 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid.ko 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas 2023-08-11T13:27:14.140 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/mvumi.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/pmcraid.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla1280.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-11T13:27:14.141 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/raid_class.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_debug.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-11T13:27:14.142 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/ses.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/st.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/stex.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-11T13:27:14.143 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/virtio_scsi.ko 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/scsi/wd719x.ko 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ssb/ssb.ko 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-11T13:27:14.144 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_file.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_iblock.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_mod.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_pscsi.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/target_core_user.ko 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc 2023-08-11T13:27:14.145 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-11T13:27:14.146 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-11T13:27:14.147 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-11T13:27:14.148 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_gsm.ko 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/n_hdlc.ko 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/nozomi.ko 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/arc_uart.ko 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm 2023-08-11T13:27:14.149 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/tty/synclink_gt.ko 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio 2023-08-11T13:27:14.150 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_aec.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_cif.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_pci_generic.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/uio/uio_sercos3.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-acm.ko 2023-08-11T13:27:14.151 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usblp.ko 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/class/usbtmc.ko 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/mdc800.ko 2023-08-11T13:27:14.152 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/image/microtek.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/adutux.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/appledisplay.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi26.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/emi62.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ezusb.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/idmouse.ko 2023-08-11T13:27:14.153 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/iowarrior.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/ldusb.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/legousbtower.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usb3503.ko 2023-08-11T13:27:14.154 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usblcd.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/uss720.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/misc/yurex.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/phy/phy-generic.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/aircable.ko 2023-08-11T13:27:14.155 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ark3116.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ch341.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cp210x.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cyberjack.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/empeg.ko 2023-08-11T13:27:14.156 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/io_ti.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipaq.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ipw.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ir-usb.ko 2023-08-11T13:27:14.157 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mct_u232.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7720.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/mos7840.ko 2023-08-11T13:27:14.158 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/navman.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/omninet.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/opticon.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/option.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/oti6858.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/pl2303.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcaux.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/qcserial.ko 2023-08-11T13:27:14.159 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/quatech2.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/safe_serial.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/sierra.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ssu100.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/symbolserial.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-11T13:27:14.160 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_debug.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/visor.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/whiteheat.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/uas.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-11T13:27:14.161 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-karma.ko 2023-08-11T13:27:14.162 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/storage/usb-storage.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-11T13:27:14.163 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-11T13:27:14.164 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost.ko 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_net.ko 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/vhost/vhost_scsi.ko 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight 2023-08-11T13:27:14.165 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/apple_bl.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/lcd.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_balloon.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_mmio.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci.ko 2023-08-11T13:27:14.166 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/virtio/virtio_ring.ko 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2482.ko 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/masters/ds2490.ko 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves 2023-08-11T13:27:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-11T13:27:14.168 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/w1/wire.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-11T13:27:14.169 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/hpwdt.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/i6300esb.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ib700wdt.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ibmasr.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-11T13:27:14.170 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/it87_wdt.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/machzwd.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/nv_tco.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/softdog.ko 2023-08-11T13:27:14.171 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/via_wdt.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/wdt_pci.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/watchdog/xen_wdt.ko 2023-08-11T13:27:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-evtchn.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntalloc.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-gntdev.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xen-privcmd.ko 2023-08-11T13:27:14.173 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/9p/9p.ko 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/affs/affs.ko 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs 2023-08-11T13:27:14.174 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/befs/befs.ko 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/binfmt_misc.ko 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/btrfs/btrfs.ko 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cachefiles/cachefiles.ko 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ceph/ceph.ko 2023-08-11T13:27:14.175 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/coda/coda.ko 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/cramfs/cramfs.ko 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/dlm/dlm.ko 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-11T13:27:14.176 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/f2fs/f2fs.ko 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/fat.ko 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/msdos.ko 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fat/vfat.ko 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache 2023-08-11T13:27:14.177 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fscache/fscache.ko 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/cuse.ko 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/fuse/fuse.ko 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/gfs2/gfs2.ko 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfs/hfs.ko 2023-08-11T13:27:14.178 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/hfsplus/hfsplus.ko 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/isofs/isofs.ko 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/jfs/jfs.ko 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/lockd/lockd.ko 2023-08-11T13:27:14.179 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/minix/minix.ko 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/netfs/netfs.ko 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout 2023-08-11T13:27:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfs.ko 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv3.ko 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs/nfsv4.ko 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common 2023-08-11T13:27:14.181 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/grace.ko 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfs_common/nfs_acl.ko 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nfsd/nfsd.ko 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nilfs2/nilfs2.ko 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-celtic.ko 2023-08-11T13:27:14.182 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-centeuro.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-croatian.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-cyrillic.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-gaelic.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-greek.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-iceland.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-inuit.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-roman.ko 2023-08-11T13:27:14.183 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-romanian.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/mac-turkish.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1250.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1251.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp1255.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp737.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp775.ko 2023-08-11T13:27:14.184 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp850.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp852.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp855.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp857.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp860.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp861.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp862.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp863.ko 2023-08-11T13:27:14.185 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp864.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp865.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp866.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp869.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp874.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp932.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp936.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp949.ko 2023-08-11T13:27:14.186 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_cp950.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_euc-jp.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-1.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-13.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-14.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-15.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-2.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-3.ko 2023-08-11T13:27:14.187 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-4.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-5.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-6.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-7.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_iso8859-9.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-r.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-ru.ko 2023-08-11T13:27:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_koi8-u.ko 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/nls/nls_utf8.ko 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs 2023-08-11T13:27:14.189 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/overlayfs/overlay.ko 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore 2023-08-11T13:27:14.190 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/pstore/ramoops.ko 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/reiserfs/reiserfs.ko 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/romfs/romfs.ko 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/client/cifs.ko 2023-08-11T13:27:14.191 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_arc4.ko 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/smb/common/cifs_md4.ko 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/squashfs/squashfs.ko 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/sysv/sysv.ko 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf 2023-08-11T13:27:14.192 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/udf/udf.ko 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/ufs/ufs.ko 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/fs/xfs/xfs.ko 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/rcu/rcutorture.ko 2023-08-11T13:27:14.193 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/torture.ko 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/asn1_encoder.ko 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc-itu-t.ko 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crc8.ko 2023-08-11T13:27:14.194 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libarc4.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/crypto/libdes.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/libcrc32c.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lru_cache.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4_compress.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/lz4/lz4hc_compress.ko 2023-08-11T13:27:14.195 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/math/cordic.ko 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/raid6/raid6_pq.ko 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_bm.ko 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_fsm.ko 2023-08-11T13:27:14.196 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/ts_kmp.ko 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/lib/zstd/zstd_compress.ko 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/mm/hwpoison-inject.ko 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan 2023-08-11T13:27:14.197 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/6lowpan.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_dest.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_fragment.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_hop.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_mobility.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_routing.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/6lowpan/nhc_udp.ko 2023-08-11T13:27:14.198 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/garp.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/mrp.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/p8022.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/psnap.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/802/stp.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/8021q/8021q.ko 2023-08-11T13:27:14.199 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet.ko 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_fd.ko 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_rdma.ko 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/9p/9pnet_virtio.ko 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/batman-adv/batman-adv.ko 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge 2023-08-11T13:27:14.200 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/br_netfilter.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/bridge.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-11T13:27:14.201 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-11T13:27:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-11T13:27:14.203 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/ebtables.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ceph/libceph.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/failover.ko 2023-08-11T13:27:14.204 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/core/pktgen.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_diag.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv4.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dccp/dccp_ipv6.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dns_resolver/dns_resolver.ko 2023-08-11T13:27:14.205 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/dsa_core.ko 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_brcm.ko 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_dsa.ko 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_none.ko 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/dsa/tag_trailer.ko 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154 2023-08-11T13:27:14.206 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ah4.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/esp4.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/fou.ko 2023-08-11T13:27:14.207 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/gre.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/inet_diag.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_gre.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_tunnel.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ip_vti.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipcomp.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/ipip.ko 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter 2023-08-11T13:27:14.208 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-11T13:27:14.209 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-11T13:27:14.210 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_bic.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_dctcp.ko 2023-08-11T13:27:14.211 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_diag.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_highspeed.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_htcp.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_hybla.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_illinois.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_lp.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_scalable.ko 2023-08-11T13:27:14.212 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_vegas.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_veno.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_westwood.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tcp_yeah.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/tunnel4.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_diag.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/udp_tunnel.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-11T13:27:14.213 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ah6.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/esp6.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/fou6.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_tunnel.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ip6_vti.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/ipcomp6.ko 2023-08-11T13:27:14.214 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-11T13:27:14.215 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-11T13:27:14.216 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-11T13:27:14.217 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/sit.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/tunnel6.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/key/af_key.ko 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp 2023-08-11T13:27:14.218 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_core.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_eth.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ip6.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_netlink.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/l2tp/l2tp_ppp.ko 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc 2023-08-11T13:27:14.219 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/llc/llc.ko 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mac802154/mac802154.ko 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/mpls/mpls_gso.ko 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set.ko 2023-08-11T13:27:14.220 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-11T13:27:14.221 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-11T13:27:14.222 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-11T13:27:14.223 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conncount.ko 2023-08-11T13:27:14.224 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-11T13:27:14.225 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table.ko 2023-08-11T13:27:14.226 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_log_syslog.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_irc.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_sip.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-11T13:27:14.227 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nf_tables.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_log.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_chain_nat.ko 2023-08-11T13:27:14.228 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_compat.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_ct.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fib.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_hash.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_limit.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_log.ko 2023-08-11T13:27:14.229 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_masq.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_nat.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_numgen.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_queue.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_quota.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_redir.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject.ko 2023-08-11T13:27:14.230 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_reject_inet.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_socket.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/nft_tunnel.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_AUDIT.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_CT.ko 2023-08-11T13:27:14.231 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_DSCP.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HL.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_HMARK.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LED.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_LOG.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NETMAP.ko 2023-08-11T13:27:14.232 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFLOG.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_RATEEST.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_SECMARK.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TEE.ko 2023-08-11T13:27:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TPROXY.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_TRACE.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_addrtype.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_bpf.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cgroup.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cluster.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_comment.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connbytes.ko 2023-08-11T13:27:14.234 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlabel.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connlimit.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_connmark.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_conntrack.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_cpu.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dccp.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_devgroup.ko 2023-08-11T13:27:14.235 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_dscp.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ecn.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_esp.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hashlimit.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_helper.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_hl.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipcomp.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_iprange.ko 2023-08-11T13:27:14.236 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_ipvs.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_l2tp.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_length.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_limit.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mac.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_mark.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_multiport.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nat.ko 2023-08-11T13:27:14.237 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_nfacct.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_osf.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_owner.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_physdev.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_pkttype.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_policy.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_quota.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_rateest.ko 2023-08-11T13:27:14.238 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_realm.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_recent.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_sctp.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_set.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_socket.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_state.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_statistic.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_string.ko 2023-08-11T13:27:14.239 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_tcpmss.ko 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_time.ko 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netfilter/xt_u32.ko 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/netlink/netlink_diag.ko 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/nsh/nsh.ko 2023-08-11T13:27:14.240 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/openvswitch.ko 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-geneve.ko 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-gre.ko 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/openvswitch/vport-vxlan.ko 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/packet/af_packet_diag.ko 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds 2023-08-11T13:27:14.241 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_rdma.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/rds/rds_tcp.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_csum.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_gact.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_ipt.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_mirred.ko 2023-08-11T13:27:14.242 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_nat.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_pedit.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_police.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_simple.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_skbedit.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/act_vlan.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_basic.ko 2023-08-11T13:27:14.243 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_bpf.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_flow.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_fw.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_route.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/cls_u32.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_cmp.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_ipset.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_meta.ko 2023-08-11T13:27:14.244 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_nbyte.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_text.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/em_u32.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_choke.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_codel.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_drr.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_fq.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_gred.ko 2023-08-11T13:27:14.245 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hfsc.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_hhf.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_htb.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_ingress.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_mqprio_lib.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_multiq.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_netem.ko 2023-08-11T13:27:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_pie.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_plug.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_prio.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_qfq.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_red.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfb.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_sfq.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_tbf.ko 2023-08-11T13:27:14.247 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sched/sch_teql.ko 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp.ko 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sctp/sctp_diag.ko 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-11T13:27:14.248 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/sunrpc.ko 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/diag.ko 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/tipc/tipc.ko 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix 2023-08-11T13:27:14.249 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/unix/unix_diag.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm 2023-08-11T13:27:14.250 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/security/keys/trusted-keys/trusted.ko 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt 2023-08-11T13:27:14.251 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/kernel/virt/lib/irqbypass.ko 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.alias.bin 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.bin 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.builtin.modinfo 2023-08-11T13:27:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.dep.bin 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.devname 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.order 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.softdep 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols 2023-08-11T13:27:14.253 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.5.0-rc4-ga7fb1265323d/modules.symbols.bin 2023-08-11T13:27:14.254 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:14.254 DEBUG:teuthology.orchestra.run.smithi003:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-11T13:27:14.282 INFO:teuthology.orchestra.run.smithi003.stdout:postinstall scriptlet (using /bin/sh): 2023-08-11T13:27:14.282 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d -a -r /boot/System.map-6.5.0-rc4-ga7fb1265323d ]; then 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:cp /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:cp /boot/System.map-6.5.0-rc4-ga7fb1265323d /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:rm -f /boot/vmlinuz-6.5.0-rc4-ga7fb1265323d /boot/System.map-6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/installkernel 6.5.0-rc4-ga7fb1265323d /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:rm -f /boot/.vmlinuz-6.5.0-rc4-ga7fb1265323d-rpm /boot/.System.map-6.5.0-rc4-ga7fb1265323d-rpm 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2023-08-11T13:27:14.283 INFO:teuthology.orchestra.run.smithi003.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:new-kernel-pkg --remove 6.5.0-rc4-ga7fb1265323d --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc4-ga7fb1265323d.img 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-install remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-11T13:27:14.284 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-11T13:27:14.285 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/update-bootloader --remove 6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:14.285 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2023-08-11T13:27:14.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2023-08-11T13:27:14.365 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2023-08-11T13:27:14.366 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2023-08-11T13:27:14.366 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.02 2023-08-11T13:27:14.366 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 123.el8 2023-08-11T13:27:14.366 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2023-08-11T13:27:14.366 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Group : System Environment/Base 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 9265450 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-08-11T13:27:14.367 INFO:teuthology.orchestra.run.smithi003.stdout:Relocations : (not relocatable) 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : Red Hat, Inc. 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : Red Hat, Inc. 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-11T13:27:14.368 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-11T13:27:14.369 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-11T13:27:14.369 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2023-08-11T13:27:14.369 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:27:14.369 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2023-08-11T13:27:14.370 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-11T13:27:14.370 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:14.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-11T13:27:15.940 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2023-08-11T13:27:17.924 INFO:teuthology.orchestra.run.smithi003.stderr:done 2023-08-11T13:27:17.926 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2023-08-11T13:27:17.944 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.6qXipy0nOz 2023-08-11T13:27:17.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6qXipy0nOz 2023-08-11T13:27:18.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.6qXipy0nOz 2023-08-11T13:27:18.164 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.6qXipy0nOz is 6KB 2023-08-11T13:27:18.212 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.6qXipy0nOz 2023-08-11T13:27:18.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2023-08-11T13:27:18.294 INFO:teuthology.orchestra.run.smithi003.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-08-11T13:27:18.294 INFO:teuthology.orchestra.run.smithi003.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-08-11T13:27:18.295 INFO:teuthology.orchestra.run.smithi003.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc4-ga7fb1265323d.conf 2023-08-11T13:27:18.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc4-ga7fb1265323d 2023-08-11T13:27:18.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2023-08-11T13:27:48.453 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-08-11T13:27:48.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:27:48.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:28:06.768 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-08-11T13:28:15.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:28:15.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:29:15.772 DEBUG:teuthology.orchestra.remote:timed out 2023-08-11T13:29:27.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:29:27.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:29:28.131 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-08-11T13:29:28.574 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:29:28.574 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc4-ga7fb1265323d"... 2023-08-11T13:29:28.574 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-08-11T13:29:28.617 INFO:teuthology.orchestra.run.smithi003.stdout:6.5.0-rc4-ga7fb1265323d 2023-08-11T13:29:28.617 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc4-ga7fb1265323d vs 6.5.0-rc4-ga7fb1265323d 2023-08-11T13:29:28.617 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-11T13:29:28.618 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-08-11T13:29:29.619 DEBUG:teuthology.parallel:result is None 2023-08-11T13:29:29.619 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-11T13:29:29.627 INFO:teuthology.task.internal:Creating test directory... 2023-08-11T13:29:29.627 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T13:29:29.632 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T13:29:29.635 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-11T13:29:29.674 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-11T13:29:29.682 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-11T13:29:29.691 INFO:teuthology.task.internal:Creating archive directory... 2023-08-11T13:29:29.692 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T13:29:29.720 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T13:29:29.723 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-11T13:29:29.831 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-11T13:29:29.847 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-11T13:29:29.848 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T13:29:29.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:29:29.867 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-11T13:29:29.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:29:29.887 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T13:29:29.891 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T13:29:29.911 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-11T13:29:29.957 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:29.975 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:29.990 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:30.006 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:30.009 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:30.041 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-11T13:29:30.045 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-11T13:29:30.057 INFO:teuthology.task.internal:Configuring sudo... 2023-08-11T13:29:30.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T13:29:30.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T13:29:30.065 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-11T13:29:30.127 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-11T13:29:30.138 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-11T13:29:30.139 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T13:29:30.143 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T13:29:30.146 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-11T13:29:30.181 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.361 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:30.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:30.505 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:29:30.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T13:29:30.574 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.677 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:30.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:30.808 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:29:30.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T13:29:30.879 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-11T13:29:30.993 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:31.038 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-11T13:29:31.082 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:29:31.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-11T13:29:31.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2023-08-11T13:29:31.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2023-08-11T13:29:31.158 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2023-08-11T13:29:31.227 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T13:29:31.251 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T13:29:31.266 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T13:29:31.693 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-11T13:29:31.704 INFO:teuthology.task.internal:Starting timer... 2023-08-11T13:29:31.704 INFO:teuthology.run_tasks:Running task pcp... 2023-08-11T13:29:31.764 INFO:teuthology.run_tasks:Running task selinux... 2023-08-11T13:29:31.776 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-08-11T13:29:31.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2023-08-11T13:29:31.897 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2023-08-11T13:29:31.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2023-08-11T13:29:31.969 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2023-08-11T13:29:31.970 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2023-08-11T13:29:32.034 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2023-08-11T13:29:32.036 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-11T13:29:32.036 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2023-08-11T13:29:32.100 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2023-08-11T13:29:32.100 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2023-08-11T13:29:32.126 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2023-08-11T13:29:32.127 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2023-08-11T13:29:32.156 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2023-08-11T13:29:32.156 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2023-08-11T13:29:32.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T13:29:32.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:29:32.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T13:29:32.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:29:32.263 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T13:29:32.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T13:29:32.292 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-11T13:29:32.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2023-08-11T13:29:32.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2023-08-11T13:29:32.391 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2023-08-11T13:29:32.419 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-11T13:29:32.429 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-11T13:29:32.429 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-11T13:29:32.457 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-11T13:29:32.459 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi103.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-11T13:42:02.356 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2023-08-11T13:42:02.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:42:02.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:42:02.447 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-08-11T13:42:02.567 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-08-11T13:42:02.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-08-11T13:42:02.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:42:02.636 DEBUG:teuthology.orchestra.run.smithi103:> true 2023-08-11T13:42:02.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-08-11T13:42:02.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-08-11T13:42:02.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:42:02.772 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-08-11T13:42:02.843 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-08-11T13:42:02.844 INFO:teuthology.run_tasks:Running task clock... 2023-08-11T13:42:02.855 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-11T13:42:02.855 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T13:42:02.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T13:42:02.861 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T13:42:02.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T13:42:02.863 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-11T13:42:02.864 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T13:42:02.894 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T13:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T13:42:02.917 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T13:42:02.932 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T13:42:02.935 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-11T13:42:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-11T13:42:02.978 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2023-08-11T13:42:02.991 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2023-08-11T13:42:03.006 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T13:42:03.013 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2023-08-11T13:42:03.019 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T13:42:03.025 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2023-08-11T13:42:03.040 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T13:42:03.054 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T13:42:03.066 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2023-08-11T13:42:03.082 INFO:teuthology.orchestra.run.smithi103.stderr:bash: ntpq: command not found 2023-08-11T13:42:03.086 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T13:42:03.086 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2023-08-11T13:42:03.086 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.087 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.087 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.087 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.095 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2023-08-11T13:42:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-11T13:42:03.137 INFO:teuthology.orchestra.run.smithi187.stderr:bash: ntpq: command not found 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.141 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.166 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-11T13:42:03.253 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-08-11T13:42:03.260 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T13:42:03.261 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-08-11T13:42:03.261 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.261 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.261 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.261 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T13:42:03.262 INFO:teuthology.run_tasks:Running task pexec... 2023-08-11T13:42:03.272 INFO:teuthology.task.pexec:Executing custom commands... 2023-08-11T13:42:03.272 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T13:42:03.273 DEBUG:teuthology.orchestra.run.smithi003:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T13:42:03.273 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi103.front.sepia.ceph.com 2023-08-11T13:42:03.274 DEBUG:teuthology.orchestra.run.smithi103:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T13:42:03.274 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi187.front.sepia.ceph.com 2023-08-11T13:42:03.274 DEBUG:teuthology.orchestra.run.smithi187:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-08-11T13:42:03.646 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:03.648 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:03.794 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:04.575 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:27 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:04.585 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:27 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:04.890 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:51 ago on Fri 11 Aug 2023 01:40:13 PM UTC. 2023-08-11T13:42:06.292 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-08-11T13:42:06.293 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-08-11T13:42:06.293 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout:Resetting modules: 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-08-11T13:42:06.294 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-08-11T13:42:06.295 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:06.312 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2023-08-11T13:42:06.314 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2023-08-11T13:42:06.314 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2023-08-11T13:42:06.314 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2023-08-11T13:42:06.314 INFO:teuthology.orchestra.run.smithi103.stdout:Resetting modules: 2023-08-11T13:42:06.314 INFO:teuthology.orchestra.run.smithi103.stdout: container-tools 2023-08-11T13:42:06.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:06.315 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2023-08-11T13:42:06.315 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2023-08-11T13:42:06.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:06.952 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-08-11T13:42:06.953 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout:Resetting modules: 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout: container-tools 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-08-11T13:42:06.954 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2023-08-11T13:42:06.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:07.259 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:29 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:07.276 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:29 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:08.237 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:54 ago on Fri 11 Aug 2023 01:40:13 PM UTC. 2023-08-11T13:42:10.257 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2023-08-11T13:42:10.331 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-08-11T13:42:10.728 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:10.813 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:10.819 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-08-11T13:42:11.450 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:11.508 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:34 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:11.648 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:34 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:12.440 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:58 ago on Fri 11 Aug 2023 01:40:13 PM UTC. 2023-08-11T13:42:13.219 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2023-08-11T13:42:13.311 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================================ 2023-08-11T13:42:13.311 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================================ 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout:Installing group/module packages: 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T13:42:13.312 INFO:teuthology.orchestra.run.smithi103.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout:Installing module profiles: 2023-08-11T13:42:13.313 INFO:teuthology.orchestra.run.smithi103.stdout: container-tools/common 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling module streams: 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout: container-tools rhel8 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================================ 2023-08-11T13:42:13.314 INFO:teuthology.orchestra.run.smithi103.stdout:Install 10 Packages 2023-08-11T13:42:13.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:13.315 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 18 M 2023-08-11T13:42:13.316 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 65 M 2023-08-11T13:42:13.316 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2023-08-11T13:42:13.379 INFO:teuthology.orchestra.run.smithi103.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-08-11T13:42:13.404 INFO:teuthology.orchestra.run.smithi103.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 276 kB/s | 25 kB 00:00 2023-08-11T13:42:13.429 INFO:teuthology.orchestra.run.smithi103.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 354 kB/s | 41 kB 00:00 2023-08-11T13:42:13.455 INFO:teuthology.orchestra.run.smithi103.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-08-11T13:42:13.496 INFO:teuthology.orchestra.run.smithi103.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-08-11T13:42:13.506 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-08-11T13:42:13.522 INFO:teuthology.orchestra.run.smithi103.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-08-11T13:42:13.547 INFO:teuthology.orchestra.run.smithi103.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-08-11T13:42:13.602 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-08-11T13:42:13.602 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-08-11T13:42:13.602 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-08-11T13:42:13.602 INFO:teuthology.orchestra.run.smithi187.stdout:Installing group/module packages: 2023-08-11T13:42:13.602 INFO:teuthology.orchestra.run.smithi187.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T13:42:13.603 INFO:teuthology.orchestra.run.smithi187.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T13:42:13.603 INFO:teuthology.orchestra.run.smithi187.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T13:42:13.603 INFO:teuthology.orchestra.run.smithi187.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T13:42:13.603 INFO:teuthology.orchestra.run.smithi187.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T13:42:13.603 INFO:teuthology.orchestra.run.smithi187.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T13:42:13.604 INFO:teuthology.orchestra.run.smithi187.stdout:Installing module profiles: 2023-08-11T13:42:13.605 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools/common 2023-08-11T13:42:13.605 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:13.605 INFO:teuthology.orchestra.run.smithi187.stdout:Enabling module streams: 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools rhel8 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout:Install 10 Packages 2023-08-11T13:42:13.606 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:13.607 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 18 M 2023-08-11T13:42:13.607 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 65 M 2023-08-11T13:42:13.607 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-08-11T13:42:13.647 INFO:teuthology.orchestra.run.smithi103.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-08-11T13:42:13.764 INFO:teuthology.orchestra.run.smithi103.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-08-11T13:42:13.856 INFO:teuthology.orchestra.run.smithi103.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-08-11T13:42:13.857 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:42:13.857 INFO:teuthology.orchestra.run.smithi103.stdout:Total 34 MB/s | 18 MB 00:00 2023-08-11T13:42:13.985 INFO:teuthology.orchestra.run.smithi187.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 247 kB/s | 94 kB 00:00 2023-08-11T13:42:14.006 INFO:teuthology.orchestra.run.smithi187.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 62 kB/s | 25 kB 00:00 2023-08-11T13:42:14.023 INFO:teuthology.orchestra.run.smithi187.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 97 kB/s | 41 kB 00:00 2023-08-11T13:42:14.040 INFO:teuthology.orchestra.run.smithi187.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 898 kB/s | 49 kB 00:00 2023-08-11T13:42:14.059 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2023-08-11T13:42:14.075 INFO:teuthology.orchestra.run.smithi187.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-08-11T13:42:14.079 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2023-08-11T13:42:14.079 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2023-08-11T13:42:14.099 INFO:teuthology.orchestra.run.smithi187.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-08-11T13:42:14.124 INFO:teuthology.orchestra.run.smithi187.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-08-11T13:42:14.224 INFO:teuthology.orchestra.run.smithi187.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-08-11T13:42:14.260 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2023-08-11T13:42:14.264 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2023-08-11T13:42:14.349 INFO:teuthology.orchestra.run.smithi187.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-08-11T13:42:14.433 INFO:teuthology.orchestra.run.smithi187.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-08-11T13:42:14.433 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:42:14.434 INFO:teuthology.orchestra.run.smithi187.stdout:Total 22 MB/s | 18 MB 00:00 2023-08-11T13:42:14.446 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-08-11T13:42:14.552 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================================ 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================================ 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout:Installing group/module packages: 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-08-11T13:42:14.553 INFO:teuthology.orchestra.run.smithi003.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-08-11T13:42:14.554 INFO:teuthology.orchestra.run.smithi003.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-08-11T13:42:14.554 INFO:teuthology.orchestra.run.smithi003.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-08-11T13:42:14.554 INFO:teuthology.orchestra.run.smithi003.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-08-11T13:42:14.554 INFO:teuthology.orchestra.run.smithi003.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-08-11T13:42:14.554 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-08-11T13:42:14.555 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-08-11T13:42:14.555 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-08-11T13:42:14.555 INFO:teuthology.orchestra.run.smithi003.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-08-11T13:42:14.555 INFO:teuthology.orchestra.run.smithi003.stdout:Installing module profiles: 2023-08-11T13:42:14.555 INFO:teuthology.orchestra.run.smithi003.stdout: container-tools/common 2023-08-11T13:42:14.556 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:14.556 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling module streams: 2023-08-11T13:42:14.556 INFO:teuthology.orchestra.run.smithi003.stdout: container-tools rhel8 2023-08-11T13:42:14.556 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:14.556 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-08-11T13:42:14.557 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================================================ 2023-08-11T13:42:14.557 INFO:teuthology.orchestra.run.smithi003.stdout:Install 10 Packages 2023-08-11T13:42:14.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:14.558 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 18 M 2023-08-11T13:42:14.558 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 65 M 2023-08-11T13:42:14.558 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-08-11T13:42:14.632 INFO:teuthology.orchestra.run.smithi003.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 537 kB/s | 41 kB 00:00 2023-08-11T13:42:14.633 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-08-11T13:42:14.652 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-08-11T13:42:14.652 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-08-11T13:42:14.657 INFO:teuthology.orchestra.run.smithi003.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 245 kB/s | 25 kB 00:00 2023-08-11T13:42:14.691 INFO:teuthology.orchestra.run.smithi003.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 844 kB/s | 49 kB 00:00 2023-08-11T13:42:14.716 INFO:teuthology.orchestra.run.smithi003.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 585 kB/s | 94 kB 00:00 2023-08-11T13:42:14.758 INFO:teuthology.orchestra.run.smithi003.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.1 MB/s | 524 kB 00:00 2023-08-11T13:42:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2023-08-11T13:42:14.784 INFO:teuthology.orchestra.run.smithi003.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-08-11T13:42:14.817 INFO:teuthology.orchestra.run.smithi003.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-08-11T13:42:14.832 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-08-11T13:42:14.836 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-08-11T13:42:14.943 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T13:42:14.951 INFO:teuthology.orchestra.run.smithi003.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 16 MB/s | 2.2 MB 00:00 2023-08-11T13:42:15.118 INFO:teuthology.orchestra.run.smithi003.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 21 MB/s | 7.0 MB 00:00 2023-08-11T13:42:15.138 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:15.210 INFO:teuthology.orchestra.run.smithi003.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-08-11T13:42:15.210 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:42:15.210 INFO:teuthology.orchestra.run.smithi003.stdout:Total 28 MB/s | 18 MB 00:00 2023-08-11T13:42:15.323 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T13:42:15.360 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-08-11T13:42:15.540 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T13:42:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-08-11T13:42:15.573 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T13:42:15.577 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-08-11T13:42:15.577 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-08-11T13:42:15.726 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:15.911 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T13:42:15.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T13:42:16.099 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-08-11T13:42:16.103 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-08-11T13:42:16.111 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T13:42:16.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T13:42:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-08-11T13:42:16.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T13:42:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-08-11T13:42:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:17.199 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T13:42:17.344 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-08-11T13:42:17.556 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-08-11T13:42:17.654 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T13:42:17.910 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T13:42:17.965 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-08-11T13:42:18.075 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T13:42:18.137 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T13:42:18.290 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T13:42:18.330 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:18.458 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T13:42:18.596 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T13:42:18.597 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T13:42:18.598 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T13:42:18.598 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T13:42:18.598 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T13:42:18.651 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-08-11T13:42:18.908 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:18.908 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T13:42:18.908 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:18.908 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T13:42:18.909 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T13:42:19.828 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T13:42:19.828 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:42 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-08-11T13:42:20.085 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T13:42:20.085 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:42 ago on Fri 11 Aug 2023 01:37:37 PM UTC. 2023-08-11T13:42:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-08-11T13:42:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-08-11T13:42:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:21.282 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-08-11T13:42:21.282 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-08-11T13:42:21.282 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-08-11T13:42:21.282 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-08-11T13:42:21.282 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-08-11T13:42:21.283 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-08-11T13:42:21.283 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-08-11T13:42:21.283 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-08-11T13:42:21.283 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-08-11T13:42:21.283 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-08-11T13:42:22.653 INFO:teuthology.orchestra.run.smithi103.stdout:Installed products updated. 2023-08-11T13:42:22.799 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-08-11T13:42:22.800 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:09 ago on Fri 11 Aug 2023 01:40:13 PM UTC. 2023-08-11T13:42:22.853 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2023-08-11T13:42:22.853 INFO:teuthology.orchestra.run.smithi103.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:22.853 INFO:teuthology.orchestra.run.smithi103.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T13:42:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T13:42:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T13:42:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:22.855 INFO:teuthology.orchestra.run.smithi103.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:22.855 INFO:teuthology.orchestra.run.smithi103.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:22.855 INFO:teuthology.orchestra.run.smithi103.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T13:42:22.855 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2023-08-11T13:42:23.019 INFO:teuthology.orchestra.run.smithi187.stdout:Installed products updated. 2023-08-11T13:42:23.063 DEBUG:teuthology.parallel:result is None 2023-08-11T13:42:23.232 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T13:42:23.233 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:42:23.234 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-08-11T13:42:23.422 DEBUG:teuthology.parallel:result is None 2023-08-11T13:42:25.588 INFO:teuthology.orchestra.run.smithi003.stdout:Installed products updated. 2023-08-11T13:42:25.813 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:25.814 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-08-11T13:42:25.814 INFO:teuthology.orchestra.run.smithi003.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:25.814 INFO:teuthology.orchestra.run.smithi003.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:25.814 INFO:teuthology.orchestra.run.smithi003.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:25.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-08-11T13:42:25.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-08-11T13:42:25.815 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-08-11T13:42:25.815 INFO:teuthology.orchestra.run.smithi003.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:25.816 INFO:teuthology.orchestra.run.smithi003.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-08-11T13:42:25.816 INFO:teuthology.orchestra.run.smithi003.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-08-11T13:42:25.816 INFO:teuthology.orchestra.run.smithi003.stdout: yajl-2.1.0-10.el8.x86_64 2023-08-11T13:42:25.817 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:42:25.817 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-08-11T13:42:26.033 DEBUG:teuthology.parallel:result is None 2023-08-11T13:42:26.033 INFO:teuthology.run_tasks:Running task install... 2023-08-11T13:42:26.041 DEBUG:teuthology.task.install:project ceph 2023-08-11T13:42:26.042 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'}} 2023-08-11T13:42:26.042 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc'} 2023-08-11T13:42:26.042 INFO:teuthology.task.install:Using flavor: default 2023-08-11T13:42:26.048 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-11T13:42:26.049 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-11T13:42:26.049 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T13:42:26.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:42:26.051 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T13:42:26.052 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:42:26.053 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'tag': None, 'wait_for_package': False} 2023-08-11T13:42:26.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:42:26.190 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T13:42:26.191 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T13:42:26.196 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T13:42:26.197 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T13:42:26.264 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/ 2023-08-11T13:42:26.264 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5437.g92684807 2023-08-11T13:42:26.355 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T13:42:26.356 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:42:26.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T13:42:26.361 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T13:42:26.361 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:42:26.362 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T13:42:26.368 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-11T13:42:26.368 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:42:26.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-11T13:42:26.390 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T13:42:26.390 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T13:42:26.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T13:42:26.395 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T13:42:26.425 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-11T13:42:26.426 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92684807f016ce045ac66e37fdde6f8c47857bcc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-11T13:42:26.457 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T13:42:26.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T13:42:26.534 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T13:42:26.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-11T13:42:26.537 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T13:42:26.599 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2023-08-11T13:42:26.601 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2023-08-11T13:42:26.604 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2023-08-11T13:42:26.605 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2023-08-11T13:42:26.633 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-11T13:42:26.701 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2023-08-11T13:42:26.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2023-08-11T13:42:26.996 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:27.036 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:27.185 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:27.366 INFO:teuthology.orchestra.run.smithi103.stdout:44 files removed 2023-08-11T13:42:27.404 INFO:teuthology.orchestra.run.smithi187.stdout:44 files removed 2023-08-11T13:42:27.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T13:42:27.462 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T13:42:27.565 INFO:teuthology.orchestra.run.smithi003.stdout:44 files removed 2023-08-11T13:42:27.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-11T13:42:27.778 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:27.880 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:28.099 INFO:teuthology.orchestra.run.smithi003.stdout:Updating Subscription Management repositories. 2023-08-11T13:42:28.667 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 205 kB/s | 81 kB 00:00 2023-08-11T13:42:28.850 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 160 kB/s | 81 kB 00:00 2023-08-11T13:42:29.089 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 161 kB/s | 81 kB 00:00 2023-08-11T13:42:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-08-11T13:42:29.635 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-08-11T13:42:30.014 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 11 kB/s | 9.4 kB 00:00 2023-08-11T13:42:30.029 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-08-11T13:42:30.281 INFO:teuthology.orchestra.run.smithi103.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-08-11T13:42:30.345 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2023-08-11T13:42:30.616 INFO:teuthology.orchestra.run.smithi187.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-08-11T13:42:30.714 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2023-08-11T13:42:31.065 INFO:teuthology.orchestra.run.smithi003.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-08-11T13:42:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 5.9 MB/s | 16 MB 00:02 2023-08-11T13:42:36.523 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-08-11T13:42:37.408 INFO:teuthology.orchestra.run.smithi103.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-08-11T13:42:43.417 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2023-08-11T13:42:45.609 INFO:teuthology.orchestra.run.smithi103.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 MB/s | 53 MB 00:00 2023-08-11T13:42:47.656 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 457 kB/s | 24 kB 00:00 2023-08-11T13:42:48.812 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-08-11T13:42:56.743 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:42:56.744 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:42:56.760 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T13:42:56.761 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T13:42:56.762 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T13:42:56.763 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T13:42:56.766 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T13:42:56.769 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T13:42:56.771 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T13:42:56.910 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T13:42:56.923 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T13:42:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T13:42:56.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T13:42:56.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T13:42:56.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T13:42:56.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T13:42:56.925 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T13:42:56.926 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T13:42:56.927 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T13:42:56.928 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2023-08-11T13:42:56.929 INFO:teuthology.orchestra.run.smithi103.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T13:42:56.930 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T13:42:56.931 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T13:42:56.932 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T13:42:56.933 INFO:teuthology.orchestra.run.smithi103.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T13:42:56.934 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T13:42:56.935 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T13:42:56.936 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T13:42:56.936 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T13:42:56.936 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T13:42:56.937 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T13:42:56.937 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T13:42:56.937 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T13:42:56.937 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T13:42:56.938 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T13:42:56.938 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T13:42:56.938 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T13:42:56.938 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T13:42:56.939 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T13:42:56.939 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T13:42:56.939 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:42:56.939 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T13:42:56.940 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T13:42:56.940 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T13:42:56.940 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T13:42:56.940 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T13:42:56.941 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T13:42:56.941 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T13:42:56.941 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T13:42:56.942 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T13:42:56.943 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T13:42:56.944 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T13:42:56.945 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling module streams: 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql 10 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2023-08-11T13:42:56.946 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-08-11T13:42:56.947 INFO:teuthology.orchestra.run.smithi103.stdout:Install 128 Packages 2023-08-11T13:42:56.947 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 3 Packages 2023-08-11T13:42:56.947 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:42:56.947 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 206 M 2023-08-11T13:42:56.948 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2023-08-11T13:42:57.402 INFO:teuthology.orchestra.run.smithi103.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-08-11T13:42:57.644 INFO:teuthology.orchestra.run.smithi103.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 3.5 MB/s | 860 kB 00:00 2023-08-11T13:42:57.694 INFO:teuthology.orchestra.run.smithi103.stdout:(3/131): ceph-immutable-object-cache-18.0.0-543 3.3 MB/s | 167 kB 00:00 2023-08-11T13:42:57.893 INFO:teuthology.orchestra.run.smithi003.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-08-11T13:42:57.987 INFO:teuthology.orchestra.run.smithi103.stdout:(4/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 7.9 MB/s | 2.3 MB 00:00 2023-08-11T13:42:58.070 INFO:teuthology.orchestra.run.smithi103.stdout:(5/131): ceph-base-18.0.0-5437.g92684807.el8.x8 6.8 MB/s | 5.4 MB 00:00 2023-08-11T13:42:58.187 INFO:teuthology.orchestra.run.smithi103.stdout:(6/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 7.8 MB/s | 1.6 MB 00:00 2023-08-11T13:42:58.479 INFO:teuthology.orchestra.run.smithi103.stdout:(7/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 11 MB/s | 4.6 MB 00:00 2023-08-11T13:42:59.579 INFO:teuthology.orchestra.run.smithi103.stdout:(8/131): ceph-radosgw-18.0.0-5437.g92684807.el8 12 MB/s | 14 MB 00:01 2023-08-11T13:42:59.788 INFO:teuthology.orchestra.run.smithi103.stdout:(9/131): ceph-osd-18.0.0-5437.g92684807.el8.x86 11 MB/s | 18 MB 00:01 2023-08-11T13:42:59.813 INFO:teuthology.orchestra.run.smithi103.stdout:(10/131): ceph-selinux-18.0.0-5437.g92684807.el 104 kB/s | 24 kB 00:00 2023-08-11T13:42:59.872 INFO:teuthology.orchestra.run.smithi103.stdout:(11/131): cephfs-mirror-18.0.0-5437.g92684807.e 4.4 MB/s | 261 kB 00:00 2023-08-11T13:42:59.922 INFO:teuthology.orchestra.run.smithi103.stdout:(12/131): libcephfs-devel-18.0.0-5437.g92684807 620 kB/s | 31 kB 00:00 2023-08-11T13:43:00.039 INFO:teuthology.orchestra.run.smithi103.stdout:(13/131): libcephfs2-18.0.0-5437.g92684807.el8. 6.6 MB/s | 783 kB 00:00 2023-08-11T13:43:00.106 INFO:teuthology.orchestra.run.smithi103.stdout:(14/131): libcephsqlite-18.0.0-5437.g92684807.e 2.0 MB/s | 139 kB 00:00 2023-08-11T13:43:00.165 INFO:teuthology.orchestra.run.smithi103.stdout:(15/131): librados-devel-18.0.0-5437.g92684807. 2.3 MB/s | 139 kB 00:00 2023-08-11T13:43:00.248 INFO:teuthology.orchestra.run.smithi103.stdout:(16/131): libradosstriper1-18.0.0-5437.g9268480 5.6 MB/s | 473 kB 00:00 2023-08-11T13:43:00.957 INFO:teuthology.orchestra.run.smithi103.stdout:(17/131): librgw2-18.0.0-5437.g92684807.el8.x86 9.6 MB/s | 6.8 MB 00:00 2023-08-11T13:43:01.016 INFO:teuthology.orchestra.run.smithi103.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 767 kB/s | 45 kB 00:00 2023-08-11T13:43:01.074 INFO:teuthology.orchestra.run.smithi103.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 2.1 MB/s | 125 kB 00:00 2023-08-11T13:43:01.141 INFO:teuthology.orchestra.run.smithi103.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 2.9 MB/s | 197 kB 00:00 2023-08-11T13:43:01.217 INFO:teuthology.orchestra.run.smithi103.stdout:(21/131): python3-rados-18.0.0-5437.g92684807.e 4.8 MB/s | 370 kB 00:00 2023-08-11T13:43:01.292 INFO:teuthology.orchestra.run.smithi103.stdout:(22/131): python3-rbd-18.0.0-5437.g92684807.el8 4.7 MB/s | 364 kB 00:00 2023-08-11T13:43:01.359 INFO:teuthology.orchestra.run.smithi103.stdout:(23/131): python3-rgw-18.0.0-5437.g92684807.el8 1.6 MB/s | 107 kB 00:00 2023-08-11T13:43:01.426 INFO:teuthology.orchestra.run.smithi103.stdout:(24/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.4 MB/s | 94 kB 00:00 2023-08-11T13:43:01.793 INFO:teuthology.orchestra.run.smithi103.stdout:(25/131): rbd-mirror-18.0.0-5437.g92684807.el8. 9.6 MB/s | 3.5 MB 00:00 2023-08-11T13:43:01.860 INFO:teuthology.orchestra.run.smithi103.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 2.6 MB/s | 179 kB 00:00 2023-08-11T13:43:01.918 INFO:teuthology.orchestra.run.smithi103.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 396 kB/s | 23 kB 00:00 2023-08-11T13:43:01.985 INFO:teuthology.orchestra.run.smithi103.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.9 MB/s | 131 kB 00:00 2023-08-11T13:43:02.194 INFO:teuthology.orchestra.run.smithi103.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5437.g92684 8.3 MB/s | 1.7 MB 00:00 2023-08-11T13:43:02.896 INFO:teuthology.orchestra.run.smithi103.stdout:(30/131): ceph-test-18.0.0-5437.g92684807.el8.x 15 MB/s | 48 MB 00:03 2023-08-11T13:43:02.945 INFO:teuthology.orchestra.run.smithi103.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5437.g92 4.9 MB/s | 241 kB 00:00 2023-08-11T13:43:03.071 INFO:teuthology.orchestra.run.smithi103.stdout:(32/131): ceph-mgr-diskprediction-local-18.0.0- 8.5 MB/s | 7.4 MB 00:00 2023-08-11T13:43:03.096 INFO:teuthology.orchestra.run.smithi103.stdout:(33/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 316 kB/s | 47 kB 00:00 2023-08-11T13:43:03.121 INFO:teuthology.orchestra.run.smithi103.stdout:(34/131): ceph-prometheus-alerts-18.0.0-5437.g9 291 kB/s | 15 kB 00:00 2023-08-11T13:43:03.146 INFO:teuthology.orchestra.run.smithi103.stdout:(35/131): ceph-volume-18.0.0-5437.g92684807.el8 5.1 MB/s | 261 kB 00:00 2023-08-11T13:43:03.172 INFO:teuthology.orchestra.run.smithi103.stdout:(36/131): cephadm-18.0.0-5437.g92684807.el8.noa 3.3 MB/s | 170 kB 00:00 2023-08-11T13:43:03.197 INFO:teuthology.orchestra.run.smithi103.stdout:(37/131): cephfs-top-18.0.0-5437.g92684807.el8. 358 kB/s | 18 kB 00:00 2023-08-11T13:43:03.506 INFO:teuthology.orchestra.run.smithi103.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 191 kB/s | 59 kB 00:00 2023-08-11T13:43:03.598 INFO:teuthology.orchestra.run.smithi103.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 720 kB/s | 306 kB 00:00 2023-08-11T13:43:03.623 INFO:teuthology.orchestra.run.smithi103.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 642 kB/s | 75 kB 00:00 2023-08-11T13:43:03.698 INFO:teuthology.orchestra.run.smithi103.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 581 kB/s | 44 kB 00:00 2023-08-11T13:43:03.840 INFO:teuthology.orchestra.run.smithi103.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-08-11T13:43:03.866 INFO:teuthology.orchestra.run.smithi103.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-08-11T13:43:03.916 INFO:teuthology.orchestra.run.smithi103.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 439 kB/s | 33 kB 00:00 2023-08-11T13:43:03.941 INFO:teuthology.orchestra.run.smithi103.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-08-11T13:43:04.041 INFO:teuthology.orchestra.run.smithi103.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-08-11T13:43:04.117 INFO:teuthology.orchestra.run.smithi103.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-08-11T13:43:04.142 INFO:teuthology.orchestra.run.smithi103.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 811 kB/s | 81 kB 00:00 2023-08-11T13:43:04.192 INFO:teuthology.orchestra.run.smithi103.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-08-11T13:43:04.218 INFO:teuthology.orchestra.run.smithi103.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2023-08-11T13:43:04.293 INFO:teuthology.orchestra.run.smithi103.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 651 kB/s | 49 kB 00:00 2023-08-11T13:43:04.552 INFO:teuthology.orchestra.run.smithi103.stdout:(52/131): ceph-common-18.0.0-5437.g92684807.el8 3.1 MB/s | 22 MB 00:07 2023-08-11T13:43:04.577 INFO:teuthology.orchestra.run.smithi103.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 207 kB/s | 59 kB 00:00 2023-08-11T13:43:04.752 INFO:teuthology.orchestra.run.smithi103.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-08-11T13:43:04.786 INFO:teuthology.orchestra.run.smithi103.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-08-11T13:43:04.854 INFO:teuthology.orchestra.run.smithi103.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 153 kB/s | 16 kB 00:00 2023-08-11T13:43:04.886 INFO:teuthology.orchestra.run.smithi103.stdout:(57/131): python3-natsort-7.1.1-2.el8.noarch.rp 180 kB/s | 60 kB 00:00 2023-08-11T13:43:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-08-11T13:43:04.962 INFO:teuthology.orchestra.run.smithi103.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-08-11T13:43:04.987 INFO:teuthology.orchestra.run.smithi103.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-08-11T13:43:05.013 INFO:teuthology.orchestra.run.smithi103.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-08-11T13:43:05.038 INFO:teuthology.orchestra.run.smithi103.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-08-11T13:43:05.055 INFO:teuthology.orchestra.run.smithi103.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 435 kB/s | 29 kB 00:00 2023-08-11T13:43:05.080 INFO:teuthology.orchestra.run.smithi103.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-08-11T13:43:05.147 INFO:teuthology.orchestra.run.smithi103.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 918 kB/s | 61 kB 00:00 2023-08-11T13:43:05.172 INFO:teuthology.orchestra.run.smithi103.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-08-11T13:43:05.223 INFO:teuthology.orchestra.run.smithi103.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-08-11T13:43:05.248 INFO:teuthology.orchestra.run.smithi103.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-08-11T13:43:05.265 INFO:teuthology.orchestra.run.smithi103.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 249 kB/s | 23 kB 00:00 2023-08-11T13:43:05.290 INFO:teuthology.orchestra.run.smithi103.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-08-11T13:43:05.382 INFO:teuthology.orchestra.run.smithi103.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-08-11T13:43:05.432 INFO:teuthology.orchestra.run.smithi103.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-08-11T13:43:05.483 INFO:teuthology.orchestra.run.smithi103.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-11T13:43:05.525 INFO:teuthology.orchestra.run.smithi103.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-08-11T13:43:05.558 INFO:teuthology.orchestra.run.smithi103.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-08-11T13:43:05.609 INFO:teuthology.orchestra.run.smithi103.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-08-11T13:43:05.767 INFO:teuthology.orchestra.run.smithi103.stdout:(77/131): python3-scipy-1.0.0-21.module+el8.5.0 27 MB/s | 14 MB 00:00 2023-08-11T13:43:05.801 INFO:teuthology.orchestra.run.smithi103.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.7 MB/s | 538 kB 00:00 2023-08-11T13:43:05.818 INFO:teuthology.orchestra.run.smithi103.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-08-11T13:43:05.843 INFO:teuthology.orchestra.run.smithi103.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-08-11T13:43:05.912 INFO:teuthology.orchestra.run.smithi103.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-08-11T13:43:05.952 INFO:teuthology.orchestra.run.smithi103.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2023-08-11T13:43:05.978 INFO:teuthology.orchestra.run.smithi103.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 395 kB/s | 27 kB 00:00 2023-08-11T13:43:06.028 INFO:teuthology.orchestra.run.smithi103.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 988 kB/s | 44 kB 00:00 2023-08-11T13:43:06.053 INFO:teuthology.orchestra.run.smithi103.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-08-11T13:43:06.104 INFO:teuthology.orchestra.run.smithi103.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-08-11T13:43:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:(87/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-08-11T13:43:06.205 INFO:teuthology.orchestra.run.smithi103.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-08-11T13:43:06.230 INFO:teuthology.orchestra.run.smithi103.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-08-11T13:43:06.255 INFO:teuthology.orchestra.run.smithi103.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-11T13:43:06.281 INFO:teuthology.orchestra.run.smithi103.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-08-11T13:43:06.306 INFO:teuthology.orchestra.run.smithi103.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-08-11T13:43:06.339 INFO:teuthology.orchestra.run.smithi103.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-08-11T13:43:06.406 INFO:teuthology.orchestra.run.smithi103.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 9.9 MB/s | 3.7 MB 00:00 2023-08-11T13:43:06.432 INFO:teuthology.orchestra.run.smithi103.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-08-11T13:43:06.474 INFO:teuthology.orchestra.run.smithi103.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-08-11T13:43:06.524 INFO:teuthology.orchestra.run.smithi103.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-08-11T13:43:06.608 INFO:teuthology.orchestra.run.smithi103.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-08-11T13:43:06.666 INFO:teuthology.orchestra.run.smithi103.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-08-11T13:43:06.700 INFO:teuthology.orchestra.run.smithi103.stdout:(100/131): postgresql-contrib-10.21-2.module+el 8.6 MB/s | 808 kB 00:00 2023-08-11T13:43:06.733 INFO:teuthology.orchestra.run.smithi103.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-08-11T13:43:06.767 INFO:teuthology.orchestra.run.smithi103.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-08-11T13:43:06.792 INFO:teuthology.orchestra.run.smithi103.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-08-11T13:43:06.817 INFO:teuthology.orchestra.run.smithi103.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 853 kB/s | 43 kB 00:00 2023-08-11T13:43:06.859 INFO:teuthology.orchestra.run.smithi103.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2023-08-11T13:43:06.909 INFO:teuthology.orchestra.run.smithi103.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 523 kB/s | 48 kB 00:00 2023-08-11T13:43:06.926 INFO:teuthology.orchestra.run.smithi103.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-08-11T13:43:06.952 INFO:teuthology.orchestra.run.smithi103.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-08-11T13:43:06.985 INFO:teuthology.orchestra.run.smithi103.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2023-08-11T13:43:07.010 INFO:teuthology.orchestra.run.smithi103.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-08-11T13:43:07.044 INFO:teuthology.orchestra.run.smithi103.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-08-11T13:43:07.069 INFO:teuthology.orchestra.run.smithi103.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-08-11T13:43:07.095 INFO:teuthology.orchestra.run.smithi103.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-08-11T13:43:07.162 INFO:teuthology.orchestra.run.smithi103.stdout:(114/131): postgresql-server-10.21-2.module+el8 7.9 MB/s | 5.1 MB 00:00 2023-08-11T13:43:07.187 INFO:teuthology.orchestra.run.smithi103.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 194 kB/s | 18 kB 00:00 2023-08-11T13:43:07.221 INFO:teuthology.orchestra.run.smithi103.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-08-11T13:43:07.246 INFO:teuthology.orchestra.run.smithi103.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-08-11T13:43:07.271 INFO:teuthology.orchestra.run.smithi103.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-08-11T13:43:07.297 INFO:teuthology.orchestra.run.smithi103.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-08-11T13:43:07.322 INFO:teuthology.orchestra.run.smithi103.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-08-11T13:43:07.347 INFO:teuthology.orchestra.run.smithi103.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-08-11T13:43:07.372 INFO:teuthology.orchestra.run.smithi103.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-11T13:43:07.398 INFO:teuthology.orchestra.run.smithi103.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-11T13:43:07.423 INFO:teuthology.orchestra.run.smithi103.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-08-11T13:43:07.457 INFO:teuthology.orchestra.run.smithi103.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-08-11T13:43:07.482 INFO:teuthology.orchestra.run.smithi103.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-08-11T13:43:07.507 INFO:teuthology.orchestra.run.smithi103.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-08-11T13:43:07.557 INFO:teuthology.orchestra.run.smithi103.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.9 MB/s | 2.3 MB 00:00 2023-08-11T13:43:07.616 INFO:teuthology.orchestra.run.smithi103.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-08-11T13:43:07.921 INFO:teuthology.orchestra.run.smithi103.stdout:(130/131): librados2-18.0.0-5437.g92684807.el8. 8.4 MB/s | 3.7 MB 00:00 2023-08-11T13:43:08.205 INFO:teuthology.orchestra.run.smithi103.stdout:(131/131): librbd1-18.0.0-5437.g92684807.el8.x8 5.2 MB/s | 3.6 MB 00:00 2023-08-11T13:43:08.210 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:43:08.210 INFO:teuthology.orchestra.run.smithi103.stdout:Total 18 MB/s | 206 MB 00:11 2023-08-11T13:43:09.059 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2023-08-11T13:43:09.180 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2023-08-11T13:43:09.180 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2023-08-11T13:43:10.203 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:43:10.204 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:43:10.220 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T13:43:10.221 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T13:43:10.222 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T13:43:10.224 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T13:43:10.226 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T13:43:10.230 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T13:43:10.231 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T13:43:10.344 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2023-08-11T13:43:10.348 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2023-08-11T13:43:10.378 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T13:43:10.392 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T13:43:10.393 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T13:43:10.394 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:43:10.395 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T13:43:10.396 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2023-08-11T13:43:10.397 INFO:teuthology.orchestra.run.smithi003.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T13:43:10.398 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T13:43:10.399 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T13:43:10.400 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T13:43:10.401 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T13:43:10.402 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T13:43:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T13:43:10.404 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T13:43:10.405 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T13:43:10.406 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T13:43:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T13:43:10.408 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T13:43:10.409 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling module streams: 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql 10 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:=========================================================================================================================== 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:Install 128 Packages 2023-08-11T13:43:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 3 Packages 2023-08-11T13:43:10.411 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:43:10.411 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 206 M 2023-08-11T13:43:10.411 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2023-08-11T13:43:10.618 INFO:teuthology.orchestra.run.smithi003.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 37 kB/s | 6.4 kB 00:00 2023-08-11T13:43:11.110 INFO:teuthology.orchestra.run.smithi003.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 1.7 MB/s | 860 kB 00:00 2023-08-11T13:43:11.227 INFO:teuthology.orchestra.run.smithi003.stdout:(3/131): ceph-immutable-object-cache-18.0.0-543 1.4 MB/s | 167 kB 00:00 2023-08-11T13:43:11.844 INFO:teuthology.orchestra.run.smithi003.stdout:(4/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 3.7 MB/s | 2.3 MB 00:00 2023-08-11T13:43:11.936 INFO:teuthology.orchestra.run.smithi003.stdout:(5/131): ceph-base-18.0.0-5437.g92684807.el8.x8 3.6 MB/s | 5.4 MB 00:01 2023-08-11T13:43:12.169 INFO:teuthology.orchestra.run.smithi003.stdout:(6/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 4.8 MB/s | 1.6 MB 00:00 2023-08-11T13:43:12.511 INFO:teuthology.orchestra.run.smithi003.stdout:(7/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 8.0 MB/s | 4.6 MB 00:00 2023-08-11T13:43:13.545 INFO:teuthology.orchestra.run.smithi003.stdout:(8/131): ceph-common-18.0.0-5437.g92684807.el8. 7.2 MB/s | 22 MB 00:03 2023-08-11T13:43:13.640 INFO:teuthology.orchestra.run.smithi003.stdout:(9/131): ceph-selinux-18.0.0-5437.g92684807.el8 257 kB/s | 24 kB 00:00 2023-08-11T13:43:13.953 INFO:teuthology.orchestra.run.smithi003.stdout:(10/131): ceph-radosgw-18.0.0-5437.g92684807.el 9.4 MB/s | 14 MB 00:01 2023-08-11T13:43:14.062 INFO:teuthology.orchestra.run.smithi003.stdout:(11/131): cephfs-mirror-18.0.0-5437.g92684807.e 2.4 MB/s | 261 kB 00:00 2023-08-11T13:43:14.246 INFO:teuthology.orchestra.run.smithi003.stdout:(12/131): ceph-osd-18.0.0-5437.g92684807.el8.x8 8.6 MB/s | 18 MB 00:02 2023-08-11T13:43:14.271 INFO:teuthology.orchestra.run.smithi003.stdout:(13/131): libcephfs-devel-18.0.0-5437.g92684807 149 kB/s | 31 kB 00:00 2023-08-11T13:43:14.346 INFO:teuthology.orchestra.run.smithi003.stdout:(14/131): libcephsqlite-18.0.0-5437.g92684807.e 1.8 MB/s | 139 kB 00:00 2023-08-11T13:43:14.388 INFO:teuthology.orchestra.run.smithi003.stdout:(15/131): libcephfs2-18.0.0-5437.g92684807.el8. 5.4 MB/s | 783 kB 00:00 2023-08-11T13:43:14.422 INFO:teuthology.orchestra.run.smithi003.stdout:(16/131): librados-devel-18.0.0-5437.g92684807. 1.8 MB/s | 139 kB 00:00 2023-08-11T13:43:14.480 INFO:teuthology.orchestra.run.smithi003.stdout:(17/131): libradosstriper1-18.0.0-5437.g9268480 5.0 MB/s | 473 kB 00:00 2023-08-11T13:43:14.589 INFO:teuthology.orchestra.run.smithi003.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 413 kB/s | 45 kB 00:00 2023-08-11T13:43:14.698 INFO:teuthology.orchestra.run.smithi003.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 1.1 MB/s | 125 kB 00:00 2023-08-11T13:43:14.806 INFO:teuthology.orchestra.run.smithi003.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 1.8 MB/s | 197 kB 00:00 2023-08-11T13:43:14.915 INFO:teuthology.orchestra.run.smithi003.stdout:(21/131): librgw2-18.0.0-5437.g92684807.el8.x86 14 MB/s | 6.8 MB 00:00 2023-08-11T13:43:14.948 INFO:teuthology.orchestra.run.smithi003.stdout:(22/131): python3-rados-18.0.0-5437.g92684807.e 2.6 MB/s | 370 kB 00:00 2023-08-11T13:43:15.031 INFO:teuthology.orchestra.run.smithi003.stdout:(23/131): python3-rbd-18.0.0-5437.g92684807.el8 3.1 MB/s | 364 kB 00:00 2023-08-11T13:43:15.065 INFO:teuthology.orchestra.run.smithi003.stdout:(24/131): python3-rgw-18.0.0-5437.g92684807.el8 913 kB/s | 107 kB 00:00 2023-08-11T13:43:15.116 INFO:teuthology.orchestra.run.smithi003.stdout:(25/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.1 MB/s | 94 kB 00:00 2023-08-11T13:43:15.216 INFO:teuthology.orchestra.run.smithi003.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 1.7 MB/s | 179 kB 00:00 2023-08-11T13:43:15.308 INFO:teuthology.orchestra.run.smithi003.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 252 kB/s | 23 kB 00:00 2023-08-11T13:43:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:(28/131): rbd-mirror-18.0.0-5437.g92684807.el8. 10 MB/s | 3.5 MB 00:00 2023-08-11T13:43:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.0 MB/s | 131 kB 00:00 2023-08-11T13:43:15.560 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2023-08-11T13:43:15.642 INFO:teuthology.orchestra.run.smithi003.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5437.g92684 7.5 MB/s | 1.7 MB 00:00 2023-08-11T13:43:15.740 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T13:43:15.751 INFO:teuthology.orchestra.run.smithi003.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5437.g92 2.2 MB/s | 241 kB 00:00 2023-08-11T13:43:15.851 INFO:teuthology.orchestra.run.smithi003.stdout:(32/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 474 kB/s | 47 kB 00:00 2023-08-11T13:43:15.898 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T13:43:15.977 INFO:teuthology.orchestra.run.smithi003.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5437.g9 116 kB/s | 15 kB 00:00 2023-08-11T13:43:16.077 INFO:teuthology.orchestra.run.smithi003.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-08-11T13:43:16.099 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-11T13:43:16.102 INFO:teuthology.orchestra.run.smithi003.stdout:(35/131): ceph-volume-18.0.0-5437.g92684807.el8 2.0 MB/s | 261 kB 00:00 2023-08-11T13:43:16.161 INFO:teuthology.orchestra.run.smithi003.stdout:(36/131): cephadm-18.0.0-5437.g92684807.el8.noa 2.0 MB/s | 170 kB 00:00 2023-08-11T13:43:16.186 INFO:teuthology.orchestra.run.smithi003.stdout:(37/131): cephfs-top-18.0.0-5437.g92684807.el8. 215 kB/s | 18 kB 00:00 2023-08-11T13:43:16.317 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T13:43:16.428 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T13:43:16.528 INFO:teuthology.orchestra.run.smithi003.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-08-11T13:43:16.687 INFO:teuthology.orchestra.run.smithi003.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 475 kB/s | 75 kB 00:00 2023-08-11T13:43:16.692 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T13:43:16.712 INFO:teuthology.orchestra.run.smithi003.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 556 kB/s | 306 kB 00:00 2023-08-11T13:43:16.787 INFO:teuthology.orchestra.run.smithi003.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 583 kB/s | 44 kB 00:00 2023-08-11T13:43:16.937 INFO:teuthology.orchestra.run.smithi003.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-08-11T13:43:16.971 INFO:teuthology.orchestra.run.smithi003.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-08-11T13:43:17.005 INFO:teuthology.orchestra.run.smithi003.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2023-08-11T13:43:17.038 INFO:teuthology.orchestra.run.smithi003.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-08-11T13:43:17.095 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T13:43:17.139 INFO:teuthology.orchestra.run.smithi003.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-08-11T13:43:17.222 INFO:teuthology.orchestra.run.smithi003.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 975 kB/s | 81 kB 00:00 2023-08-11T13:43:17.306 INFO:teuthology.orchestra.run.smithi003.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-08-11T13:43:17.309 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:43:17.362 INFO:teuthology.orchestra.run.smithi003.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 79 kB/s | 11 kB 00:00 2023-08-11T13:43:17.523 INFO:teuthology.orchestra.run.smithi003.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 81 kB/s | 18 kB 00:00 2023-08-11T13:43:17.607 INFO:teuthology.orchestra.run.smithi003.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 584 kB/s | 49 kB 00:00 2023-08-11T13:43:17.691 INFO:teuthology.orchestra.run.smithi003.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 707 kB/s | 59 kB 00:00 2023-08-11T13:43:17.701 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:43:17.774 INFO:teuthology.orchestra.run.smithi003.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 725 kB/s | 60 kB 00:00 2023-08-11T13:43:17.909 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:43:17.950 INFO:teuthology.orchestra.run.smithi003.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-08-11T13:43:17.975 INFO:teuthology.orchestra.run.smithi003.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-08-11T13:43:18.035 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:43:18.259 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:43:18.360 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:43:18.451 INFO:teuthology.orchestra.run.smithi003.stdout:(56/131): ceph-test-18.0.0-5437.g92684807.el8.x 10 MB/s | 48 MB 00:04 2023-08-11T13:43:18.476 INFO:teuthology.orchestra.run.smithi003.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 30 kB/s | 16 kB 00:00 2023-08-11T13:43:18.501 INFO:teuthology.orchestra.run.smithi003.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 64 kB/s | 34 kB 00:00 2023-08-11T13:43:18.559 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T13:43:18.635 INFO:teuthology.orchestra.run.smithi003.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-08-11T13:43:18.735 INFO:teuthology.orchestra.run.smithi003.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 241 kB/s | 24 kB 00:00 2023-08-11T13:43:18.795 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T13:43:18.827 INFO:teuthology.orchestra.run.smithi003.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 523 kB/s | 196 kB 00:00 2023-08-11T13:43:18.852 INFO:teuthology.orchestra.run.smithi003.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 251 kB/s | 29 kB 00:00 2023-08-11T13:43:18.877 INFO:teuthology.orchestra.run.smithi003.stdout:(63/131): python3-rsa-4.9-2.el8.noarch.rpm 160 kB/s | 64 kB 00:00 2023-08-11T13:43:18.928 INFO:teuthology.orchestra.run.smithi003.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 367 kB/s | 27 kB 00:00 2023-08-11T13:43:19.003 INFO:teuthology.orchestra.run.smithi003.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 818 kB/s | 61 kB 00:00 2023-08-11T13:43:19.012 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T13:43:19.062 INFO:teuthology.orchestra.run.smithi003.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-08-11T13:43:19.087 INFO:teuthology.orchestra.run.smithi003.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-08-11T13:43:19.112 INFO:teuthology.orchestra.run.smithi003.stdout:(68/131): python3-waitress-1.4.3-1.el8.noarch.r 985 kB/s | 255 kB 00:00 2023-08-11T13:43:19.137 INFO:teuthology.orchestra.run.smithi003.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-11T13:43:19.162 INFO:teuthology.orchestra.run.smithi003.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-08-11T13:43:19.263 INFO:teuthology.orchestra.run.smithi003.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-08-11T13:43:19.313 INFO:teuthology.orchestra.run.smithi003.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-08-11T13:43:19.364 INFO:teuthology.orchestra.run.smithi003.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-11T13:43:19.414 INFO:teuthology.orchestra.run.smithi003.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.4 MB/s | 323 kB 00:00 2023-08-11T13:43:19.455 INFO:teuthology.orchestra.run.smithi003.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-08-11T13:43:19.472 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T13:43:19.514 INFO:teuthology.orchestra.run.smithi003.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-08-11T13:43:19.573 INFO:teuthology.orchestra.run.smithi003.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-08-11T13:43:19.615 INFO:teuthology.orchestra.run.smithi003.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-08-11T13:43:19.665 INFO:teuthology.orchestra.run.smithi003.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-08-11T13:43:19.712 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T13:43:19.757 INFO:teuthology.orchestra.run.smithi003.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-08-11T13:43:19.858 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:43:19.945 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:43:19.949 INFO:teuthology.orchestra.run.smithi003.stdout:(81/131): python3-scipy-1.0.0-21.module+el8.5.0 17 MB/s | 14 MB 00:00 2023-08-11T13:43:19.991 INFO:teuthology.orchestra.run.smithi003.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-08-11T13:43:20.016 INFO:teuthology.orchestra.run.smithi003.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 396 kB/s | 27 kB 00:00 2023-08-11T13:43:20.067 INFO:teuthology.orchestra.run.smithi003.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-08-11T13:43:20.100 INFO:teuthology.orchestra.run.smithi003.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-08-11T13:43:20.109 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:43:20.182 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:43:20.186 INFO:teuthology.orchestra.run.smithi003.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 5.2 MB/s | 457 kB 00:00 2023-08-11T13:43:20.234 INFO:teuthology.orchestra.run.smithi003.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.6 MB/s | 225 kB 00:00 2023-08-11T13:43:20.310 INFO:teuthology.orchestra.run.smithi003.stdout:(88/131): python3-babel-2.5.1-7.el8.noarch.rpm 8.7 MB/s | 4.8 MB 00:00 2023-08-11T13:43:20.335 INFO:teuthology.orchestra.run.smithi003.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2023-08-11T13:43:20.350 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T13:43:20.360 INFO:teuthology.orchestra.run.smithi003.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-08-11T13:43:20.385 INFO:teuthology.orchestra.run.smithi003.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-08-11T13:43:20.411 INFO:teuthology.orchestra.run.smithi003.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-08-11T13:43:20.461 INFO:teuthology.orchestra.run.smithi003.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-08-11T13:43:20.487 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T13:43:20.544 INFO:teuthology.orchestra.run.smithi003.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.8 MB/s | 3.7 MB 00:00 2023-08-11T13:43:20.578 INFO:teuthology.orchestra.run.smithi003.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.6 MB/s | 198 kB 00:00 2023-08-11T13:43:20.648 INFO:teuthology.orchestra.run.smithi003.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-08-11T13:43:20.650 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T13:43:20.712 INFO:teuthology.orchestra.run.smithi003.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.4 MB/s | 354 kB 00:00 2023-08-11T13:43:20.829 INFO:teuthology.orchestra.run.smithi003.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-08-11T13:43:20.913 INFO:teuthology.orchestra.run.smithi003.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-08-11T13:43:20.954 INFO:teuthology.orchestra.run.smithi003.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-08-11T13:43:20.996 INFO:teuthology.orchestra.run.smithi003.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-08-11T13:43:21.047 INFO:teuthology.orchestra.run.smithi003.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-08-11T13:43:21.064 INFO:teuthology.orchestra.run.smithi003.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-08-11T13:43:21.089 INFO:teuthology.orchestra.run.smithi003.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-08-11T13:43:21.156 INFO:teuthology.orchestra.run.smithi003.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-08-11T13:43:21.181 INFO:teuthology.orchestra.run.smithi003.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 523 kB/s | 48 kB 00:00 2023-08-11T13:43:21.206 INFO:teuthology.orchestra.run.smithi003.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-08-11T13:43:21.232 INFO:teuthology.orchestra.run.smithi003.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-08-11T13:43:21.257 INFO:teuthology.orchestra.run.smithi003.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-08-11T13:43:21.268 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T13:43:21.290 INFO:teuthology.orchestra.run.smithi003.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-08-11T13:43:21.358 INFO:teuthology.orchestra.run.smithi003.stdout:(111/131): postgresql-server-10.21-2.module+el8 7.8 MB/s | 5.1 MB 00:00 2023-08-11T13:43:21.375 INFO:teuthology.orchestra.run.smithi003.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 223 kB/s | 19 kB 00:00 2023-08-11T13:43:21.400 INFO:teuthology.orchestra.run.smithi003.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-08-11T13:43:21.442 INFO:teuthology.orchestra.run.smithi003.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-08-11T13:43:21.459 INFO:teuthology.orchestra.run.smithi003.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 307 kB/s | 18 kB 00:00 2023-08-11T13:43:21.467 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T13:43:21.501 INFO:teuthology.orchestra.run.smithi003.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.3 MB/s | 559 kB 00:00 2023-08-11T13:43:21.534 INFO:teuthology.orchestra.run.smithi003.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-08-11T13:43:21.560 INFO:teuthology.orchestra.run.smithi003.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-08-11T13:43:21.627 INFO:teuthology.orchestra.run.smithi003.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.1 MB/s | 72 kB 00:00 2023-08-11T13:43:21.652 INFO:teuthology.orchestra.run.smithi003.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 904 kB/s | 83 kB 00:00 2023-08-11T13:43:21.677 INFO:teuthology.orchestra.run.smithi003.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-08-11T13:43:21.702 INFO:teuthology.orchestra.run.smithi003.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-11T13:43:21.728 INFO:teuthology.orchestra.run.smithi003.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-11T13:43:21.753 INFO:teuthology.orchestra.run.smithi003.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-08-11T13:43:21.803 INFO:teuthology.orchestra.run.smithi003.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-08-11T13:43:21.820 INFO:teuthology.orchestra.run.smithi003.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-08-11T13:43:21.846 INFO:teuthology.orchestra.run.smithi003.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-08-11T13:43:21.888 INFO:teuthology.orchestra.run.smithi003.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.0 MB/s | 2.3 MB 00:00 2023-08-11T13:43:21.946 INFO:teuthology.orchestra.run.smithi003.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-08-11T13:43:22.069 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T13:43:22.291 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T13:43:22.371 INFO:teuthology.orchestra.run.smithi003.stdout:(130/131): librbd1-18.0.0-5437.g92684807.el8.x8 6.9 MB/s | 3.6 MB 00:00 2023-08-11T13:43:22.430 INFO:teuthology.orchestra.run.smithi003.stdout:(131/131): librados2-18.0.0-5437.g92684807.el8. 6.0 MB/s | 3.7 MB 00:00 2023-08-11T13:43:22.434 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:43:22.434 INFO:teuthology.orchestra.run.smithi003.stdout:Total 17 MB/s | 206 MB 00:12 2023-08-11T13:43:22.991 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T13:43:23.251 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-11T13:43:23.501 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-11T13:43:23.728 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-11T13:43:23.914 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-11T13:43:24.108 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2023-08-11T13:43:24.138 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-11T13:43:24.255 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2023-08-11T13:43:24.255 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2023-08-11T13:43:24.600 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T13:43:24.773 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T13:43:24.895 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T13:43:25.819 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T13:43:25.864 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2023-08-11T13:43:25.869 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2023-08-11T13:43:26.123 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-11T13:43:26.427 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-11T13:43:26.715 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-11T13:43:26.924 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T13:43:28.693 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T13:43:29.008 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-11T13:43:29.219 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T13:43:29.418 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T13:43:29.639 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T13:43:29.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T13:43:30.014 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-11T13:43:30.192 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-11T13:43:30.393 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-11T13:43:30.561 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-11T13:43:30.807 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-11T13:43:31.005 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-11T13:43:31.356 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2023-08-11T13:43:31.379 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-11T13:43:31.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T13:43:31.694 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-11T13:43:31.883 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-11T13:43:31.919 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T13:43:32.101 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-11T13:43:32.155 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-11T13:43:32.353 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-11T13:43:32.383 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T13:43:32.534 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-11T13:43:32.690 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-11T13:43:32.733 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T13:43:32.906 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-11T13:43:33.100 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-11T13:43:33.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-11T13:43:33.312 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-11T13:43:33.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-11T13:43:33.668 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-11T13:43:33.717 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:43:33.980 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-11T13:43:34.207 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:43:34.228 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-11T13:43:34.408 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:43:34.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-11T13:43:34.551 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:43:34.747 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-11T13:43:34.833 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:43:34.906 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-11T13:43:34.951 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:43:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-11T13:43:35.197 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T13:43:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T13:43:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T13:43:35.461 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T13:43:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-11T13:43:35.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T13:43:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-11T13:43:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T13:43:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T13:43:36.144 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T13:43:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-11T13:43:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T13:43:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-11T13:43:36.591 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:43:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T13:43:36.707 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:43:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T13:43:36.866 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:43:36.983 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:43:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T13:43:37.124 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T13:43:37.302 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-11T13:43:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T13:43:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T13:43:37.516 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T13:43:37.660 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T13:43:37.867 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T13:43:38.053 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T13:43:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T13:43:38.262 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-11T13:43:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T13:43:38.424 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T13:43:38.563 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T13:43:38.820 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T13:43:39.044 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T13:43:39.174 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-11T13:43:39.245 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T13:43:39.374 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T13:43:39.495 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T13:43:39.755 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T13:43:39.930 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T13:43:40.157 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T13:43:40.274 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-11T13:43:40.574 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-11T13:43:40.824 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-11T13:43:41.048 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-11T13:43:41.234 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-11T13:43:41.292 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T13:43:41.498 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-11T13:43:41.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T13:43:41.749 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T13:43:41.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T13:43:41.931 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T13:43:42.080 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T13:43:42.183 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T13:43:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T13:43:42.671 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T13:43:43.209 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T13:43:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T13:43:43.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-11T13:43:43.656 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:43:43.713 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:43:43.877 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-11T13:43:43.923 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T13:43:44.195 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-11T13:43:44.390 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T13:43:46.601 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-11T13:43:46.975 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-11T13:43:47.214 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T13:43:47.456 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T13:43:47.556 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:43:47.657 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T13:43:47.781 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:43:47.911 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T13:43:48.065 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-11T13:43:48.252 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-11T13:43:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:43:48.501 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-11T13:43:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:43:48.680 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-11T13:43:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:43:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:43:48.975 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-11T13:43:49.209 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-11T13:43:49.685 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-11T13:43:50.015 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-11T13:43:50.205 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-11T13:43:50.472 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-11T13:43:50.743 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-11T13:43:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-11T13:43:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-11T13:43:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-11T13:43:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-11T13:43:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-11T13:43:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-11T13:43:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-11T13:43:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-11T13:43:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-11T13:43:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-11T13:43:53.657 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-11T13:43:53.827 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-11T13:43:54.046 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T13:43:54.140 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-11T13:43:54.393 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-11T13:43:54.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-11T13:43:54.772 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T13:43:54.877 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-11T13:43:55.134 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-11T13:43:55.326 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-11T13:43:55.559 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T13:43:55.744 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T13:43:56.052 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T13:43:56.281 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T13:43:56.477 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T13:43:56.645 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T13:43:56.866 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T13:43:57.192 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T13:43:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T13:43:57.651 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T13:43:57.863 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T13:43:58.093 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T13:43:58.374 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T13:43:58.554 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T13:43:58.853 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T13:43:59.065 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T13:43:59.267 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T13:43:59.509 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T13:44:01.097 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T13:44:01.360 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T13:44:01.646 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T13:44:02.031 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T13:44:02.330 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T13:44:02.571 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T13:44:03.486 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T13:44:04.320 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:44:04.449 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:44:04.637 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T13:44:07.565 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /dev 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /run 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2023-08-11T13:44:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2023-08-11T13:44:07.567 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2023-08-11T13:44:07.567 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /dev 2023-08-11T13:44:07.567 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:44:07.919 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:44:09.495 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:44:09.697 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:44:09.917 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:44:10.119 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:44:10.198 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:44:10.403 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:44:10.574 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:44:10.769 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:44:10.786 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:44:11.133 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:44:11.408 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:44:11.568 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:44:11.719 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:44:12.429 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:44:12.633 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:44:12.775 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T13:44:12.959 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T13:44:13.151 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:44:13.369 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:44:13.672 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:44:14.635 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:44:14.877 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:44:14.969 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:44:15.309 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T13:44:15.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:44:15.561 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:44:15.718 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:44:15.729 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:44:17.050 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T13:44:17.234 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:44:17.268 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:44:17.419 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:44:17.468 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:44:17.635 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:44:17.986 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:44:18.143 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:44:22.726 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:44:23.039 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T13:44:23.183 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T13:44:23.384 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T13:44:24.271 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-11T13:44:24.583 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-11T13:44:24.753 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 119/134 2023-08-11T13:44:24.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T13:44:25.171 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T13:44:25.326 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T13:44:27.523 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:27.808 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:27.826 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:28.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T13:44:28.324 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T13:44:28.631 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T13:44:28.918 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T13:44:29.100 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:44:29.108 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:44:29.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-11T13:44:29.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T13:44:29.587 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T13:44:29.783 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T13:44:29.795 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T13:44:29.796 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:44:30.002 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:44:30.003 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:44:30.263 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:44:30.473 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:44:30.975 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:44:30.975 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T13:44:30.975 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T13:44:30.976 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T13:44:30.976 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T13:44:30.976 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T13:44:30.976 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:44:30.976 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T13:44:30.977 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T13:44:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T13:44:30.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T13:44:30.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T13:44:30.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T13:44:30.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T13:44:30.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T13:44:30.980 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T13:44:30.981 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T13:44:30.982 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T13:44:30.983 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T13:44:30.984 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T13:44:30.984 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T13:44:30.984 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T13:44:30.984 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T13:44:30.984 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T13:44:30.985 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T13:44:30.985 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T13:44:30.985 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T13:44:30.986 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-11T13:44:30.987 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-11T13:44:30.988 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-11T13:44:30.989 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-11T13:44:30.990 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-11T13:44:30.991 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-11T13:44:30.992 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-11T13:44:30.993 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-11T13:44:30.994 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:44:30.995 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T13:44:32.936 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /run 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2023-08-11T13:44:32.937 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2023-08-11T13:44:32.938 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2023-08-11T13:44:32.938 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /dev 2023-08-11T13:44:32.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:44:33.374 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:44:33.428 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:44:33.428 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:48 ago on Fri 11 Aug 2023 01:42:45 PM UTC. 2023-08-11T13:44:35.241 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:44:35.404 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:44:35.691 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:44:35.866 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:44:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:Installed products updated. 2023-08-11T13:44:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:44:36.172 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2023-08-11T13:44:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T13:44:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.172 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T13:44:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: jq-1.6-3.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T13:44:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T13:44:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:44:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T13:44:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T13:44:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:44:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T13:44:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T13:44:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T13:44:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T13:44:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T13:44:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T13:44:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:44:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:44:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:44:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T13:44:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T13:44:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T13:44:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T13:44:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T13:44:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T13:44:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T13:44:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T13:44:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T13:44:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T13:44:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T13:44:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T13:44:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T13:44:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:44:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T13:44:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T13:44:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T13:44:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T13:44:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T13:44:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T13:44:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-08-11T13:44:36.195 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2023-08-11T13:44:36.387 DEBUG:teuthology.parallel:result is None 2023-08-11T13:44:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:44:36.666 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:44:36.708 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:44:39.598 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:44:39.782 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:44:39.940 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T13:44:40.156 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T13:44:40.339 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:44:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:44:40.845 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:44:42.037 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:44:42.264 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:44:42.401 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:44:42.765 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T13:44:43.003 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:44:43.070 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:44:43.306 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:44:43.335 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:44:44.867 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T13:44:45.240 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:44:45.305 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:44:45.472 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:44:45.550 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:44:45.764 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:44:46.210 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:44:46.356 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:44:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:44:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T13:44:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T13:44:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-11T13:44:54.023 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-11T13:44:54.350 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-11T13:44:54.569 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 119/134 2023-08-11T13:44:54.768 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T13:44:55.002 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T13:44:55.155 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T13:44:58.615 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:58.871 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:58.909 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:44:59.180 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T13:44:59.415 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T13:44:59.822 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T13:45:00.131 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T13:45:00.295 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:45:00.318 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:45:00.543 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-11T13:45:00.711 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T13:45:00.828 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-11T13:45:01.045 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T13:45:01.074 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-11T13:45:01.074 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:45:01.293 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:45:01.293 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:45:01.602 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:45:01.823 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:45:02.135 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:45:02.136 INFO:teuthology.orchestra.run.smithi003.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-11T13:45:02.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:45:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T13:45:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T13:45:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T13:45:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T13:45:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T13:45:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:45:02.788 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:45:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:45:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T13:45:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T13:45:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T13:45:02.789 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T13:45:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T13:45:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T13:45:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T13:45:02.790 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T13:45:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T13:45:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T13:45:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T13:45:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T13:45:02.791 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T13:45:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T13:45:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T13:45:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T13:45:02.792 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T13:45:02.793 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T13:45:02.804 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T13:45:02.805 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T13:45:02.806 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T13:45:02.807 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T13:45:02.808 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T13:45:02.809 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-11T13:45:02.810 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-11T13:45:02.811 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-11T13:45:02.812 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-11T13:45:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-11T13:45:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-11T13:45:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-11T13:45:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-11T13:45:02.813 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-11T13:45:02.815 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-11T13:45:02.816 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-11T13:45:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-11T13:45:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-11T13:45:02.818 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-11T13:45:02.819 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-11T13:45:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-11T13:45:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-11T13:45:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-11T13:45:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-11T13:45:02.820 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-11T13:45:02.821 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-11T13:45:02.822 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:45:02.823 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T13:45:05.490 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:45:05.491 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:08 ago on Fri 11 Aug 2023 01:42:57 PM UTC. 2023-08-11T13:45:08.502 INFO:teuthology.orchestra.run.smithi003.stdout:Installed products updated. 2023-08-11T13:45:08.746 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:45:08.746 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2023-08-11T13:45:08.746 INFO:teuthology.orchestra.run.smithi003.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T13:45:08.747 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.749 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.750 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.751 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T13:45:08.752 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: jq-1.6-3.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.753 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:45:08.754 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T13:45:08.755 INFO:teuthology.orchestra.run.smithi003.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T13:45:08.756 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T13:45:08.757 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T13:45:08.758 INFO:teuthology.orchestra.run.smithi003.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T13:45:08.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T13:45:08.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.759 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T13:45:08.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T13:45:08.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T13:45:08.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T13:45:08.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T13:45:08.760 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T13:45:08.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T13:45:08.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T13:45:08.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T13:45:08.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T13:45:08.761 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T13:45:08.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T13:45:08.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T13:45:08.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T13:45:08.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T13:45:08.762 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T13:45:08.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T13:45:08.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T13:45:08.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T13:45:08.763 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T13:45:08.764 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.765 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T13:45:08.766 INFO:teuthology.orchestra.run.smithi003.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.767 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:45:08.768 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T13:45:08.768 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T13:45:08.768 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T13:45:08.768 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T13:45:08.768 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T13:45:08.769 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T13:45:08.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:45:08.769 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2023-08-11T13:45:09.067 DEBUG:teuthology.parallel:result is None 2023-08-11T13:49:02.540 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 41 kB/s | 16 MB 06:31 2023-08-11T13:49:05.998 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 425 kB/s | 24 kB 00:00 2023-08-11T13:49:06.950 INFO:teuthology.orchestra.run.smithi187.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-08-11T13:49:15.117 INFO:teuthology.orchestra.run.smithi187.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-08-11T13:49:26.203 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:49:26.204 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-11T13:49:26.219 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-11T13:49:26.220 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-11T13:49:26.221 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-11T13:49:26.222 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-11T13:49:26.224 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-11T13:49:26.227 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-11T13:49:26.228 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-11T13:49:26.361 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-08-11T13:49:26.374 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-08-11T13:49:26.374 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-08-11T13:49:26.374 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-08-11T13:49:26.374 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-08-11T13:49:26.374 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.4 k 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:18.0.0-5437.g92684807.el8 ceph 5.4 M 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 860 k 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5437.g92684807.el8 ceph 167 k 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:18.0.0-5437.g92684807.el8 ceph 1.6 M 2023-08-11T13:49:26.375 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 131 k 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 1.7 M 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 7.4 M 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 47 k 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 14 M 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:18.0.0-5437.g92684807.el8 ceph 48 M 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 261 k 2023-08-11T13:49:26.376 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 170 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 261 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 18 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 31 k 2023-08-11T13:49:26.377 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 783 k 2023-08-11T13:49:26.378 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:49:26.378 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-11T13:49:26.378 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-11T13:49:26.378 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:18.0.0-5437.g92684807.el8 ceph 197 k 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:18.0.0-5437.g92684807.el8 ceph 370 k 2023-08-11T13:49:26.379 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 364 k 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:18.0.0-5437.g92684807.el8 ceph 107 k 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:18.0.0-5437.g92684807.el8 ceph 94 k 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.5 M 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:18.0.0-5437.g92684807.el8 ceph 179 k 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2023-08-11T13:49:26.380 INFO:teuthology.orchestra.run.smithi187.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.7 M 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 3.6 M 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 22 M 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 23 k 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:18.0.0-5437.g92684807.el8 ceph 2.3 M 2023-08-11T13:49:26.381 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 241 k 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:18.0.0-5437.g92684807.el8 ceph 4.6 M 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:18.0.0-5437.g92684807.el8 ceph 18 M 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5437.g92684807.el8 ceph-noarch 15 k 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:18.0.0-5437.g92684807.el8 ceph 24 k 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-11T13:49:26.382 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:18.0.0-5437.g92684807.el8 ceph 139 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-11T13:49:26.383 INFO:teuthology.orchestra.run.smithi187.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:18.0.0-5437.g92684807.el8 ceph 473 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:18.0.0-5437.g92684807.el8 ceph 6.8 M 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-11T13:49:26.384 INFO:teuthology.orchestra.run.smithi187.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-11T13:49:26.385 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-11T13:49:26.386 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:18.0.0-5437.g92684807.el8 ceph 45 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:18.0.0-5437.g92684807.el8 ceph 125 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-11T13:49:26.387 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:49:26.388 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-11T13:49:26.389 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-11T13:49:26.390 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-11T13:49:26.391 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-11T13:49:26.392 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout:Enabling module streams: 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql 10 2023-08-11T13:49:26.393 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:49:26.394 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-08-11T13:49:26.394 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-08-11T13:49:26.394 INFO:teuthology.orchestra.run.smithi187.stdout:Install 128 Packages 2023-08-11T13:49:26.394 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 3 Packages 2023-08-11T13:49:26.394 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:49:26.395 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 206 M 2023-08-11T13:49:26.395 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-08-11T13:49:26.609 INFO:teuthology.orchestra.run.smithi187.stdout:(1/131): ceph-18.0.0-5437.g92684807.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-08-11T13:49:26.859 INFO:teuthology.orchestra.run.smithi187.stdout:(2/131): ceph-fuse-18.0.0-5437.g92684807.el8.x8 3.4 MB/s | 860 kB 00:00 2023-08-11T13:49:26.925 INFO:teuthology.orchestra.run.smithi187.stdout:(3/131): ceph-immutable-object-cache-18.0.0-543 2.5 MB/s | 167 kB 00:00 2023-08-11T13:49:27.284 INFO:teuthology.orchestra.run.smithi187.stdout:(4/131): ceph-mds-18.0.0-5437.g92684807.el8.x86 6.4 MB/s | 2.3 MB 00:00 2023-08-11T13:49:27.368 INFO:teuthology.orchestra.run.smithi187.stdout:(5/131): ceph-base-18.0.0-5437.g92684807.el8.x8 6.1 MB/s | 5.4 MB 00:00 2023-08-11T13:49:27.560 INFO:teuthology.orchestra.run.smithi187.stdout:(6/131): ceph-mgr-18.0.0-5437.g92684807.el8.x86 5.7 MB/s | 1.6 MB 00:00 2023-08-11T13:49:27.902 INFO:teuthology.orchestra.run.smithi187.stdout:(7/131): ceph-mon-18.0.0-5437.g92684807.el8.x86 8.7 MB/s | 4.6 MB 00:00 2023-08-11T13:49:28.944 INFO:teuthology.orchestra.run.smithi187.stdout:(8/131): ceph-common-18.0.0-5437.g92684807.el8. 9.0 MB/s | 22 MB 00:02 2023-08-11T13:49:29.010 INFO:teuthology.orchestra.run.smithi187.stdout:(9/131): ceph-selinux-18.0.0-5437.g92684807.el8 366 kB/s | 24 kB 00:00 2023-08-11T13:49:29.394 INFO:teuthology.orchestra.run.smithi187.stdout:(10/131): ceph-radosgw-18.0.0-5437.g92684807.el 9.1 MB/s | 14 MB 00:01 2023-08-11T13:49:29.461 INFO:teuthology.orchestra.run.smithi187.stdout:(11/131): cephfs-mirror-18.0.0-5437.g92684807.e 3.8 MB/s | 261 kB 00:00 2023-08-11T13:49:29.527 INFO:teuthology.orchestra.run.smithi187.stdout:(12/131): libcephfs-devel-18.0.0-5437.g92684807 467 kB/s | 31 kB 00:00 2023-08-11T13:49:29.678 INFO:teuthology.orchestra.run.smithi187.stdout:(13/131): libcephfs2-18.0.0-5437.g92684807.el8. 5.1 MB/s | 783 kB 00:00 2023-08-11T13:49:29.845 INFO:teuthology.orchestra.run.smithi187.stdout:(14/131): ceph-osd-18.0.0-5437.g92684807.el8.x8 7.8 MB/s | 18 MB 00:02 2023-08-11T13:49:29.862 INFO:teuthology.orchestra.run.smithi187.stdout:(15/131): libcephsqlite-18.0.0-5437.g92684807.e 756 kB/s | 139 kB 00:00 2023-08-11T13:49:29.912 INFO:teuthology.orchestra.run.smithi187.stdout:(16/131): librados-devel-18.0.0-5437.g92684807. 2.0 MB/s | 139 kB 00:00 2023-08-11T13:49:29.937 INFO:teuthology.orchestra.run.smithi187.stdout:(17/131): libradosstriper1-18.0.0-5437.g9268480 6.2 MB/s | 473 kB 00:00 2023-08-11T13:49:30.013 INFO:teuthology.orchestra.run.smithi187.stdout:(18/131): python3-ceph-argparse-18.0.0-5437.g92 595 kB/s | 45 kB 00:00 2023-08-11T13:49:30.088 INFO:teuthology.orchestra.run.smithi187.stdout:(19/131): python3-ceph-common-18.0.0-5437.g9268 1.6 MB/s | 125 kB 00:00 2023-08-11T13:49:30.163 INFO:teuthology.orchestra.run.smithi187.stdout:(20/131): python3-cephfs-18.0.0-5437.g92684807. 2.6 MB/s | 197 kB 00:00 2023-08-11T13:49:30.247 INFO:teuthology.orchestra.run.smithi187.stdout:(21/131): python3-rados-18.0.0-5437.g92684807.e 4.3 MB/s | 370 kB 00:00 2023-08-11T13:49:30.330 INFO:teuthology.orchestra.run.smithi187.stdout:(22/131): python3-rbd-18.0.0-5437.g92684807.el8 4.3 MB/s | 364 kB 00:00 2023-08-11T13:49:30.406 INFO:teuthology.orchestra.run.smithi187.stdout:(23/131): python3-rgw-18.0.0-5437.g92684807.el8 1.4 MB/s | 107 kB 00:00 2023-08-11T13:49:30.481 INFO:teuthology.orchestra.run.smithi187.stdout:(24/131): rbd-fuse-18.0.0-5437.g92684807.el8.x8 1.2 MB/s | 94 kB 00:00 2023-08-11T13:49:30.673 INFO:teuthology.orchestra.run.smithi187.stdout:(25/131): librgw2-18.0.0-5437.g92684807.el8.x86 8.9 MB/s | 6.8 MB 00:00 2023-08-11T13:49:30.757 INFO:teuthology.orchestra.run.smithi187.stdout:(26/131): rbd-nbd-18.0.0-5437.g92684807.el8.x86 2.1 MB/s | 179 kB 00:00 2023-08-11T13:49:30.857 INFO:teuthology.orchestra.run.smithi187.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5437.g 230 kB/s | 23 kB 00:00 2023-08-11T13:49:30.916 INFO:teuthology.orchestra.run.smithi187.stdout:(28/131): rbd-mirror-18.0.0-5437.g92684807.el8. 8.1 MB/s | 3.5 MB 00:00 2023-08-11T13:49:30.941 INFO:teuthology.orchestra.run.smithi187.stdout:(29/131): ceph-mgr-cephadm-18.0.0-5437.g9268480 1.5 MB/s | 131 kB 00:00 2023-08-11T13:49:31.149 INFO:teuthology.orchestra.run.smithi187.stdout:(30/131): ceph-mgr-dashboard-18.0.0-5437.g92684 7.5 MB/s | 1.7 MB 00:00 2023-08-11T13:49:31.233 INFO:teuthology.orchestra.run.smithi187.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5437.g92 2.8 MB/s | 241 kB 00:00 2023-08-11T13:49:31.308 INFO:teuthology.orchestra.run.smithi187.stdout:(32/131): ceph-mgr-rook-18.0.0-5437.g92684807.e 631 kB/s | 47 kB 00:00 2023-08-11T13:49:31.384 INFO:teuthology.orchestra.run.smithi187.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5437.g9 194 kB/s | 15 kB 00:00 2023-08-11T13:49:31.467 INFO:teuthology.orchestra.run.smithi187.stdout:(34/131): ceph-volume-18.0.0-5437.g92684807.el8 3.1 MB/s | 261 kB 00:00 2023-08-11T13:49:31.551 INFO:teuthology.orchestra.run.smithi187.stdout:(35/131): cephadm-18.0.0-5437.g92684807.el8.noa 2.0 MB/s | 170 kB 00:00 2023-08-11T13:49:31.618 INFO:teuthology.orchestra.run.smithi187.stdout:(36/131): cephfs-top-18.0.0-5437.g92684807.el8. 270 kB/s | 18 kB 00:00 2023-08-11T13:49:31.805 INFO:teuthology.orchestra.run.smithi187.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 8.6 MB/s | 7.4 MB 00:00 2023-08-11T13:49:32.377 INFO:teuthology.orchestra.run.smithi187.stdout:(38/131): ceph-test-18.0.0-5437.g92684807.el8.x 14 MB/s | 48 MB 00:03 2023-08-11T13:51:41.883 INFO:teuthology.orchestra.run.smithi187.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-08-11T13:51:41.884 INFO:teuthology.orchestra.run.smithi187.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-08-11T13:51:41.884 INFO:teuthology.orchestra.run.smithi187.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-08-11T13:51:42.151 INFO:teuthology.orchestra.run.smithi187.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 462 B/s | 59 kB 02:10 2023-08-11T13:51:42.181 INFO:teuthology.orchestra.run.smithi187.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 592 B/s | 75 kB 02:09 2023-08-11T13:51:42.223 INFO:teuthology.orchestra.run.smithi187.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 kB/s | 306 kB 02:10 2023-08-11T13:51:42.248 INFO:teuthology.orchestra.run.smithi187.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 653 kB/s | 44 kB 00:00 2023-08-11T13:51:42.282 INFO:teuthology.orchestra.run.smithi187.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.6 MB/s | 476 kB 00:00 2023-08-11T13:51:42.307 INFO:teuthology.orchestra.run.smithi187.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-08-11T13:51:42.333 INFO:teuthology.orchestra.run.smithi187.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-08-11T13:51:42.433 INFO:teuthology.orchestra.run.smithi187.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-08-11T13:51:42.467 INFO:teuthology.orchestra.run.smithi187.stdout:(47/131): python3-cachetools-3.1.1-4.el8.noarch 151 kB/s | 33 kB 00:00 2023-08-11T13:51:42.492 INFO:teuthology.orchestra.run.smithi187.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-08-11T13:51:42.517 INFO:teuthology.orchestra.run.smithi187.stdout:(49/131): python3-cheroot-8.5.2-1.el8.noarch.rp 824 kB/s | 173 kB 00:00 2023-08-11T13:51:42.542 INFO:teuthology.orchestra.run.smithi187.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-08-11T13:51:42.567 INFO:teuthology.orchestra.run.smithi187.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2023-08-11T13:51:42.593 INFO:teuthology.orchestra.run.smithi187.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 973 kB/s | 49 kB 00:00 2023-08-11T13:51:42.618 INFO:teuthology.orchestra.run.smithi187.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-08-11T13:51:42.652 INFO:teuthology.orchestra.run.smithi187.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-08-11T13:51:42.677 INFO:teuthology.orchestra.run.smithi187.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-08-11T13:51:42.711 INFO:teuthology.orchestra.run.smithi187.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 6.9 MB/s | 1.2 MB 00:00 2023-08-11T13:51:42.736 INFO:teuthology.orchestra.run.smithi187.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-08-11T13:51:42.761 INFO:teuthology.orchestra.run.smithi187.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 401 kB/s | 34 kB 00:00 2023-08-11T13:51:42.786 INFO:teuthology.orchestra.run.smithi187.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-08-11T13:51:42.812 INFO:teuthology.orchestra.run.smithi187.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-08-11T13:51:42.837 INFO:teuthology.orchestra.run.smithi187.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-08-11T13:51:42.862 INFO:teuthology.orchestra.run.smithi187.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2023-08-11T13:51:42.887 INFO:teuthology.orchestra.run.smithi187.stdout:(63/131): python3-rsa-4.9-2.el8.noarch.rpm 425 kB/s | 64 kB 00:00 2023-08-11T13:51:42.904 INFO:teuthology.orchestra.run.smithi187.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 410 kB/s | 27 kB 00:00 2023-08-11T13:51:42.930 INFO:teuthology.orchestra.run.smithi187.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-08-11T13:51:42.955 INFO:teuthology.orchestra.run.smithi187.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-08-11T13:51:42.980 INFO:teuthology.orchestra.run.smithi187.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 811 kB/s | 61 kB 00:00 2023-08-11T13:51:43.006 INFO:teuthology.orchestra.run.smithi187.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-08-11T13:51:43.022 INFO:teuthology.orchestra.run.smithi187.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 342 kB/s | 23 kB 00:00 2023-08-11T13:51:43.048 INFO:teuthology.orchestra.run.smithi187.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-08-11T13:51:43.090 INFO:teuthology.orchestra.run.smithi187.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-08-11T13:51:43.116 INFO:teuthology.orchestra.run.smithi187.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.2 MB/s | 200 kB 00:00 2023-08-11T13:51:43.140 INFO:teuthology.orchestra.run.smithi187.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 852 kB/s | 43 kB 00:00 2023-08-11T13:51:43.165 INFO:teuthology.orchestra.run.smithi187.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-08-11T13:51:43.183 INFO:teuthology.orchestra.run.smithi187.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-08-11T13:51:43.208 INFO:teuthology.orchestra.run.smithi187.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-08-11T13:51:43.233 INFO:teuthology.orchestra.run.smithi187.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 846 kB/s | 43 kB 00:00 2023-08-11T13:51:43.259 INFO:teuthology.orchestra.run.smithi187.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-08-11T13:51:43.300 INFO:teuthology.orchestra.run.smithi187.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 5.4 MB/s | 1.7 MB 00:00 2023-08-11T13:51:43.326 INFO:teuthology.orchestra.run.smithi187.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-08-11T13:51:43.360 INFO:teuthology.orchestra.run.smithi187.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.4 MB/s | 560 kB 00:00 2023-08-11T13:51:43.385 INFO:teuthology.orchestra.run.smithi187.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 223 kB/s | 19 kB 00:00 2023-08-11T13:51:43.410 INFO:teuthology.orchestra.run.smithi187.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.9 MB/s | 155 kB 00:00 2023-08-11T13:51:43.460 INFO:teuthology.orchestra.run.smithi187.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 238 kB/s | 18 kB 00:00 2023-08-11T13:51:43.494 INFO:teuthology.orchestra.run.smithi187.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.1 MB/s | 559 kB 00:00 2023-08-11T13:51:43.527 INFO:teuthology.orchestra.run.smithi187.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-08-11T13:51:43.553 INFO:teuthology.orchestra.run.smithi187.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-08-11T13:51:43.578 INFO:teuthology.orchestra.run.smithi187.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-08-11T13:51:43.603 INFO:teuthology.orchestra.run.smithi187.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-08-11T13:51:43.629 INFO:teuthology.orchestra.run.smithi187.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-08-11T13:51:43.654 INFO:teuthology.orchestra.run.smithi187.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-08-11T13:51:43.679 INFO:teuthology.orchestra.run.smithi187.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-08-11T13:51:43.704 INFO:teuthology.orchestra.run.smithi187.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-08-11T13:51:43.738 INFO:teuthology.orchestra.run.smithi187.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-08-11T13:51:43.763 INFO:teuthology.orchestra.run.smithi187.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-08-11T13:51:43.788 INFO:teuthology.orchestra.run.smithi187.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-08-11T13:51:43.847 INFO:teuthology.orchestra.run.smithi187.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2023-08-11T13:51:43.864 INFO:teuthology.orchestra.run.smithi187.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-08-11T13:51:43.889 INFO:teuthology.orchestra.run.smithi187.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-11T13:51:43.914 INFO:teuthology.orchestra.run.smithi187.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-08-11T13:51:43.948 INFO:teuthology.orchestra.run.smithi187.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-08-11T13:51:43.973 INFO:teuthology.orchestra.run.smithi187.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-08-11T13:51:44.015 INFO:teuthology.orchestra.run.smithi187.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-08-11T13:51:44.049 INFO:teuthology.orchestra.run.smithi187.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-08-11T13:51:44.074 INFO:teuthology.orchestra.run.smithi187.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-08-11T13:51:44.108 INFO:teuthology.orchestra.run.smithi187.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-08-11T13:51:44.142 INFO:teuthology.orchestra.run.smithi187.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-08-11T13:51:44.175 INFO:teuthology.orchestra.run.smithi187.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-08-11T13:51:44.217 INFO:teuthology.orchestra.run.smithi187.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-11T13:51:44.259 INFO:teuthology.orchestra.run.smithi187.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-08-11T13:51:44.343 INFO:teuthology.orchestra.run.smithi187.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-08-11T13:51:44.385 INFO:teuthology.orchestra.run.smithi187.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-08-11T13:51:44.427 INFO:teuthology.orchestra.run.smithi187.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-08-11T13:51:44.485 INFO:teuthology.orchestra.run.smithi187.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-08-11T13:51:44.511 INFO:teuthology.orchestra.run.smithi187.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 764 kB/s | 64 kB 00:00 2023-08-11T13:51:44.536 INFO:teuthology.orchestra.run.smithi187.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-08-11T13:51:44.561 INFO:teuthology.orchestra.run.smithi187.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-08-11T13:51:44.586 INFO:teuthology.orchestra.run.smithi187.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-08-11T13:51:44.628 INFO:teuthology.orchestra.run.smithi187.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-08-11T13:51:44.804 INFO:teuthology.orchestra.run.smithi187.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-08-11T13:51:44.829 INFO:teuthology.orchestra.run.smithi187.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 988 kB/s | 198 kB 00:00 2023-08-11T13:51:44.871 INFO:teuthology.orchestra.run.smithi187.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-08-11T13:51:44.913 INFO:teuthology.orchestra.run.smithi187.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-08-11T13:51:44.996 INFO:teuthology.orchestra.run.smithi187.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-08-11T13:51:45.072 INFO:teuthology.orchestra.run.smithi187.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-08-11T13:51:45.105 INFO:teuthology.orchestra.run.smithi187.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-08-11T13:51:45.156 INFO:teuthology.orchestra.run.smithi187.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-08-11T13:51:45.231 INFO:teuthology.orchestra.run.smithi187.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-08-11T13:51:45.281 INFO:teuthology.orchestra.run.smithi187.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 17 MB/s | 798 kB 00:00 2023-08-11T13:51:45.606 INFO:teuthology.orchestra.run.smithi187.stdout:(130/131): librados2-18.0.0-5437.g92684807.el8. 7.4 MB/s | 3.7 MB 00:00 2023-08-11T13:51:45.698 INFO:teuthology.orchestra.run.smithi187.stdout:(131/131): librbd1-18.0.0-5437.g92684807.el8.x8 6.7 MB/s | 3.6 MB 00:00 2023-08-11T13:51:45.700 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:51:45.701 INFO:teuthology.orchestra.run.smithi187.stdout:Total 1.5 MB/s | 206 MB 02:19 2023-08-11T13:51:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-08-11T13:51:46.590 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-08-11T13:51:46.590 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-08-11T13:51:47.697 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-08-11T13:51:47.701 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-08-11T13:51:52.494 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-08-11T13:51:52.673 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-11T13:51:52.848 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-11T13:51:53.025 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-11T13:51:53.131 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-11T13:51:53.343 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-11T13:51:53.766 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-11T13:51:53.944 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:18.0.0-5437.g92684807.el8.x86_64 5/134 2023-08-11T13:51:54.342 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:18.0.0-5437.g92684807.el8.x86_64 5/134 2023-08-11T13:51:54.500 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:51:54.631 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:51:54.834 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:51:54.915 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:51:55.097 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:51:55.259 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-11T13:51:55.488 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-11T13:51:55.694 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-11T13:51:56.069 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-11T13:51:56.278 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-11T13:51:56.441 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:51:56.533 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 14/134 2023-08-11T13:51:56.666 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:51:56.817 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 15/134 2023-08-11T13:51:57.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-11T13:51:57.197 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-11T13:51:57.375 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-11T13:51:57.499 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-11T13:51:57.645 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-11T13:51:57.799 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-11T13:51:58.413 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-11T13:51:58.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-11T13:51:59.147 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-11T13:51:59.315 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-11T13:52:00.020 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-11T13:52:00.282 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-11T13:52:00.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-11T13:52:00.683 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-11T13:52:00.855 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 26/134 2023-08-11T13:52:00.976 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 27/134 2023-08-11T13:52:01.185 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 28/134 2023-08-11T13:52:01.373 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-11T13:52:03.134 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-11T13:52:04.132 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-11T13:52:04.470 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-11T13:52:04.733 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-11T13:52:04.934 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-11T13:52:05.142 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-11T13:52:05.366 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-11T13:52:05.529 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 36/134 2023-08-11T13:52:05.679 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 37/134 2023-08-11T13:52:05.830 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-11T13:52:05.966 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-11T13:52:06.114 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-11T13:52:06.165 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-11T13:52:06.383 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-11T13:52:06.601 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-11T13:52:06.739 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-11T13:52:06.802 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-11T13:52:06.993 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-11T13:52:07.160 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-11T13:52:07.321 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-11T13:52:07.474 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-11T13:52:07.636 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-11T13:52:07.767 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-11T13:52:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-11T13:52:08.184 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-11T13:52:08.539 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-11T13:52:08.815 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-11T13:52:09.041 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-11T13:52:09.284 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-11T13:52:09.502 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-11T13:52:09.789 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-11T13:52:09.996 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-11T13:52:10.191 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-11T13:52:10.443 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-11T13:52:10.593 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-11T13:52:10.767 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-11T13:52:11.060 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-11T13:52:11.294 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-11T13:52:11.518 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-11T13:52:11.773 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-11T13:52:11.938 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-11T13:52:12.107 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-11T13:52:12.372 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-11T13:52:12.603 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-11T13:52:12.742 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-11T13:52:12.890 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-11T13:52:13.122 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-11T13:52:13.324 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-11T13:52:13.520 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-11T13:52:13.699 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-11T13:52:13.845 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-11T13:52:14.068 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-11T13:52:14.325 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-11T13:52:14.502 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-11T13:52:14.719 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-11T13:52:14.929 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-11T13:52:15.096 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-11T13:52:15.322 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-11T13:52:16.437 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-11T13:52:16.791 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-11T13:52:17.045 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-11T13:52:17.345 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-11T13:52:17.613 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-11T13:52:17.812 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-11T13:52:18.582 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-11T13:52:18.729 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:52:18.801 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 92/134 2023-08-11T13:52:18.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 93/134 2023-08-11T13:52:22.609 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:52:22.795 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:52:23.390 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 94/134 2023-08-11T13:52:23.681 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:52:23.776 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 95/134 2023-08-11T13:52:23.903 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:52:42.762 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 96/134 2023-08-11T13:52:42.762 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2023-08-11T13:52:42.762 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2023-08-11T13:52:42.762 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /run 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2023-08-11T13:52:42.763 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-08-11T13:52:42.764 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:52:43.149 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:52:44.731 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 97/134 2023-08-11T13:52:44.877 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:52:45.094 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 98/134 2023-08-11T13:52:45.249 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:52:45.702 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 99/134 2023-08-11T13:52:45.850 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:52:45.865 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 100/134 2023-08-11T13:52:47.466 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:52:47.646 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:18.0.0-5437.g92684807.el8.noarch 101/134 2023-08-11T13:52:47.754 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 102/134 2023-08-11T13:52:47.956 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 103/134 2023-08-11T13:52:48.165 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:52:48.382 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 104/134 2023-08-11T13:52:48.660 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:52:49.618 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 105/134 2023-08-11T13:52:49.790 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:52:49.880 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 106/134 2023-08-11T13:52:50.222 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 107/134 2023-08-11T13:52:50.412 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:52:50.445 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 108/134 2023-08-11T13:52:50.581 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:52:50.592 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 109/134 2023-08-11T13:52:51.917 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:18.0.0-5437.g92684807.el8.x86_64 110/134 2023-08-11T13:52:52.093 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:52:52.127 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 111/134 2023-08-11T13:52:52.256 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:52:52.306 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5437.g92684 112/134 2023-08-11T13:52:52.448 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:52:52.805 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 113/134 2023-08-11T13:52:52.948 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:52:57.499 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 114/134 2023-08-11T13:52:58.350 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 115/134 2023-08-11T13:52:58.648 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-11T13:52:58.824 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 117/134 2023-08-11T13:52:59.061 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-11T13:52:59.229 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-11T13:52:59.427 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-11T13:52:59.676 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-11T13:52:59.825 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 121/134 2023-08-11T13:52:59.972 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-11T13:53:02.137 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:53:02.404 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:53:02.422 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-11T13:53:02.546 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 124/134 2023-08-11T13:53:02.822 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 125/134 2023-08-11T13:53:03.112 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 126/134 2023-08-11T13:53:03.412 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-11T13:53:03.633 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-11T13:53:03.713 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-11T13:53:03.888 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-11T13:53:03.899 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-11T13:53:04.046 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-11T13:53:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-11T13:53:04.173 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-11T13:53:04.173 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:53:04.365 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:53:04.366 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:53:04.602 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-11T13:53:04.811 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:53:05.350 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:53:05.350 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:18.0.0-5437.g92684807.el8.x86_64 1/134 2023-08-11T13:53:05.351 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2/134 2023-08-11T13:53:05.351 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 3/134 2023-08-11T13:53:05.351 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 4/134 2023-08-11T13:53:05.351 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5437.g92684 5/134 2023-08-11T13:53:05.352 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 6/134 2023-08-11T13:53:05.352 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 7/134 2023-08-11T13:53:05.352 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 8/134 2023-08-11T13:53:05.352 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 9/134 2023-08-11T13:53:05.352 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 10/134 2023-08-11T13:53:05.353 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 11/134 2023-08-11T13:53:05.353 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 12/134 2023-08-11T13:53:05.353 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 13/134 2023-08-11T13:53:05.353 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_ 14/134 2023-08-11T13:53:05.354 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 15/134 2023-08-11T13:53:05.354 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 16/134 2023-08-11T13:53:05.354 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:18.0.0-5437.g92684807.el8.x86_6 17/134 2023-08-11T13:53:05.354 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:18.0.0-5437.g92684807.el8.x86 18/134 2023-08-11T13:53:05.354 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:18.0.0-5437.g92684807.el8.x86_64 19/134 2023-08-11T13:53:05.355 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5437.g92684807.el 20/134 2023-08-11T13:53:05.355 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:18.0.0-5437.g92684807.el8. 21/134 2023-08-11T13:53:05.355 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_6 22/134 2023-08-11T13:53:05.355 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 23/134 2023-08-11T13:53:05.356 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 24/134 2023-08-11T13:53:05.356 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 25/134 2023-08-11T13:53:05.356 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 26/134 2023-08-11T13:53:05.356 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 27/134 2023-08-11T13:53:05.356 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 28/134 2023-08-11T13:53:05.357 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5437.g92684807. 29/134 2023-08-11T13:53:05.357 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noa 30/134 2023-08-11T13:53:05.357 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.n 31/134 2023-08-11T13:53:05.357 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5437.g926 32/134 2023-08-11T13:53:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el 33/134 2023-08-11T13:53:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 34/134 2023-08-11T13:53:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5437.g92684807.e 35/134 2023-08-11T13:53:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 36/134 2023-08-11T13:53:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:18.0.0-5437.g92684807.el8.noarch 37/134 2023-08-11T13:53:05.359 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 38/134 2023-08-11T13:53:05.359 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-11T13:53:05.359 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-11T13:53:05.359 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-11T13:53:05.360 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-11T13:53:05.360 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-11T13:53:05.360 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-11T13:53:05.360 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-11T13:53:05.360 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-11T13:53:05.361 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-11T13:53:05.362 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-11T13:53:05.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-11T13:53:05.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-11T13:53:05.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-11T13:53:05.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-11T13:53:05.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-11T13:53:05.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-11T13:53:05.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-11T13:53:05.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-11T13:53:05.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-11T13:53:05.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-11T13:53:05.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-11T13:53:05.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-11T13:53:05.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-11T13:53:05.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-11T13:53:05.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-11T13:53:05.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-11T13:53:05.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-11T13:53:05.368 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-11T13:53:05.368 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-11T13:53:05.368 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-11T13:53:05.368 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-11T13:53:05.369 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-11T13:53:05.369 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-11T13:53:05.369 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-11T13:53:05.370 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-11T13:53:05.370 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-11T13:53:05.370 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-11T13:53:05.370 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-11T13:53:05.371 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-11T13:53:05.371 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-11T13:53:05.371 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-11T13:53:05.371 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-11T13:53:05.372 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-11T13:53:05.372 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-11T13:53:05.372 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-11T13:53:05.372 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-11T13:53:05.373 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-11T13:53:05.373 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-11T13:53:05.373 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-11T13:53:05.374 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-11T13:53:05.374 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-11T13:53:05.374 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-11T13:53:05.374 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-11T13:53:05.374 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-11T13:53:05.375 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-11T13:53:05.375 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-11T13:53:05.375 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-11T13:53:05.376 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-11T13:53:05.376 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-11T13:53:05.376 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-11T13:53:05.377 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-11T13:53:05.377 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-11T13:53:05.377 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-11T13:53:05.377 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-11T13:53:05.378 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-11T13:53:05.378 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-11T13:53:05.378 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-11T13:53:05.379 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-11T13:53:05.379 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-11T13:53:05.379 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-11T13:53:05.379 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-11T13:53:05.380 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-11T13:53:05.380 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-11T13:53:05.380 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-11T13:53:05.380 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:18.0.0-5437.g92684807.el8.x86_64 129/134 2023-08-11T13:53:05.381 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-11T13:53:05.381 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:18.0.0-5437.g92684807.el8.x86_64 131/134 2023-08-11T13:53:05.381 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-11T13:53:05.382 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-11T13:53:07.777 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-11T13:53:07.777 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:03:53 ago on Fri 11 Aug 2023 01:49:14 PM UTC. 2023-08-11T13:53:10.530 INFO:teuthology.orchestra.run.smithi187.stdout:Installed products updated. 2023-08-11T13:53:10.741 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:53:10.741 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2023-08-11T13:53:10.741 INFO:teuthology.orchestra.run.smithi187.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-11T13:53:10.741 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.741 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.1-7.el8.noarch 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.0.4-10.el8.x86_64 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.742 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.743 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.744 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:18.0.0-5437.g92684807.el8.noarch 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.1-9.el8.x86_64 2023-08-11T13:53:10.745 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-3.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.746 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: libpq-13.5-1.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-11T13:53:10.747 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.748 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.748 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-11T13:53:10.748 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.748 INFO:teuthology.orchestra.run.smithi187.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-11T13:53:10.748 INFO:teuthology.orchestra.run.smithi187.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-11T13:53:10.749 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-11T13:53:10.749 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-11T13:53:10.749 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-11T13:53:10.749 INFO:teuthology.orchestra.run.smithi187.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-11T13:53:10.750 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-11T13:53:10.751 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-11T13:53:10.751 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:53:10.751 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:53:10.751 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-11T13:53:10.751 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.752 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-11T13:53:10.753 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-11T13:53:10.754 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-11T13:53:10.755 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-2.6-1.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-11T13:53:10.756 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-11T13:53:10.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-11T13:53:10.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-11T13:53:10.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-11T13:53:10.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-11T13:53:10.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-11T13:53:10.759 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-11T13:53:10.759 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-11T13:53:10.759 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-11T13:53:10.759 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-11T13:53:10.759 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.760 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:18.0.0-5437.g92684807.el8.x86_64 2023-08-11T13:53:10.761 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-11T13:53:10.761 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-11T13:53:10.762 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-11T13:53:10.762 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-11T13:53:10.762 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-11T13:53:10.763 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-11T13:53:10.763 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:53:10.763 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-08-11T13:53:10.956 DEBUG:teuthology.parallel:result is None 2023-08-11T13:53:10.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:11.184 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T13:53:11.246 INFO:teuthology.orchestra.run.smithi003.stdout:18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.247 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.248 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T13:53:11.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:11.394 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T13:53:11.423 INFO:teuthology.orchestra.run.smithi103.stdout:18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.423 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.423 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T13:53:11.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:11.574 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-11T13:53:11.600 INFO:teuthology.orchestra.run.smithi187.stdout:18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.600 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5437.g92684807.el8 2023-08-11T13:53:11.601 INFO:teuthology.task.install:The correct ceph version 18.0.0-5437.g92684807 is installed. 2023-08-11T13:53:11.601 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-11T13:53:11.602 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:53:11.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T13:53:11.699 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:53:11.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T13:53:11.748 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:53:11.749 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-11T13:53:11.793 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-11T13:53:11.794 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:53:11.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T13:53:11.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T13:53:11.974 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:53:11.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T13:53:12.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T13:53:12.078 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:53:12.079 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2023-08-11T13:53:12.110 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-11T13:53:12.178 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-11T13:53:12.178 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:53:12.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T13:53:12.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T13:53:12.354 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:53:12.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T13:53:12.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T13:53:12.457 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:53:12.457 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-11T13:53:12.486 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-11T13:53:12.556 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-11T13:53:12.557 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:53:12.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T13:53:12.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T13:53:12.737 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:53:12.737 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T13:53:12.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T13:53:12.845 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:53:12.845 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2023-08-11T13:53:12.876 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-11T13:53:12.944 INFO:teuthology.run_tasks:Running task cephadm... 2023-08-11T13:53:13.072 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-08-11T13:53:13.075 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:13.091 INFO:tasks.cephadm:Cluster fsid is 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:53:13.092 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-08-11T13:53:13.092 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.103', 'mon.c': '172.21.15.187'} 2023-08-11T13:53:13.092 INFO:tasks.cephadm:First mon is mon.a on smithi003 2023-08-11T13:53:13.093 INFO:tasks.cephadm:First mgr is x 2023-08-11T13:53:13.093 INFO:tasks.cephadm:Normalizing hostnames... 2023-08-11T13:53:13.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2023-08-11T13:53:13.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2023-08-11T13:53:13.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname $(hostname -s) 2023-08-11T13:53:13.230 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-08-11T13:53:13.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:13.466 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '92684807f016ce045ac66e37fdde6f8c47857bcc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71982/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-5437-g92684807', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-5437.g92684807'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-08-10 16:30:02.468413', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Aug1-b4', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-08-11T13:53:13.682 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-2023Aug1-b4, sha1 92684807f016ce045ac66e37fdde6f8c47857bcc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=92684807f016ce045ac66e37fdde6f8c47857bcc 2023-08-11T13:53:13.684 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm 2023-08-11T13:53:13.684 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:14.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 13:53 /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:14.553 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.031 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 13:53 /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.031 DEBUG:teuthology.orchestra.run.smithi187:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Aug1-b4/92684807f016ce045ac66e37fdde6f8c47857bcc/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.502 INFO:teuthology.orchestra.run.smithi187.stdout:-rw-rw-r--. 1 ubuntu ubuntu 698557 Aug 11 13:53 /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.503 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.557 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.581 DEBUG:teuthology.orchestra.run.smithi187:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-08-11T13:53:15.643 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc on all hosts... 2023-08-11T13:53:15.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T13:53:15.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T13:53:15.651 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc pull 2023-08-11T13:53:16.919 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T13:53:17.144 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T13:53:21.547 INFO:teuthology.orchestra.run.smithi187.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T13:53:46.428 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-08-11T13:53:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T13:53:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T13:53:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: "repo_digests": [ 2023-08-11T13:53:46.429 INFO:teuthology.orchestra.run.smithi187.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T13:53:46.430 INFO:teuthology.orchestra.run.smithi187.stdout: ] 2023-08-11T13:53:46.430 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T13:53:48.705 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T13:53:48.706 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T13:53:50.426 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2023-08-11T13:53:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T13:53:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c", 2023-08-11T13:53:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2023-08-11T13:53:50.426 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75a49d67f8e63bcfe7c72420c0bd76f2f6be8c4f86dd373c0bd2e3def3a2e2f3" 2023-08-11T13:53:50.427 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2023-08-11T13:53:50.427 INFO:teuthology.orchestra.run.smithi103.stdout:} 2023-08-11T13:53:50.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2023-08-11T13:53:50.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2023-08-11T13:53:50.555 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph 2023-08-11T13:53:50.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2023-08-11T13:53:50.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2023-08-11T13:53:50.693 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /etc/ceph 2023-08-11T13:53:50.724 INFO:tasks.cephadm:Writing seed config... 2023-08-11T13:53:50.726 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-08-11T13:53:50.727 INFO:tasks.cephadm: override: [client] debug client = 20 2023-08-11T13:53:50.729 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-08-11T13:53:50.730 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-08-11T13:53:50.731 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-08-11T13:53:50.731 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-08-11T13:53:50.731 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-08-11T13:53:50.731 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-08-11T13:53:50.731 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-08-11T13:53:50.732 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-08-11T13:53:50.732 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-08-11T13:53:50.733 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-08-11T13:53:50.733 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-08-11T13:53:50.734 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-08-11T13:53:50.734 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-08-11T13:53:50.735 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-08-11T13:53:50.735 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-08-11T13:53:50.735 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-08-11T13:53:50.735 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-08-11T13:53:50.736 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-08-11T13:53:50.736 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-08-11T13:53:50.736 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-08-11T13:53:50.736 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-08-11T13:53:50.737 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-08-11T13:53:50.737 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-08-11T13:53:50.737 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-08-11T13:53:50.737 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-08-11T13:53:50.737 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-08-11T13:53:50.738 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-08-11T13:53:50.740 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-08-11T13:53:50.740 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-08-11T13:53:50.741 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-08-11T13:53:50.741 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-08-11T13:53:50.742 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-08-11T13:53:50.742 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-08-11T13:53:50.742 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-08-11T13:53:50.742 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-08-11T13:53:50.743 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-08-11T13:53:50.743 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-08-11T13:53:50.743 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-08-11T13:53:50.743 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-08-11T13:53:50.743 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-08-11T13:53:50.744 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-08-11T13:53:50.745 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:53:50.745 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-08-11T13:53:50.789 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 69ee59aa-384e-11ee-9b37-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-08-11T13:53:50.789 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service 2023-08-11T13:53:50.834 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x.service 2023-08-11T13:53:50.879 INFO:tasks.cephadm:Bootstrapping... 2023-08-11T13:53:50.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc -v bootstrap --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-08-11T13:53:50.902 INFO:journalctl@ceph.mon.a.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T13:53:50.947 INFO:journalctl@ceph.mgr.x.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T13:53:51.110 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2023-08-11T13:53:51.110 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc', '-v', 'bootstrap', '--fsid', '69ee59aa-384e-11ee-9b37-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2023-08-11T13:53:51.283 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.1.1 2023-08-11T13:53:51.283 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-08-11T13:53:51.284 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2023-08-11T13:53:51.453 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.1.1 2023-08-11T13:53:51.454 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2023-08-11T13:53:51.454 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2023-08-11T13:53:51.468 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-11T13:53:51.468 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-11T13:53:51.481 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-11T13:53:51.481 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:53:51.495 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2023-08-11T13:53:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2023-08-11T13:53:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2023-08-11T13:53:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2023-08-11T13:53:51.678 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.1.1 2023-08-11T13:53:51.679 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 4.1.1 is present 2023-08-11T13:53:51.679 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2023-08-11T13:53:51.679 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2023-08-11T13:53:51.692 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-08-11T13:53:51.692 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-08-11T13:53:51.705 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-08-11T13:53:51.705 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:53:51.718 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2023-08-11T13:53:51.731 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2023-08-11T13:53:51.732 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2023-08-11T13:53:51.732 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2023-08-11T13:53:51.732 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:53:51.733 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 139849725520360 on /run/cephadm/69ee59aa-384e-11ee-9b37-001a4aab830c.lock 2023-08-11T13:53:51.733 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 139849725520360 acquired on /run/cephadm/69ee59aa-384e-11ee-9b37-001a4aab830c.lock 2023-08-11T13:53:51.733 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2023-08-11T13:53:51.733 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2023-08-11T13:53:51.734 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2023-08-11T13:53:51.741 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-08-11T13:53:51.741 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2023-08-11T13:53:51.747 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-08-11T13:53:51.748 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-08-11T13:53:51.748 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1784sec hoplimit 64 pref medium 2023-08-11T13:53:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-08-11T13:53:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-08-11T13:53:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-11T13:53:51.754 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-08-11T13:53:51.755 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-08-11T13:53:51.755 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-08-11T13:53:51.755 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2023-08-11T13:53:51.755 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2023-08-11T13:53:51.756 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-08-11T13:53:51.756 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-08-11T13:53:51.757 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T13:53:52.232 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc... 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:1466c0ce567d8cab08280e456a5ef839cd0bf60a0cec2e3199abbfddb92a2725 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:a2667c7aa6ee86ee2d3da372adfcb22033620f9a001092468edeacf683f0ad8c 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2023-08-11T13:53:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Storing signatures 2023-08-11T13:53:53.108 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev) 2023-08-11T13:53:53.109 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev) 2023-08-11T13:53:53.109 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2023-08-11T13:53:53.992 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2023-08-11T13:53:53.992 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2023-08-11T13:53:54.768 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQByPdZkZcNAGBAAeVoQYij8gn8ic7uckf3DbA== 2023-08-11T13:53:55.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQBzPdZkO2yTDBAAxWKPRE4qHj/krZ0yvGajfA== 2023-08-11T13:53:56.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQB0PdZkb7TXAxAAxdzDro6IgVOelRhHeNS09Q== 2023-08-11T13:53:56.519 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2023-08-11T13:53:57.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = pacific 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:53:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-08-11T13:53:57.402 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:53:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2023-08-11T13:53:58.581 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2023-08-11T13:53:58.847 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-08-11T13:53:59.051 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-08-11T13:53:59.306 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target → /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target. 2023-08-11T13:53:59.306 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target → /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target. 2023-08-11T13:53:59.654 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a 2023-08-11T13:53:59.654 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service: Unit ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service not loaded. 2023-08-11T13:53:59.866 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target.wants/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@.service. 2023-08-11T13:54:00.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:53:59 smithi003 systemd[1]: Starting Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:54:00.626 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:00 smithi003 podman[96643]: 2023-08-11T13:54:00.626 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:00 smithi003 bash[96643]: 364c074ca181d6217c5b2a8ed3fa3fe5a2eff352b7f4513c340b8113b6c4e2a5 2023-08-11T13:54:00.682 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T13:54:00.683 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2023-08-11T13:54:00.698 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T13:54:00.699 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:54:00.699 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2023-08-11T13:54:00.700 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2023-08-11T13:54:00.701 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2023-08-11T13:54:00.701 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2023-08-11T13:54:00.958 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:00 smithi003 ceph-mon[96683]: mkfs 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:00.958 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:00 smithi003 ceph-mon[96683]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T13:54:00.958 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:00 smithi003 systemd[1]: Started Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:54:01.784 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2023-08-11T13:54:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:01.785 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.697729s) 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:01.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.229+0000 7f31e2ab4700 1 Processor -- start 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.230+0000 7f31e2ab4700 1 -- start start 2023-08-11T13:54:01.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.230+0000 7f31e2ab4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.230+0000 7f31e2ab4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc106ac0 con 0x7f31dc1060d0 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.231+0000 7f31e0850700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.231+0000 7f31e0850700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52556/0 (socket says 172.21.15.3:52556) 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.231+0000 7f31e0850700 1 -- 172.21.15.3:0/3499173009 learned_addr learned my addr 172.21.15.3:0/3499173009 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.231+0000 7f31e0850700 1 -- 172.21.15.3:0/3499173009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc1072d0 con 0x7f31dc1060d0 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31e0850700 1 --2- 172.21.15.3:0/3499173009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31d00098a0 tx=0x7f31d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f71e9f0e1a661c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:01.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31db7fe700 1 -- 172.21.15.3:0/3499173009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d0004160 con 0x7f31dc1060d0 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31db7fe700 1 -- 172.21.15.3:0/3499173009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d00042c0 con 0x7f31dc1060d0 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31db7fe700 1 -- 172.21.15.3:0/3499173009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d00045a0 con 0x7f31dc1060d0 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3499173009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 msgr2=0x7f31dc1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.232+0000 7f31e2ab4700 1 --2- 172.21.15.3:0/3499173009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31d00098a0 tx=0x7f31d0009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.233+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3499173009 shutdown_connections 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.233+0000 7f31e2ab4700 1 --2- 172.21.15.3:0/3499173009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc1064f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.233+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3499173009 >> 172.21.15.3:0/3499173009 conn(0x7f31dc1016b0 msgr2=0x7f31dc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:01.789 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.233+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3499173009 shutdown_connections 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.233+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3499173009 wait complete. 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.234+0000 7f31e2ab4700 1 Processor -- start 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.234+0000 7f31e2ab4700 1 -- start start 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.234+0000 7f31e2ab4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.234+0000 7f31e2ab4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc07c1b0 con 0x7f31dc1060d0 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.235+0000 7f31e0850700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.235+0000 7f31e0850700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52560/0 (socket says 172.21.15.3:52560) 2023-08-11T13:54:01.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.235+0000 7f31e0850700 1 -- 172.21.15.3:0/3323293372 learned_addr learned my addr 172.21.15.3:0/3323293372 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.235+0000 7f31e0850700 1 -- 172.21.15.3:0/3323293372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009550 con 0x7f31dc1060d0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.235+0000 7f31e0850700 1 --2- 172.21.15.3:0/3323293372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f31d0004000 tx=0x7f31d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d0011630 con 0x7f31dc1060d0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d0011790 con 0x7f31dc1060d0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc07a2c0 con 0x7f31dc1060d0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31d0011a30 con 0x7f31dc1060d0 2023-08-11T13:54:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc07a6e0 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f31d0022070 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.236+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f31d001c450 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.237+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc07b0f0 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.240+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f31d001b450 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.317+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f31dc192860 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.317+0000 7f31d9ffb700 1 -- 172.21.15.3:0/3323293372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f31d001b5f0 con 0x7f31dc1060d0 2023-08-11T13:54:01.792 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 msgr2=0x7f31dc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 --2- 172.21.15.3:0/3323293372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f31d0004000 tx=0x7f31d0004750 comp rx=0 tx=0).stop 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 shutdown_connections 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 --2- 172.21.15.3:0/3323293372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31dc1060d0 0x7f31dc07bc70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 >> 172.21.15.3:0/3323293372 conn(0x7f31dc1016b0 msgr2=0x7f31dc18da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 shutdown_connections 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:01.318+0000 7f31e2ab4700 1 -- 172.21.15.3:0/3323293372 wait complete. 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2023-08-11T13:54:01.793 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2023-08-11T13:54:02.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T13:54:02.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} 2023-08-11T13:54:02.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: fsmap 2023-08-11T13:54:02.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: osdmap e1: 0 total, 0 up, 0 in 2023-08-11T13:54:02.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: mgrmap e1: no daemons active 2023-08-11T13:54:02.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:01 smithi003 ceph-mon[96683]: from='client.? 172.21.15.3:0/3323293372' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-08-11T13:54:02.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:02 smithi003 ceph-mon[96683]: from='client.? 172.21.15.3:0/1211583563' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-11T13:54:02.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:02 smithi003 ceph-mon[96683]: from='client.? 172.21.15.3:0/1211583563' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-08-11T13:54:04.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-11T13:54:04.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2023-08-11T13:54:04.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.297+0000 7fb55799d700 1 Processor -- start 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.297+0000 7fb55799d700 1 -- start start 2023-08-11T13:54:04.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb55799d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb55799d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55007a8b0 con 0x7fb55007be80 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb555739700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb555739700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52124/0 (socket says 172.21.15.3:52124) 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb555739700 1 -- 172.21.15.3:0/2728121796 learned_addr learned my addr 172.21.15.3:0/2728121796 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.298+0000 7fb555739700 1 -- 172.21.15.3:0/2728121796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55007a9f0 con 0x7fb55007be80 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb555739700 1 --2- 172.21.15.3:0/2728121796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0098a0 tx=0x7fb54c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d7a6ef22c54149 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb547fff700 1 -- 172.21.15.3:0/2728121796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c004160 con 0x7fb55007be80 2023-08-11T13:54:04.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb547fff700 1 -- 172.21.15.3:0/2728121796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c0042c0 con 0x7fb55007be80 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb547fff700 1 -- 172.21.15.3:0/2728121796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c0045a0 con 0x7fb55007be80 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb55799d700 1 -- 172.21.15.3:0/2728121796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 msgr2=0x7fb55007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.299+0000 7fb55799d700 1 --2- 172.21.15.3:0/2728121796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0098a0 tx=0x7fb54c009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.300+0000 7fb55799d700 1 -- 172.21.15.3:0/2728121796 shutdown_connections 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.300+0000 7fb55799d700 1 --2- 172.21.15.3:0/2728121796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55007a2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:04.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.300+0000 7fb55799d700 1 -- 172.21.15.3:0/2728121796 >> 172.21.15.3:0/2728121796 conn(0x7fb5501016b0 msgr2=0x7fb550103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.300+0000 7fb55799d700 1 -- 172.21.15.3:0/2728121796 shutdown_connections 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.300+0000 7fb55799d700 1 -- 172.21.15.3:0/2728121796 wait complete. 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.301+0000 7fb55799d700 1 Processor -- start 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.301+0000 7fb55799d700 1 -- start start 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.301+0000 7fb55799d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.301+0000 7fb55799d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55019bcd0 con 0x7fb55007be80 2023-08-11T13:54:04.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb555739700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb555739700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52136/0 (socket says 172.21.15.3:52136) 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb555739700 1 -- 172.21.15.3:0/1211583563 learned_addr learned my addr 172.21.15.3:0/1211583563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb555739700 1 -- 172.21.15.3:0/1211583563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009550 con 0x7fb55007be80 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb555739700 1 --2- 172.21.15.3:0/1211583563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb55007b350 tx=0x7fb54c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c011630 con 0x7fb55007be80 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c011790 con 0x7fb55007be80 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55019bed0 con 0x7fb55007be80 2023-08-11T13:54:04.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.302+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54c011a30 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.303+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55019c2f0 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.303+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb54c022070 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.303+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb54c01b790 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.304+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb550195660 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.306+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fb54c01bd40 con 0x7fb55007be80 2023-08-11T13:54:04.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.379+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb55007afe0 con 0x7fb55007be80 2023-08-11T13:54:04.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.383+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb54c011e90 con 0x7fb55007be80 2023-08-11T13:54:04.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.383+0000 7fb5467fc700 1 -- 172.21.15.3:0/1211583563 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fb54c0244d0 con 0x7fb55007be80 2023-08-11T13:54:04.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 msgr2=0x7fb55019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:04.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 --2- 172.21.15.3:0/1211583563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb55007b350 tx=0x7fb54c004750 comp rx=0 tx=0).stop 2023-08-11T13:54:04.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 shutdown_connections 2023-08-11T13:54:04.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 --2- 172.21.15.3:0/1211583563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb55007be80 0x7fb55019b790 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:04.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 >> 172.21.15.3:0/1211583563 conn(0x7fb5501016b0 msgr2=0x7fb5501030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:04.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.384+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 shutdown_connections 2023-08-11T13:54:04.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:02.385+0000 7fb55799d700 1 -- 172.21.15.3:0/1211583563 wait complete. 2023-08-11T13:54:04.080 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2023-08-11T13:54:05.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:04 smithi003 ceph-mon[96683]: from='client.? 172.21.15.3:0/46432681' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:54:05.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.623+0000 7f86f6362700 1 Processor -- start 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.623+0000 7f86f6362700 1 -- start start 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.623+0000 7f86f6362700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.623+0000 7f86f6362700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f007a910 con 0x7f86f007bee0 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52146/0 (socket says 172.21.15.3:52146) 2023-08-11T13:54:05.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86effff700 1 -- 172.21.15.3:0/3484968261 learned_addr learned my addr 172.21.15.3:0/3484968261 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86effff700 1 -- 172.21.15.3:0/3484968261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f007aa50 con 0x7f86f007bee0 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86effff700 1 --2- 172.21.15.3:0/3484968261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86e00098a0 tx=0x7f86e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebc4e11fe628ada server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.624+0000 7f86eeffd700 1 -- 172.21.15.3:0/3484968261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e0004160 con 0x7f86f007bee0 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86eeffd700 1 -- 172.21.15.3:0/3484968261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86e00042c0 con 0x7f86f007bee0 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86eeffd700 1 -- 172.21.15.3:0/3484968261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e0011a00 con 0x7f86f007bee0 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 -- 172.21.15.3:0/3484968261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 msgr2=0x7f86f007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:05.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 --2- 172.21.15.3:0/3484968261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f86e00098a0 tx=0x7f86e0009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 -- 172.21.15.3:0/3484968261 shutdown_connections 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 --2- 172.21.15.3:0/3484968261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f007a340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 -- 172.21.15.3:0/3484968261 >> 172.21.15.3:0/3484968261 conn(0x7f86f01016b0 msgr2=0x7f86f0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 -- 172.21.15.3:0/3484968261 shutdown_connections 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.625+0000 7f86f6362700 1 -- 172.21.15.3:0/3484968261 wait complete. 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.626+0000 7f86f6362700 1 Processor -- start 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86f6362700 1 -- start start 2023-08-11T13:54:05.156 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86f6362700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86f6362700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f019c5a0 con 0x7f86f007bee0 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52150/0 (socket says 172.21.15.3:52150) 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86effff700 1 -- 172.21.15.3:0/46432681 learned_addr learned my addr 172.21.15.3:0/46432681 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.627+0000 7f86effff700 1 -- 172.21.15.3:0/46432681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009550 con 0x7f86f007bee0 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86effff700 1 --2- 172.21.15.3:0/46432681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009fb0 tx=0x7f86e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:05.157 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e0003a40 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86e0003ba0 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86e001a750 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f019c7a0 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f019cbc0 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86e001ac80 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.628+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f86e00193c0 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.629+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f01960b0 con 0x7f86f007bee0 2023-08-11T13:54:05.158 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.632+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f86e002de30 con 0x7f86f007bee0 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.712+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f86f0061f50 con 0x7f86f007bee0 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.713+0000 7f86ed7fa700 1 -- 172.21.15.3:0/46432681 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f86e0024b60 con 0x7f86f007bee0 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 msgr2=0x7f86f019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 --2- 172.21.15.3:0/46432681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009fb0 tx=0x7f86e0011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 shutdown_connections 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 --2- 172.21.15.3:0/46432681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f007bee0 0x7f86f019c060 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:05.159 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 >> 172.21.15.3:0/46432681 conn(0x7f86f01016b0 msgr2=0x7f86f0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:05.160 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 shutdown_connections 2023-08-11T13:54:05.160 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:04.714+0000 7f86f6362700 1 -- 172.21.15.3:0/46432681 wait complete. 2023-08-11T13:54:05.160 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2023-08-11T13:54:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:05 smithi003 systemd[1]: Stopping Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:54:05.896 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:05 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[96658]: 2023-08-11T13:54:05.630+0000 7fb8c1783700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T13:54:05.896 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:05 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[96658]: 2023-08-11T13:54:05.630+0000 7fb8c1783700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-08-11T13:54:06.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:05 smithi003 bash[97024]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a 2023-08-11T13:54:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:06 smithi003 systemd[1]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service: Succeeded. 2023-08-11T13:54:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:06 smithi003 systemd[1]: Stopped Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:54:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:06 smithi003 systemd[1]: Starting Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:54:07.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:06 smithi003 podman[97154]: 2023-08-11T13:54:07.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 bash[97154]: da8973229e7e03612cd37a785845787c7279b0e7a4d5b3f0a61e8fc9b74d8b30 2023-08-11T13:54:07.095 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T13:54:07.174 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: pidfile_write: ignore empty --pid-file 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: load: jerasure load: lrc 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: RocksDB version: 7.9.2 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Git sha 0 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: DB SUMMARY 2023-08-11T13:54:07.175 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: DB Session ID: 7OE93NAP2NSVA6SB7PCE 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: CURRENT file: CURRENT 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98792 ; 2023-08-11T13:54:07.176 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.error_if_exists: 0 2023-08-11T13:54:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.create_if_missing: 0 2023-08-11T13:54:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.paranoid_checks: 1 2023-08-11T13:54:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T13:54:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T13:54:07.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T13:54:07.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.env: 0x55f334e90da0 2023-08-11T13:54:07.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T13:54:07.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.info_log: 0x55f33721f460 2023-08-11T13:54:07.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.statistics: (nil) 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.use_fsync: 0 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_log_file_size: 0 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T13:54:07.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T13:54:07.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_fallocate: 1 2023-08-11T13:54:07.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T13:54:07.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T13:54:07.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.use_direct_reads: 0 2023-08-11T13:54:07.180 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T13:54:07.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T13:54:07.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.db_log_dir: 2023-08-11T13:54:07.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.wal_dir: 2023-08-11T13:54:07.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T13:54:07.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T13:54:07.183 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T13:54:07.184 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.write_buffer_manager: 0x55f3371d83c0 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T13:54:07.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.unordered_write: 0 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.row_cache: None 2023-08-11T13:54:07.186 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.wal_filter: None 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.two_write_queues: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.wal_compression: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.atomic_flush: 0 2023-08-11T13:54:07.187 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T13:54:07.188 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T13:54:07.188 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T13:54:07.188 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.log_readahead_size: 0 2023-08-11T13:54:07.188 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T13:54:07.188 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T13:54:07.189 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T13:54:07.189 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T13:54:07.189 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T13:54:07.189 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T13:54:07.189 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_background_jobs: 2 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_background_compactions: -1 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_subcompactions: 1 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T13:54:07.190 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_open_files: -1 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T13:54:07.191 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_background_flushes: -1 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Compression algorithms supported: 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kZSTD supported: 0 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kXpressCompression supported: 0 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T13:54:07.192 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kZlibCompression supported: 1 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kSnappyCompression supported: 1 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kLZ4Compression supported: 1 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: kBZip2Compression supported: 0 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T13:54:07.193 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.merge_operator: 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_filter: None 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f33721f560) 2023-08-11T13:54:07.194 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2023-08-11T13:54:07.195 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x55f3372771f0 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2023-08-11T13:54:07.196 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2023-08-11T13:54:07.197 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2023-08-11T13:54:07.198 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T13:54:07.199 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression: NoCompression 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.num_levels: 7 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T13:54:07.200 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T13:54:07.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:54:07.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T13:54:07.203 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T13:54:07.204 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T13:54:07.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T13:54:07.206 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.table_properties_collectors: 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.inplace_update_support: 0 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T13:54:07.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.bloom_locality: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.max_successive_merges: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T13:54:07.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.ttl: 2592000 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enable_blob_files: false 2023-08-11T13:54:07.209 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.min_blob_size: 0 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T13:54:07.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 91396897-d654-4c15-89b1-e8449b7e07ff 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762047107644, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762047110453, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 321, "table_properties": {"data_size": 91844, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13547, "raw_average_key_size": 49, "raw_value_size": 84445, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 272, "num_filter_entries": 272, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691762047, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "91396897-d654-4c15-89b1-e8449b7e07ff", "db_session_id": "7OE93NAP2NSVA6SB7PCE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762047110643, "job": 1, "event": "recovery_finished"} 2023-08-11T13:54:07.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f337314000 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: DB pointer 0x55f337300000 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2023-08-11T13:54:07.212 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.213 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 93.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 93.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 50.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.214 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 50.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:54:07.215 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 8.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 8.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T13:54:07.216 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x55f3372771f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???) e1 preinit fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).mds e1 new map 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).mds e1 print_map 2023-08-11T13:54:07.217 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:54:07.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expand map: {default=false} 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta from 'false' to 'false' 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expanded map: {default=false} 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expand map: {default=info} 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta from 'info' to 'info' 2023-08-11T13:54:07.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expanded map: {default=info} 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expand map: {default=daemon} 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expand map: {default=debug} 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: expand_channel_meta expanded map: {default=debug} 2023-08-11T13:54:07.220 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 systemd[1]: Started Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:54:07.221 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-08-11T13:54:07.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mon.a is new leader, mons a in quorum (ranks 0) 2023-08-11T13:54:07.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: monmap e1: 1 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0]} removed_ranks: {} 2023-08-11T13:54:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: fsmap 2023-08-11T13:54:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: osdmap e1: 0 total, 0 up, 0 in 2023-08-11T13:54:07.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:07 smithi003 ceph-mon[97195]: mgrmap e1: no daemons active 2023-08-11T13:54:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.692+0000 7fec4c319700 1 Processor -- start 2023-08-11T13:54:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.692+0000 7fec4c319700 1 -- start start 2023-08-11T13:54:08.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.692+0000 7fec4c319700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.692+0000 7fec4c319700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4407a8b0 con 0x7fec4407be80 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.693+0000 7fec4a0b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.693+0000 7fec4a0b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52154/0 (socket says 172.21.15.3:52154) 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.693+0000 7fec4a0b5700 1 -- 172.21.15.3:0/4147352917 learned_addr learned my addr 172.21.15.3:0/4147352917 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.693+0000 7fec4a0b5700 1 -- 172.21.15.3:0/4147352917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4407a9f0 con 0x7fec4407be80 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.694+0000 7fec4a0b5700 1 --2- 172.21.15.3:0/4147352917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec400098a0 tx=0x7fec40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55865abef9475689 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:08.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.694+0000 7fec490b3700 1 -- 172.21.15.3:0/4147352917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec40004160 con 0x7fec4407be80 2023-08-11T13:54:08.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.694+0000 7fec490b3700 1 -- 172.21.15.3:0/4147352917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec400042c0 con 0x7fec4407be80 2023-08-11T13:54:08.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.694+0000 7fec490b3700 1 -- 172.21.15.3:0/4147352917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec40011a00 con 0x7fec4407be80 2023-08-11T13:54:08.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 -- 172.21.15.3:0/4147352917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 msgr2=0x7fec4407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:08.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 --2- 172.21.15.3:0/4147352917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fec400098a0 tx=0x7fec40009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:08.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 -- 172.21.15.3:0/4147352917 shutdown_connections 2023-08-11T13:54:08.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 --2- 172.21.15.3:0/4147352917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4407a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:08.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 -- 172.21.15.3:0/4147352917 >> 172.21.15.3:0/4147352917 conn(0x7fec441016b0 msgr2=0x7fec44103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:08.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 -- 172.21.15.3:0/4147352917 shutdown_connections 2023-08-11T13:54:08.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.695+0000 7fec4c319700 1 -- 172.21.15.3:0/4147352917 wait complete. 2023-08-11T13:54:08.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.696+0000 7fec4c319700 1 Processor -- start 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.696+0000 7fec4c319700 1 -- start start 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.696+0000 7fec4c319700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.696+0000 7fec4c319700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419c560 con 0x7fec4407be80 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec4a0b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec4a0b5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52168/0 (socket says 172.21.15.3:52168) 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec4a0b5700 1 -- 172.21.15.3:0/180119364 learned_addr learned my addr 172.21.15.3:0/180119364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:08.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec4a0b5700 1 -- 172.21.15.3:0/180119364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40009550 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec4a0b5700 1 --2- 172.21.15.3:0/180119364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec40009130 tx=0x7fec40011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.697+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec40003b70 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec40003cd0 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec400197d0 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4419c760 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4419cb80 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fec400199c0 con 0x7fec4407be80 2023-08-11T13:54:08.271 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.698+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fec4000b720 con 0x7fec4407be80 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.699+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44061f50 con 0x7fec4407be80 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.702+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fec4002a060 con 0x7fec4407be80 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.775+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fec4410c480 con 0x7fec4407be80 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec3b7fe700 1 -- 172.21.15.3:0/180119364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fec40019e00 con 0x7fec4407be80 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 msgr2=0x7fec4419c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec4c319700 1 --2- 172.21.15.3:0/180119364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec40009130 tx=0x7fec40011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:08.272 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 shutdown_connections 2023-08-11T13:54:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec4c319700 1 --2- 172.21.15.3:0/180119364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec4407be80 0x7fec4419c020 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.778+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 >> 172.21.15.3:0/180119364 conn(0x7fec441016b0 msgr2=0x7fec44102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.779+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 shutdown_connections 2023-08-11T13:54:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:07.779+0000 7fec4c319700 1 -- 172.21.15.3:0/180119364 wait complete. 2023-08-11T13:54:08.273 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2023-08-11T13:54:08.296 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-08-11T13:54:08.296 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2023-08-11T13:54:08.296 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 9283 ... 2023-08-11T13:54:08.297 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 8765 ... 2023-08-11T13:54:08.660 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x 2023-08-11T13:54:08.660 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x.service: Unit ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x.service not loaded. 2023-08-11T13:54:08.871 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c.target.wants/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@.service. 2023-08-11T13:54:09.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:08 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/180119364' entity='client.admin' 2023-08-11T13:54:09.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:08 smithi003 systemd[1]: Starting Ceph mgr.x for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:54:09.625 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:09 smithi003 podman[97425]: 2023-08-11T13:54:09.690 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T13:54:09.690 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2023-08-11T13:54:09.707 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T13:54:09.707 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:54:09.708 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2023-08-11T13:54:09.708 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2023-08-11T13:54:09.725 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T13:54:09.725 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2023-08-11T13:54:09.742 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T13:54:09.742 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:54:09.743 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2023-08-11T13:54:09.743 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-08-11T13:54:09.743 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2023-08-11T13:54:09.743 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2023-08-11T13:54:09.912 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:09 smithi003 bash[97425]: af89c8d4f9164420b7ea7f1939cbcacc22fe1b2c73bbb92c09d53432e92cc4be 2023-08-11T13:54:09.912 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:09 smithi003 systemd[1]: Started Ceph mgr.x for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:54:09.912 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:09.801+0000 7f90e86e0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T13:54:10.167 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:09.911+0000 7f90e86e0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T13:54:10.167 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:10.033+0000 7f90e86e0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T13:54:10.168 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:10.166+0000 7f90e86e0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T13:54:10.449 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:10.404+0000 7f90e86e0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T13:54:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/729926500' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-11T13:54:10.826 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:10.826 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:10.826 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T13:54:10.826 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2023-08-11T13:54:10.826 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-11T13:54:10.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-11T13:54:10.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-11T13:54:10.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-11T13:54:10.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-11T13:54:10.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-11T13:54:10.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-11T13:54:10.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-11T13:54:10.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-11T13:54:10.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-11T13:54:10.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-11T13:54:10.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-11T13:54:10.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-11T13:54:10.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-11T13:54:10.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-11T13:54:10.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-11T13:54:10.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:10.840 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2023-08-11T13:54:00.622666+0000", 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-11T13:54:10.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.340+0000 7fc920e89700 1 Processor -- start 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.340+0000 7fc920e89700 1 -- start start 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.340+0000 7fc920e89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.340+0000 7fc920e89700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c079e70 con 0x7fc91c106ba0 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52206/0 (socket says 172.21.15.3:52206) 2023-08-11T13:54:10.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91a59c700 1 -- 172.21.15.3:0/2574562614 learned_addr learned my addr 172.21.15.3:0/2574562614 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91a59c700 1 -- 172.21.15.3:0/2574562614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c1094d0 con 0x7fc91c106ba0 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91a59c700 1 --2- 172.21.15.3:0/2574562614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0098a0 tx=0x7fc90c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960fcf03f78348a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.341+0000 7fc91959a700 1 -- 172.21.15.3:0/2574562614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90c004160 con 0x7fc91c106ba0 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc91959a700 1 -- 172.21.15.3:0/2574562614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc90c0042c0 con 0x7fc91c106ba0 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc91959a700 1 -- 172.21.15.3:0/2574562614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90c011a00 con 0x7fc91c106ba0 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 -- 172.21.15.3:0/2574562614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 msgr2=0x7fc91c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 --2- 172.21.15.3:0/2574562614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0098a0 tx=0x7fc90c009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:10.843 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 -- 172.21.15.3:0/2574562614 shutdown_connections 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 --2- 172.21.15.3:0/2574562614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c108f90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 -- 172.21.15.3:0/2574562614 >> 172.21.15.3:0/2574562614 conn(0x7fc91c100590 msgr2=0x7fc91c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 -- 172.21.15.3:0/2574562614 shutdown_connections 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.342+0000 7fc920e89700 1 -- 172.21.15.3:0/2574562614 wait complete. 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.343+0000 7fc920e89700 1 Processor -- start 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc920e89700 1 -- start start 2023-08-11T13:54:10.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc920e89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc920e89700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c198150 con 0x7fc91c106ba0 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc91a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc91a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52222/0 (socket says 172.21.15.3:52222) 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.344+0000 7fc91a59c700 1 -- 172.21.15.3:0/729926500 learned_addr learned my addr 172.21.15.3:0/729926500 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc91a59c700 1 -- 172.21.15.3:0/729926500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c009550 con 0x7fc91c106ba0 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc91a59c700 1 --2- 172.21.15.3:0/729926500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009130 tx=0x7fc90c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90c003b70 con 0x7fc91c106ba0 2023-08-11T13:54:10.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc90c003cd0 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc90c0197d0 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c198350 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.345+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c198770 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.346+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc90c0199c0 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.346+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc90c00b720 con 0x7fc91c106ba0 2023-08-11T13:54:10.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.346+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c191e90 con 0x7fc91c106ba0 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.349+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc90c02a060 con 0x7fc91c106ba0 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.425+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc91c105230 con 0x7fc91c106ba0 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.426+0000 7fc90b7fe700 1 -- 172.21.15.3:0/729926500 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc90c019db0 con 0x7fc91c106ba0 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 msgr2=0x7fc91c197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 --2- 172.21.15.3:0/729926500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009130 tx=0x7fc90c011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 shutdown_connections 2023-08-11T13:54:10.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 --2- 172.21.15.3:0/729926500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc91c106ba0 0x7fc91c197c10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:10.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 >> 172.21.15.3:0/729926500 conn(0x7fc91c100590 msgr2=0x7fc91c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:10.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 shutdown_connections 2023-08-11T13:54:10.848 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:10.427+0000 7fc920e89700 1 -- 172.21.15.3:0/729926500 wait complete. 2023-08-11T13:54:10.850 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2023-08-11T13:54:11.190 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:10.903+0000 7f90e86e0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T13:54:11.190 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:11.069+0000 7f90e86e0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T13:54:11.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:11.189+0000 7f90e86e0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T13:54:11.542 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:11.318+0000 7f90e86e0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T13:54:12.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:11.891+0000 7f90e86e0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T13:54:13.088 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:12.819+0000 7f90e86e0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T13:54:13.089 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:12.934+0000 7f90e86e0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T13:54:13.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:13 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/986868300' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-11T13:54:14.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:13.960+0000 7f90e86e0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:14.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:14.085+0000 7f90e86e0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T13:54:14.700 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:14.318+0000 7f90e86e0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:14.995 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:14.698+0000 7f90e86e0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T13:54:15.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:14.993+0000 7f90e86e0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T13:54:15.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:15.194+0000 7f90e86e0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T13:54:16.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:15.570+0000 7f90e86e0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T13:54:16.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:16.533+0000 7f90e86e0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T13:54:17.122 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:16.713+0000 7f90e86e0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T13:54:17.122 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:16.969+0000 7f90e86e0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T13:54:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:17.120+0000 7f90e86e0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T13:54:17.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:17.407+0000 7f90e86e0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T13:54:18.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:18.517+0000 7f90e86e0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T13:54:18.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: Activating manager daemon x 2023-08-11T13:54:18.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: mgrmap e2: x(active, starting, since 0.00417799s) 2023-08-11T13:54:18.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T13:54:18.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T13:54:18.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T13:54:18.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:54:18.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T13:54:18.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: Manager daemon x is now available 2023-08-11T13:54:18.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T13:54:18.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T13:54:18.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' 2023-08-11T13:54:18.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' 2023-08-11T13:54:18.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:18 smithi003 ceph-mon[97195]: from='mgr.14100 172.21.15.3:0/604294826' entity='mgr.x' 2023-08-11T13:54:20.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-11T13:54:20.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2023-08-11T13:54:20.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-11T13:54:20.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:20.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-11T13:54:20.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-11T13:54:20.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-11T13:54:20.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-11T13:54:20.340 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-11T13:54:20.341 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-11T13:54:20.342 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-11T13:54:20.342 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-11T13:54:20.342 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-11T13:54:20.342 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-11T13:54:20.342 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2023-08-11T13:54:20.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2023-08-11T13:54:00.622666+0000", 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:20.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.353+0000 7f3c771d1700 1 Processor -- start 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.354+0000 7f3c771d1700 1 -- start start 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.354+0000 7f3c771d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.354+0000 7f3c771d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7007a8d0 con 0x7f3c7007bea0 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c74f6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:20.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c74f6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34648/0 (socket says 172.21.15.3:34648) 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c74f6d700 1 -- 172.21.15.3:0/646668664 learned_addr learned my addr 172.21.15.3:0/646668664 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c74f6d700 1 -- 172.21.15.3:0/646668664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7007aa10 con 0x7f3c7007bea0 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c74f6d700 1 --2- 172.21.15.3:0/646668664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c640098a0 tx=0x7f3c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f57293b495404b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.355+0000 7f3c6f7fe700 1 -- 172.21.15.3:0/646668664 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c64004160 con 0x7f3c7007bea0 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c6f7fe700 1 -- 172.21.15.3:0/646668664 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c640042c0 con 0x7f3c7007bea0 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c6f7fe700 1 -- 172.21.15.3:0/646668664 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c64011a00 con 0x7f3c7007bea0 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 -- 172.21.15.3:0/646668664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 msgr2=0x7f3c7007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:20.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 --2- 172.21.15.3:0/646668664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3c640098a0 tx=0x7f3c64009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 -- 172.21.15.3:0/646668664 shutdown_connections 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 --2- 172.21.15.3:0/646668664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7007a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 -- 172.21.15.3:0/646668664 >> 172.21.15.3:0/646668664 conn(0x7f3c701016b0 msgr2=0x7f3c70103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 -- 172.21.15.3:0/646668664 shutdown_connections 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.356+0000 7f3c771d1700 1 -- 172.21.15.3:0/646668664 wait complete. 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.358+0000 7f3c771d1700 1 Processor -- start 2023-08-11T13:54:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.358+0000 7f3c771d1700 1 -- start start 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c771d1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c771d1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7019c560 con 0x7f3c7007bea0 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c74f6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c74f6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34656/0 (socket says 172.21.15.3:34656) 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c74f6d700 1 -- 172.21.15.3:0/986868300 learned_addr learned my addr 172.21.15.3:0/986868300 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.359+0000 7f3c74f6d700 1 -- 172.21.15.3:0/986868300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64009550 con 0x7f3c7007bea0 2023-08-11T13:54:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c74f6d700 1 --2- 172.21.15.3:0/986868300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009130 tx=0x7f3c64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c64003b70 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c64003cd0 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7019c760 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c640197d0 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7019cb80 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3c64019a10 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.360+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3c6400b720 con 0x7f3c7007bea0 2023-08-11T13:54:20.350 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.361+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7010c4c0 con 0x7f3c7007bea0 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.364+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f3c6402a060 con 0x7f3c7007bea0 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.441+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3c7019f770 con 0x7f3c7007bea0 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.442+0000 7f3c6dffb700 1 -- 172.21.15.3:0/986868300 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3c64019e50 con 0x7f3c7007bea0 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 msgr2=0x7f3c7019c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 --2- 172.21.15.3:0/986868300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009130 tx=0x7f3c64011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 shutdown_connections 2023-08-11T13:54:20.351 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 --2- 172.21.15.3:0/986868300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c7007bea0 0x7f3c7019c020 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:20.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 >> 172.21.15.3:0/986868300 conn(0x7f3c701016b0 msgr2=0x7f3c70102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:20.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 shutdown_connections 2023-08-11T13:54:20.352 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:13.443+0000 7f3c771d1700 1 -- 172.21.15.3:0/986868300 wait complete. 2023-08-11T13:54:20.352 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2023-08-11T13:54:20.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:20 smithi003 ceph-mon[97195]: mgrmap e3: x(active, since 1.00994s) 2023-08-11T13:54:21.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:21 smithi003 ceph-mon[97195]: mgrmap e4: x(active, since 2s) 2023-08-11T13:54:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:23 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3689350525' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-08-11T13:54:23.621 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:23.621 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.622 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-08-11T13:54:23.623 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-08-11T13:54:23.624 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2023-08-11T13:54:23.624 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:23.624 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-08-11T13:54:23.624 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2023-08-11T13:54:23.624 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:23.625 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-08-11T13:54:23.625 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2023-08-11T13:54:23.625 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:23.625 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-08-11T13:54:23.626 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-08-11T13:54:23.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-08-11T13:54:23.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-08-11T13:54:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2023-08-11T13:54:23.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2023-08-11T13:54:00.622666+0000", 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-08-11T13:54:23.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.907+0000 7f6d12e7e700 1 Processor -- start 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d12e7e700 1 -- start start 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d12e7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d12e7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c106ac0 con 0x7f6d0c1060d0 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d10c1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d10c1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40380/0 (socket says 172.21.15.3:40380) 2023-08-11T13:54:23.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.908+0000 7f6d10c1a700 1 -- 172.21.15.3:0/2503086784 learned_addr learned my addr 172.21.15.3:0/2503086784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.909+0000 7f6d10c1a700 1 -- 172.21.15.3:0/2503086784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c1072d0 con 0x7f6d0c1060d0 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.909+0000 7f6d10c1a700 1 --2- 172.21.15.3:0/2503086784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098a0 tx=0x7f6d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4632abaab87f2d63 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.909+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/2503086784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00004160 con 0x7f6d0c1060d0 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.909+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/2503086784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d000042c0 con 0x7f6d0c1060d0 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.909+0000 7f6d0b7fe700 1 -- 172.21.15.3:0/2503086784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00011a00 con 0x7f6d0c1060d0 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 -- 172.21.15.3:0/2503086784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 msgr2=0x7f6d0c1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/2503086784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d000098a0 tx=0x7f6d00009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:23.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 -- 172.21.15.3:0/2503086784 shutdown_connections 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/2503086784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c1064f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 -- 172.21.15.3:0/2503086784 >> 172.21.15.3:0/2503086784 conn(0x7f6d0c1016b0 msgr2=0x7f6d0c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 -- 172.21.15.3:0/2503086784 shutdown_connections 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.910+0000 7f6d12e7e700 1 -- 172.21.15.3:0/2503086784 wait complete. 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.911+0000 7f6d12e7e700 1 Processor -- start 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.911+0000 7f6d12e7e700 1 -- start start 2023-08-11T13:54:23.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d12e7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d12e7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c19c570 con 0x7f6d0c1060d0 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d10c1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d10c1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40386/0 (socket says 172.21.15.3:40386) 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d10c1a700 1 -- 172.21.15.3:0/3689350525 learned_addr learned my addr 172.21.15.3:0/3689350525 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d10c1a700 1 -- 172.21.15.3:0/3689350525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00009550 con 0x7f6d0c1060d0 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.912+0000 7f6d10c1a700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fb0 tx=0x7f6d00011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.913+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d00003a40 con 0x7f6d0c1060d0 2023-08-11T13:54:23.642 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.913+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d00003ba0 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.913+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c19c770 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.913+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0001a750 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.913+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c19cb90 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.914+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f6d00024b60 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.914+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c061f50 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.914+0000 7f6d09ffb700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 0x7f6cfc03a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.914+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d0004be10 con 0x7f6d0c1060d0 2023-08-11T13:54:23.643 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.915+0000 7f6d0bfff700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 0x7f6cfc03a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.915+0000 7f6d0bfff700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 0x7f6cfc03a530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:22.920+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6d0004f030 con 0x7f6d0c1060d0 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.212+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6d0c10a5a0 con 0x7f6d0c1060d0 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.212+0000 7f6d09ffb700 1 -- 172.21.15.3:0/3689350525 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6d00024430 con 0x7f6d0c1060d0 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 msgr2=0x7f6cfc03a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 0x7f6cfc03a530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:23.644 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 msgr2=0x7f6d0c19c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009fb0 tx=0x7f6d00011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 shutdown_connections 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f6cfc038070 0x7f6cfc03a530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 --2- 172.21.15.3:0/3689350525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d0c1060d0 0x7f6d0c19c030 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.214+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 >> 172.21.15.3:0/3689350525 conn(0x7f6d0c1016b0 msgr2=0x7f6d0c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.215+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 shutdown_connections 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:23.215+0000 7f6d12e7e700 1 -- 172.21.15.3:0/3689350525 wait complete. 2023-08-11T13:54:23.645 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2023-08-11T13:54:24.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:24 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1794188120' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-08-11T13:54:24.910 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:24.910 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2023-08-11T13:54:24.910 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:24.910 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-08-11T13:54:24.910 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-08-11T13:54:24.911 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-08-11T13:54:24.911 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-08-11T13:54:24.911 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:24.912 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2023-08-11T13:54:24.912 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-08-11T13:54:24.912 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2023-08-11T13:54:24.913 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2023-08-11T13:54:24.913 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-08-11T13:54:24.913 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-08-11T13:54:24.913 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-08-11T13:54:24.914 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-08-11T13:54:24.914 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-08-11T13:54:24.914 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.201+0000 7f5968af7700 1 Processor -- start 2023-08-11T13:54:24.914 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.201+0000 7f5968af7700 1 -- start start 2023-08-11T13:54:24.915 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.202+0000 7f5968af7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:24.915 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.202+0000 7f5968af7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596407a8d0 con 0x7f596407bea0 2023-08-11T13:54:24.916 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.202+0000 7f596259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:24.916 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.202+0000 7f596259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40390/0 (socket says 172.21.15.3:40390) 2023-08-11T13:54:24.917 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.202+0000 7f596259c700 1 -- 172.21.15.3:0/2117270909 learned_addr learned my addr 172.21.15.3:0/2117270909 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:24.917 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.203+0000 7f596259c700 1 -- 172.21.15.3:0/2117270909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596407aa10 con 0x7f596407bea0 2023-08-11T13:54:24.917 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.203+0000 7f596259c700 1 --2- 172.21.15.3:0/2117270909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f59540098a0 tx=0x7f5954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c8fd168dd4105 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:24.918 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.203+0000 7f596159a700 1 -- 172.21.15.3:0/2117270909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5954004160 con 0x7f596407bea0 2023-08-11T13:54:24.918 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.203+0000 7f596159a700 1 -- 172.21.15.3:0/2117270909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59540042c0 con 0x7f596407bea0 2023-08-11T13:54:24.918 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.203+0000 7f596159a700 1 -- 172.21.15.3:0/2117270909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5954011a00 con 0x7f596407bea0 2023-08-11T13:54:24.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 -- 172.21.15.3:0/2117270909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 msgr2=0x7f596407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:24.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 --2- 172.21.15.3:0/2117270909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f59540098a0 tx=0x7f5954009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:24.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 -- 172.21.15.3:0/2117270909 shutdown_connections 2023-08-11T13:54:24.919 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 --2- 172.21.15.3:0/2117270909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596407a300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:24.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 -- 172.21.15.3:0/2117270909 >> 172.21.15.3:0/2117270909 conn(0x7f59641016b0 msgr2=0x7f5964103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:24.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 -- 172.21.15.3:0/2117270909 shutdown_connections 2023-08-11T13:54:24.920 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.204+0000 7f5968af7700 1 -- 172.21.15.3:0/2117270909 wait complete. 2023-08-11T13:54:24.921 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.205+0000 7f5968af7700 1 Processor -- start 2023-08-11T13:54:24.921 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.205+0000 7f5968af7700 1 -- start start 2023-08-11T13:54:24.921 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.205+0000 7f5968af7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:24.922 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.205+0000 7f5968af7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596419c560 con 0x7f596407bea0 2023-08-11T13:54:24.922 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.205+0000 7f596259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:24.922 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f596259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40398/0 (socket says 172.21.15.3:40398) 2023-08-11T13:54:24.923 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f596259c700 1 -- 172.21.15.3:0/1794188120 learned_addr learned my addr 172.21.15.3:0/1794188120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:24.923 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f596259c700 1 -- 172.21.15.3:0/1794188120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954009550 con 0x7f596407bea0 2023-08-11T13:54:24.923 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f596259c700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5954009130 tx=0x7f5954011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:24.923 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5954003b70 con 0x7f596407bea0 2023-08-11T13:54:24.924 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5954003cd0 con 0x7f596407bea0 2023-08-11T13:54:24.924 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59540197d0 con 0x7f596407bea0 2023-08-11T13:54:24.924 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596419c760 con 0x7f596407bea0 2023-08-11T13:54:24.925 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.206+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596419cb80 con 0x7f596407bea0 2023-08-11T13:54:24.925 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.207+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596410c4c0 con 0x7f596407bea0 2023-08-11T13:54:24.925 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.208+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f595401abb0 con 0x7f596407bea0 2023-08-11T13:54:24.925 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.208+0000 7f59537fe700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 0x7f594c03a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:24.926 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.208+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f595404c0c0 con 0x7f596407bea0 2023-08-11T13:54:24.926 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.208+0000 7f5961d9b700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 0x7f594c03a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:24.926 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.209+0000 7f5961d9b700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 0x7f594c03a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5958006fd0 tx=0x7f5958006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:24.927 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.213+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f595407ec30 con 0x7f596407bea0 2023-08-11T13:54:24.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.420+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f596419f5c0 con 0x7f596407bea0 2023-08-11T13:54:24.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.421+0000 7f59537fe700 1 -- 172.21.15.3:0/1794188120 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f595401a430 con 0x7f596407bea0 2023-08-11T13:54:24.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 msgr2=0x7f594c03a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:24.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 0x7f594c03a5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5958006fd0 tx=0x7f5958006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:24.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 msgr2=0x7f596419c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:24.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5954009130 tx=0x7f5954011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:24.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 shutdown_connections 2023-08-11T13:54:24.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f594c038110 0x7f594c03a5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:24.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 --2- 172.21.15.3:0/1794188120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f596407bea0 0x7f596419c020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:24.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 >> 172.21.15.3:0/1794188120 conn(0x7f59641016b0 msgr2=0x7f5964102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:24.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 shutdown_connections 2023-08-11T13:54:24.932 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:24.424+0000 7f5968af7700 1 -- 172.21.15.3:0/1794188120 wait complete. 2023-08-11T13:54:24.932 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2023-08-11T13:54:26.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:25 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1700811357' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-08-11T13:54:26.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: ignoring --setuser ceph since I am not root 2023-08-11T13:54:26.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: ignoring --setgroup ceph since I am not root 2023-08-11T13:54:26.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.442+0000 7f67101cb700 1 Processor -- start 2023-08-11T13:54:26.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f67101cb700 1 -- start start 2023-08-11T13:54:26.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f67101cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:26.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f67101cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708106ac0 con 0x7f67081060d0 2023-08-11T13:54:26.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f670df67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:26.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f670df67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40410/0 (socket says 172.21.15.3:40410) 2023-08-11T13:54:26.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.443+0000 7f670df67700 1 -- 172.21.15.3:0/3631561996 learned_addr learned my addr 172.21.15.3:0/3631561996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:26.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.444+0000 7f670df67700 1 -- 172.21.15.3:0/3631561996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67081072d0 con 0x7f67081060d0 2023-08-11T13:54:26.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.444+0000 7f670df67700 1 --2- 172.21.15.3:0/3631561996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67040098a0 tx=0x7f6704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e7f430b104c409 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:26.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.444+0000 7f670cf65700 1 -- 172.21.15.3:0/3631561996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6704004160 con 0x7f67081060d0 2023-08-11T13:54:26.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.444+0000 7f670cf65700 1 -- 172.21.15.3:0/3631561996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67040042c0 con 0x7f67081060d0 2023-08-11T13:54:26.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.444+0000 7f670cf65700 1 -- 172.21.15.3:0/3631561996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6704011a00 con 0x7f67081060d0 2023-08-11T13:54:26.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 -- 172.21.15.3:0/3631561996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 msgr2=0x7f67081064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:26.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 --2- 172.21.15.3:0/3631561996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67040098a0 tx=0x7f6704009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:26.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 -- 172.21.15.3:0/3631561996 shutdown_connections 2023-08-11T13:54:26.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 --2- 172.21.15.3:0/3631561996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f67081064f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 -- 172.21.15.3:0/3631561996 >> 172.21.15.3:0/3631561996 conn(0x7f67081016b0 msgr2=0x7f6708103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 -- 172.21.15.3:0/3631561996 shutdown_connections 2023-08-11T13:54:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.445+0000 7f67101cb700 1 -- 172.21.15.3:0/3631561996 wait complete. 2023-08-11T13:54:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.446+0000 7f67101cb700 1 Processor -- start 2023-08-11T13:54:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.446+0000 7f67101cb700 1 -- start start 2023-08-11T13:54:26.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.446+0000 7f67101cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:26.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.446+0000 7f67101cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670819c570 con 0x7f67081060d0 2023-08-11T13:54:26.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f670df67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:26.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f670df67700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40422/0 (socket says 172.21.15.3:40422) 2023-08-11T13:54:26.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f670df67700 1 -- 172.21.15.3:0/1700811357 learned_addr learned my addr 172.21.15.3:0/1700811357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:26.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f670df67700 1 -- 172.21.15.3:0/1700811357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009550 con 0x7f67081060d0 2023-08-11T13:54:26.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f670df67700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6704009130 tx=0x7f6704011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:26.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6704003ab0 con 0x7f67081060d0 2023-08-11T13:54:26.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6704003c10 con 0x7f67081060d0 2023-08-11T13:54:26.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67040247d0 con 0x7f67081060d0 2023-08-11T13:54:26.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.447+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670819c770 con 0x7f67081060d0 2023-08-11T13:54:26.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.448+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670819cb90 con 0x7f67081060d0 2023-08-11T13:54:26.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.449+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708061f50 con 0x7f67081060d0 2023-08-11T13:54:26.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.449+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 45124+0+0 (secure 0 0 0) 0x7f670401abb0 con 0x7f67081060d0 2023-08-11T13:54:26.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.449+0000 7f66feffd700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 0x7f66f403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:26.302 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.449+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f670404bc80 con 0x7f67081060d0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.449+0000 7f670d766700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 0x7f66f403a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.450+0000 7f670d766700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 0x7f66f403a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66f8006fd0 tx=0x7f66f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.454+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6704018710 con 0x7f67081060d0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.721+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f670810a5a0 con 0x7f67081060d0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.774+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f670402d770 con 0x7f67081060d0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.776+0000 7f66feffd700 1 -- 172.21.15.3:0/1700811357 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f670404b9e0 con 0x7f67081060d0 2023-08-11T13:54:26.303 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 msgr2=0x7f66f403a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 0x7f66f403a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66f8006fd0 tx=0x7f66f8006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 msgr2=0x7f670819c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6704009130 tx=0x7f6704011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 shutdown_connections 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f66f4038110 0x7f66f403a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:26.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 --2- 172.21.15.3:0/1700811357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67081060d0 0x7f670819c030 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6704009130 tx=0x7f6704011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:26.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.779+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 >> 172.21.15.3:0/1700811357 conn(0x7f67081016b0 msgr2=0x7f6708102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:26.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.780+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 shutdown_connections 2023-08-11T13:54:26.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:25.780+0000 7f67101cb700 1 -- 172.21.15.3:0/1700811357 wait complete. 2023-08-11T13:54:26.305 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:26.038+0000 7fb8b8a41200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T13:54:26.305 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:26.146+0000 7fb8b8a41200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T13:54:26.305 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:26.264+0000 7fb8b8a41200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T13:54:26.779 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:26.396+0000 7fb8b8a41200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T13:54:26.779 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:26.635+0000 7fb8b8a41200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T13:54:27.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1700811357' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-08-11T13:54:27.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:26 smithi003 ceph-mon[97195]: mgrmap e5: x(active, since 7s) 2023-08-11T13:54:27.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:27.115+0000 7fb8b8a41200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T13:54:27.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:27.288+0000 7fb8b8a41200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T13:54:27.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:27.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-08-11T13:54:27.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T13:54:27.569 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-11T13:54:27.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-11T13:54:27.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:27.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.829+0000 7f577502e700 1 Processor -- start 2023-08-11T13:54:27.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.829+0000 7f577502e700 1 -- start start 2023-08-11T13:54:27.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.829+0000 7f577502e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:27.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.829+0000 7f577502e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577007a8d0 con 0x7f577007bea0 2023-08-11T13:54:27.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.830+0000 7f576ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:27.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.830+0000 7f576ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40462/0 (socket says 172.21.15.3:40462) 2023-08-11T13:54:27.582 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.830+0000 7f576ed9d700 1 -- 172.21.15.3:0/325927083 learned_addr learned my addr 172.21.15.3:0/325927083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:27.582 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.830+0000 7f576ed9d700 1 -- 172.21.15.3:0/325927083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577007aa10 con 0x7f577007bea0 2023-08-11T13:54:27.582 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.830+0000 7f576ed9d700 1 --2- 172.21.15.3:0/325927083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57600098a0 tx=0x7f5760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b709674cb8a1d17 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:27.583 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f576dd9b700 1 -- 172.21.15.3:0/325927083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5760004160 con 0x7f577007bea0 2023-08-11T13:54:27.583 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f576dd9b700 1 -- 172.21.15.3:0/325927083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57600042c0 con 0x7f577007bea0 2023-08-11T13:54:27.583 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f576dd9b700 1 -- 172.21.15.3:0/325927083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5760011a00 con 0x7f577007bea0 2023-08-11T13:54:27.584 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f577502e700 1 -- 172.21.15.3:0/325927083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 msgr2=0x7f577007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:27.584 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f577502e700 1 --2- 172.21.15.3:0/325927083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f57600098a0 tx=0x7f5760009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.584 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.831+0000 7f577502e700 1 -- 172.21.15.3:0/325927083 shutdown_connections 2023-08-11T13:54:27.584 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.832+0000 7f577502e700 1 --2- 172.21.15.3:0/325927083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577007a300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.585 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.832+0000 7f577502e700 1 -- 172.21.15.3:0/325927083 >> 172.21.15.3:0/325927083 conn(0x7f57701016b0 msgr2=0x7f5770103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:27.585 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.832+0000 7f577502e700 1 -- 172.21.15.3:0/325927083 shutdown_connections 2023-08-11T13:54:27.585 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.832+0000 7f577502e700 1 -- 172.21.15.3:0/325927083 wait complete. 2023-08-11T13:54:27.585 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.833+0000 7f577502e700 1 Processor -- start 2023-08-11T13:54:27.585 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.833+0000 7f577502e700 1 -- start start 2023-08-11T13:54:27.586 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.833+0000 7f577502e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:27.586 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.833+0000 7f577502e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019c560 con 0x7f577007bea0 2023-08-11T13:54:27.586 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.833+0000 7f576ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:27.587 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f576ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40466/0 (socket says 172.21.15.3:40466) 2023-08-11T13:54:27.587 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f576ed9d700 1 -- 172.21.15.3:0/882859215 learned_addr learned my addr 172.21.15.3:0/882859215 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:27.587 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f576ed9d700 1 -- 172.21.15.3:0/882859215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760009550 con 0x7f577007bea0 2023-08-11T13:54:27.588 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f576ed9d700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5760009130 tx=0x7f5760011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:27.588 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5760003b70 con 0x7f577007bea0 2023-08-11T13:54:27.588 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577019c760 con 0x7f577007bea0 2023-08-11T13:54:27.588 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5760003cd0 con 0x7f577007bea0 2023-08-11T13:54:27.588 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.834+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57600197d0 con 0x7f577007bea0 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.835+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577019cb80 con 0x7f577007bea0 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.836+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f5760019a10 con 0x7f577007bea0 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.836+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5770061f50 con 0x7f577007bea0 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.836+0000 7f5757fff700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 0x7f575803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.836+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f576004c350 con 0x7f577007bea0 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.837+0000 7f576e59c700 1 -- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 msgr2=0x7f575803a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:27.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.837+0000 7f576e59c700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 0x7f575803a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:54:27.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:26.842+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5760018db0 con 0x7f577007bea0 2023-08-11T13:54:27.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.038+0000 7f576e59c700 1 -- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 msgr2=0x7f575803a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:27.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.038+0000 7f576e59c700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 0x7f575803a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T13:54:27.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.106+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f577010c4c0 con 0x7f577007bea0 2023-08-11T13:54:27.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.106+0000 7f5757fff700 1 -- 172.21.15.3:0/882859215 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5760018350 con 0x7f577007bea0 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.108+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 msgr2=0x7f575803a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.108+0000 7f577502e700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 0x7f575803a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.108+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 msgr2=0x7f577019c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.108+0000 7f577502e700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5760009130 tx=0x7f5760011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 shutdown_connections 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7f5758038110 0x7f575803a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 --2- 172.21.15.3:0/882859215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f577007bea0 0x7f577019c020 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 >> 172.21.15.3:0/882859215 conn(0x7f57701016b0 msgr2=0x7f5770102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 shutdown_connections 2023-08-11T13:54:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:27.109+0000 7f577502e700 1 -- 172.21.15.3:0/882859215 wait complete. 2023-08-11T13:54:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2023-08-11T13:54:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2023-08-11T13:54:27.779 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:27.409+0000 7fb8b8a41200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T13:54:27.780 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:27.542+0000 7fb8b8a41200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T13:54:28.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:27 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/882859215' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-11T13:54:28.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:28.164+0000 7fb8b8a41200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T13:54:29.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:29.034+0000 7fb8b8a41200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T13:54:29.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:29.143+0000 7fb8b8a41200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T13:54:30.481 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:30.127+0000 7fb8b8a41200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:30.482 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:30.250+0000 7fb8b8a41200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T13:54:30.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:30.479+0000 7fb8b8a41200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:31.132 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:30.850+0000 7fb8b8a41200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T13:54:31.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:31.130+0000 7fb8b8a41200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T13:54:31.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:31.330+0000 7fb8b8a41200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T13:54:32.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:31.684+0000 7fb8b8a41200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T13:54:33.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:32.636+0000 7fb8b8a41200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T13:54:33.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:32.814+0000 7fb8b8a41200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T13:54:33.504 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:33.069+0000 7fb8b8a41200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T13:54:33.504 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:33.220+0000 7fb8b8a41200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T13:54:33.504 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:33.502+0000 7fb8b8a41200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T13:54:35.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:34.600+0000 7fb8b8a41200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T13:54:35.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: Active manager daemon x restarted 2023-08-11T13:54:35.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: Activating manager daemon x 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: osdmap e2: 0 total, 0 up, 0 in 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: mgrmap e6: x(active, starting, since 0.00384395s) 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T13:54:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T13:54:35.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: Manager daemon x is now available 2023-08-11T13:54:35.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:35.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:35.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:35.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:34 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:36.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: Found migration_current of "None". Setting to last migration. 2023-08-11T13:54:36.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:36.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T13:54:36.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T13:54:36.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:36.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:36.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: mgrmap e7: x(active, since 1.00673s) 2023-08-11T13:54:36.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:35 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:36.102 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:36.102 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-08-11T13:54:36.102 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.118+0000 7fd02c6f4700 1 Processor -- start 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.118+0000 7fd02c6f4700 1 -- start start 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02c6f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02c6f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02407a910 con 0x7fd02407bee0 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02a490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02a490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40468/0 (socket says 172.21.15.3:40468) 2023-08-11T13:54:36.103 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02a490700 1 -- 172.21.15.3:0/4241825619 learned_addr learned my addr 172.21.15.3:0/4241825619 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.119+0000 7fd02a490700 1 -- 172.21.15.3:0/4241825619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02407aa50 con 0x7fd02407bee0 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.120+0000 7fd02a490700 1 --2- 172.21.15.3:0/4241825619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0200098a0 tx=0x7fd020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c65c27ce7f0dff9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.120+0000 7fd02948e700 1 -- 172.21.15.3:0/4241825619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd020004160 con 0x7fd02407bee0 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.120+0000 7fd02948e700 1 -- 172.21.15.3:0/4241825619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0200042c0 con 0x7fd02407bee0 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.120+0000 7fd02948e700 1 -- 172.21.15.3:0/4241825619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd020011a00 con 0x7fd02407bee0 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 -- 172.21.15.3:0/4241825619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 msgr2=0x7fd02407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:36.104 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/4241825619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd0200098a0 tx=0x7fd020009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 -- 172.21.15.3:0/4241825619 shutdown_connections 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/4241825619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02407a340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 -- 172.21.15.3:0/4241825619 >> 172.21.15.3:0/4241825619 conn(0x7fd0241016b0 msgr2=0x7fd024103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 -- 172.21.15.3:0/4241825619 shutdown_connections 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.121+0000 7fd02c6f4700 1 -- 172.21.15.3:0/4241825619 wait complete. 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.122+0000 7fd02c6f4700 1 Processor -- start 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.122+0000 7fd02c6f4700 1 -- start start 2023-08-11T13:54:36.105 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02c6f4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02c6f4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02419c5a0 con 0x7fd02407bee0 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02a490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02a490700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40484/0 (socket says 172.21.15.3:40484) 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02a490700 1 -- 172.21.15.3:0/1014422724 learned_addr learned my addr 172.21.15.3:0/1014422724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02a490700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020009550 con 0x7fd02407bee0 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.123+0000 7fd02a490700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd020009fb0 tx=0x7fd020011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:36.106 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.124+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd020003a40 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.124+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd020003ba0 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.124+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd02001a750 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.124+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02419c7a0 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.124+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02419cbc0 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.125+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fd02001a980 con 0x7fd02407bee0 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.126+0000 7fd01b7fe700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.126+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd01003ace0 con 0x7fd010038110 2023-08-11T13:54:36.107 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.126+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd02004bd40 con 0x7fd02407bee0 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.126+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.126+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.327+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.327+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.728+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:28.728+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-11T13:54:36.108 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:29.529+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:29.529+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:31.130+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:31.131+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:34.334+0000 7fd029c8f700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/586553398 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:34.334+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:34.605+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 6) v1 ==== 44902+0+0 (secure 0 0 0) 0x7fd02002d910 con 0x7fd02407bee0 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:34.605+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 msgr2=0x7fd01003a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:54:36.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:34.605+0000 7fd01b7fe700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.607+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7fd02004d9a0 con 0x7fd02407bee0 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.607+0000 7fd01b7fe700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 0x7fd01003df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.607+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd01003ace0 con 0x7fd01003bb50 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.608+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 0x7fd01003df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.609+0000 7fd029c8f700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 0x7fd01003df40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd014003a10 tx=0x7fd0140092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.610+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd01003ace0 con 0x7fd01003bb50 2023-08-11T13:54:36.110 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.617+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd02419d3a0 con 0x7fd01003bb50 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.617+0000 7fd01b7fe700 1 -- 172.21.15.3:0/1014422724 <== mgr.14118 v2:172.21.15.3:6800/1168721553 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd02419d3a0 con 0x7fd01003bb50 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 msgr2=0x7fd01003df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 0x7fd01003df40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd014003a10 tx=0x7fd0140092b0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 msgr2=0x7fd02419c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd020009fb0 tx=0x7fd020011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 shutdown_connections 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd01003bb50 0x7fd01003df40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.111 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd02407bee0 0x7fd02419c060 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 --2- 172.21.15.3:0/1014422724 >> [v2:172.21.15.3:6800/586553398,v1:172.21.15.3:6801/586553398] conn(0x7fd010038110 0x7fd01003a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 >> 172.21.15.3:0/1014422724 conn(0x7fd0241016b0 msgr2=0x7fd024102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 shutdown_connections 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:35.618+0000 7fd02c6f4700 1 -- 172.21.15.3:0/1014422724 wait complete. 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2023-08-11T13:54:36.112 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2023-08-11T13:54:37.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: [11/Aug/2023:13:54:35] ENGINE Bus STARTING 2023-08-11T13:54:37.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: [11/Aug/2023:13:54:35] ENGINE Serving on http://172.21.15.3:8765 2023-08-11T13:54:37.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: [11/Aug/2023:13:54:35] ENGINE Serving on https://172.21.15.3:7150 2023-08-11T13:54:37.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: [11/Aug/2023:13:54:35] ENGINE Bus STARTED 2023-08-11T13:54:37.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-11T13:54:37.033 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:36 smithi003 ceph-mon[97195]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-11T13:54:37.281 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.617+0000 7fd36ebb9700 1 Processor -- start 2023-08-11T13:54:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.617+0000 7fd36ebb9700 1 -- start start 2023-08-11T13:54:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.617+0000 7fd36ebb9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.617+0000 7fd36ebb9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368106ac0 con 0x7fd3681060d0 2023-08-11T13:54:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.618+0000 7fd36c955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.618+0000 7fd36c955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48492/0 (socket says 172.21.15.3:48492) 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.618+0000 7fd36c955700 1 -- 172.21.15.3:0/2057877183 learned_addr learned my addr 172.21.15.3:0/2057877183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.618+0000 7fd36c955700 1 -- 172.21.15.3:0/2057877183 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3681072d0 con 0x7fd3681060d0 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.618+0000 7fd36c955700 1 --2- 172.21.15.3:0/2057877183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098a0 tx=0x7fd35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae07bb4a433cbbb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd3677fe700 1 -- 172.21.15.3:0/2057877183 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd35c004160 con 0x7fd3681060d0 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd3677fe700 1 -- 172.21.15.3:0/2057877183 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd35c0042c0 con 0x7fd3681060d0 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd3677fe700 1 -- 172.21.15.3:0/2057877183 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd35c011a00 con 0x7fd3681060d0 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 -- 172.21.15.3:0/2057877183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 msgr2=0x7fd3681064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:37.283 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/2057877183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0098a0 tx=0x7fd35c009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:37.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 -- 172.21.15.3:0/2057877183 shutdown_connections 2023-08-11T13:54:37.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/2057877183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd3681064f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:37.284 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 -- 172.21.15.3:0/2057877183 >> 172.21.15.3:0/2057877183 conn(0x7fd3681016b0 msgr2=0x7fd368103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 -- 172.21.15.3:0/2057877183 shutdown_connections 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.619+0000 7fd36ebb9700 1 -- 172.21.15.3:0/2057877183 wait complete. 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.620+0000 7fd36ebb9700 1 Processor -- start 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36ebb9700 1 -- start start 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36ebb9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:37.285 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36ebb9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36819a370 con 0x7fd3681060d0 2023-08-11T13:54:37.286 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36c955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:37.286 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36c955700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48506/0 (socket says 172.21.15.3:48506) 2023-08-11T13:54:37.286 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36c955700 1 -- 172.21.15.3:0/3504062795 learned_addr learned my addr 172.21.15.3:0/3504062795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:37.286 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.621+0000 7fd36c955700 1 -- 172.21.15.3:0/3504062795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c009550 con 0x7fd3681060d0 2023-08-11T13:54:37.286 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd36c955700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009130 tx=0x7fd35c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:37.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd35c003ab0 con 0x7fd3681060d0 2023-08-11T13:54:37.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd35c003c10 con 0x7fd3681060d0 2023-08-11T13:54:37.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36819a570 con 0x7fd3681060d0 2023-08-11T13:54:37.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd35c0247d0 con 0x7fd3681060d0 2023-08-11T13:54:37.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.622+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36819a990 con 0x7fd3681060d0 2023-08-11T13:54:37.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.623+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) v1 ==== 45029+0+0 (secure 0 0 0) 0x7fd35c01a430 con 0x7fd3681060d0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.623+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368061f50 con 0x7fd3681060d0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.624+0000 7fd365ffb700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 0x7fd35803a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.624+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd35c04bec0 con 0x7fd3681060d0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.624+0000 7fd367fff700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 0x7fd35803a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.625+0000 7fd367fff700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 0x7fd35803a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd354006fd0 tx=0x7fd354006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.629+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd35c018b40 con 0x7fd3681060d0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.661+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fd35c050070 con 0x7fd3681060d0 2023-08-11T13:54:37.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.858+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd368103a10 con 0x7fd358038030 2023-08-11T13:54:37.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.864+0000 7fd365ffb700 1 -- 172.21.15.3:0/3504062795 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd368103a10 con 0x7fd358038030 2023-08-11T13:54:37.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 msgr2=0x7fd35803a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:37.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 0x7fd35803a4f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd354006fd0 tx=0x7fd354006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:37.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 msgr2=0x7fd368199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:37.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009130 tx=0x7fd35c011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 shutdown_connections 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fd358038030 0x7fd35803a4f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 --2- 172.21.15.3:0/3504062795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3681060d0 0x7fd368199e30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 >> 172.21.15.3:0/3504062795 conn(0x7fd3681016b0 msgr2=0x7fd368102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 shutdown_connections 2023-08-11T13:54:37.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:36.867+0000 7fd36ebb9700 1 -- 172.21.15.3:0/3504062795 wait complete. 2023-08-11T13:54:38.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:37 smithi003 ceph-mon[97195]: mgrmap e8: x(active, since 2s) 2023-08-11T13:54:38.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:37 smithi003 ceph-mon[97195]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:38.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:37 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:38.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:37 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.831+0000 7f1b8bcdd700 1 Processor -- start 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b8bcdd700 1 -- start start 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b8bcdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b8bcdd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84106ac0 con 0x7f1b841060d0 2023-08-11T13:54:38.464 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b89a79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b89a79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48508/0 (socket says 172.21.15.3:48508) 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.832+0000 7f1b89a79700 1 -- 172.21.15.3:0/2451530291 learned_addr learned my addr 172.21.15.3:0/2451530291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.833+0000 7f1b89a79700 1 -- 172.21.15.3:0/2451530291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b841072d0 con 0x7f1b841060d0 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.833+0000 7f1b89a79700 1 --2- 172.21.15.3:0/2451530291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098a0 tx=0x7f1b80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3cf0194e7e47b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.833+0000 7f1b88a77700 1 -- 172.21.15.3:0/2451530291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b80004160 con 0x7f1b841060d0 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.833+0000 7f1b88a77700 1 -- 172.21.15.3:0/2451530291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b800042c0 con 0x7f1b841060d0 2023-08-11T13:54:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.833+0000 7f1b88a77700 1 -- 172.21.15.3:0/2451530291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b80011a00 con 0x7f1b841060d0 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/2451530291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 msgr2=0x7f1b841064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/2451530291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098a0 tx=0x7f1b80009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/2451530291 shutdown_connections 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/2451530291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b841064f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/2451530291 >> 172.21.15.3:0/2451530291 conn(0x7f1b841016b0 msgr2=0x7f1b84103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/2451530291 shutdown_connections 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.834+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/2451530291 wait complete. 2023-08-11T13:54:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.835+0000 7f1b8bcdd700 1 Processor -- start 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.835+0000 7f1b8bcdd700 1 -- start start 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.835+0000 7f1b8bcdd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.835+0000 7f1b8bcdd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8419c570 con 0x7f1b841060d0 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.835+0000 7f1b89a79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b89a79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48510/0 (socket says 172.21.15.3:48510) 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b89a79700 1 -- 172.21.15.3:0/4060307300 learned_addr learned my addr 172.21.15.3:0/4060307300 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b89a79700 1 -- 172.21.15.3:0/4060307300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80009550 con 0x7f1b841060d0 2023-08-11T13:54:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b89a79700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009130 tx=0x7f1b80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b80003ab0 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b80003c10 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b800247d0 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.836+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8419c770 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.837+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8419cb90 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.837+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84061f50 con 0x7f1b841060d0 2023-08-11T13:54:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.838+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f1b8001abb0 con 0x7f1b841060d0 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.839+0000 7f1b7affd700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 0x7f1b7003a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.839+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b8004c3f0 con 0x7f1b841060d0 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.839+0000 7f1b89278700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 0x7f1b7003a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.840+0000 7f1b89278700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 0x7f1b7003a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b74006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:37.844+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1b80018760 con 0x7f1b841060d0 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.050+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b84103a10 con 0x7f1b70038160 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.051+0000 7f1b7affd700 1 -- 172.21.15.3:0/4060307300 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1b84103a10 con 0x7f1b70038160 2023-08-11T13:54:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.053+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 msgr2=0x7f1b7003a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.053+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 0x7f1b7003a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b74006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 msgr2=0x7f1b8419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009130 tx=0x7f1b80011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 shutdown_connections 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1b70038160 0x7f1b7003a620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 --2- 172.21.15.3:0/4060307300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b841060d0 0x7f1b8419c030 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 >> 172.21.15.3:0/4060307300 conn(0x7f1b841016b0 msgr2=0x7f1b84102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:38.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 shutdown_connections 2023-08-11T13:54:38.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.054+0000 7f1b8bcdd700 1 -- 172.21.15.3:0/4060307300 wait complete. 2023-08-11T13:54:38.471 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2023-08-11T13:54:39.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:38 smithi003 ceph-mon[97195]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:40.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: Generating public/private rsa key pair. 2023-08-11T13:54:40.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: Your identification has been saved in /tmp/tmpzg083x8l/key. 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: Your public key has been saved in /tmp/tmpzg083x8l/key.pub. 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: The key fingerprint is: 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: SHA256:qOrR7OmdPKc/bERA6t2qtKAA6IE3TmFmbWIN5BUgr6Y ceph-69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: The key's randomart image is: 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: +---[RSA 3072]----+ 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |..+.oo. | 2023-08-11T13:54:40.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: | + = .. | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: | X = . | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |o* = . o. | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |=o+ . o.S | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |=+.+ . .. | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |E.+ = .o | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |.. * *..= | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: |..o.* =*.. | 2023-08-11T13:54:40.043 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: +----[SHA256]-----+ 2023-08-11T13:54:40.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.980+0000 7f5dc5b7a700 1 Processor -- start 2023-08-11T13:54:40.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dc5b7a700 1 -- start start 2023-08-11T13:54:40.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dc5b7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:40.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dc5b7a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0106ac0 con 0x7f5dc01060d0 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48526/0 (socket says 172.21.15.3:48526) 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.981+0000 7f5dbf7fe700 1 -- 172.21.15.3:0/687156308 learned_addr learned my addr 172.21.15.3:0/687156308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.982+0000 7f5dbf7fe700 1 -- 172.21.15.3:0/687156308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc01072d0 con 0x7f5dc01060d0 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.982+0000 7f5dbf7fe700 1 --2- 172.21.15.3:0/687156308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098a0 tx=0x7f5db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc1ccdd3bdd5f56 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.982+0000 7f5dbe7fc700 1 -- 172.21.15.3:0/687156308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db0004160 con 0x7f5dc01060d0 2023-08-11T13:54:40.145 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.982+0000 7f5dbe7fc700 1 -- 172.21.15.3:0/687156308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db00042c0 con 0x7f5dc01060d0 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.982+0000 7f5dbe7fc700 1 -- 172.21.15.3:0/687156308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db0011a00 con 0x7f5dc01060d0 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/687156308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 msgr2=0x7f5dc01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/687156308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098a0 tx=0x7f5db0009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/687156308 shutdown_connections 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/687156308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc01064f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/687156308 >> 172.21.15.3:0/687156308 conn(0x7f5dc01016b0 msgr2=0x7f5dc0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:40.146 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/687156308 shutdown_connections 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.983+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/687156308 wait complete. 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.984+0000 7f5dc5b7a700 1 Processor -- start 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.984+0000 7f5dc5b7a700 1 -- start start 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.984+0000 7f5dc5b7a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.984+0000 7f5dc5b7a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc019c570 con 0x7f5dc01060d0 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:40.147 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48530/0 (socket says 172.21.15.3:48530) 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbf7fe700 1 -- 172.21.15.3:0/1989031528 learned_addr learned my addr 172.21.15.3:0/1989031528 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbf7fe700 1 -- 172.21.15.3:0/1989031528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009550 con 0x7f5dc01060d0 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbf7fe700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009130 tx=0x7f5db0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.985+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db0003ab0 con 0x7f5dc01060d0 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.986+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5db0003c10 con 0x7f5dc01060d0 2023-08-11T13:54:40.148 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.986+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5db00247d0 con 0x7f5dc01060d0 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.986+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc019c770 con 0x7f5dc01060d0 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.986+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc019cb90 con 0x7f5dc01060d0 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.987+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f5db001abb0 con 0x7f5dc01060d0 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.987+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc0061f50 con 0x7f5dc01060d0 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.987+0000 7f5dbcff9700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 0x7f5da803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:40.149 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.987+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5db004c030 con 0x7f5dc01060d0 2023-08-11T13:54:40.150 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.987+0000 7f5dbeffd700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 0x7f5da803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:40.150 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.988+0000 7f5dbeffd700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 0x7f5da803a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5db4006fd0 tx=0x7f5db4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:40.150 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:38.992+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5db007ec30 con 0x7f5dc01060d0 2023-08-11T13:54:40.150 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.202+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dc0103a10 con 0x7f5da8038160 2023-08-11T13:54:40.150 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.659+0000 7f5dbcff9700 1 -- 172.21.15.3:0/1989031528 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5dc0103a10 con 0x7f5da8038160 2023-08-11T13:54:40.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 msgr2=0x7f5da803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:40.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 0x7f5da803a620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5db4006fd0 tx=0x7f5db4006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:40.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 msgr2=0x7f5dc019c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:40.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009130 tx=0x7f5db0011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:40.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 shutdown_connections 2023-08-11T13:54:40.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f5da8038160 0x7f5da803a620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:40.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 --2- 172.21.15.3:0/1989031528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc01060d0 0x7f5dc019c030 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:40.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 >> 172.21.15.3:0/1989031528 conn(0x7f5dc01016b0 msgr2=0x7f5dc0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:40.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 shutdown_connections 2023-08-11T13:54:40.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:39.662+0000 7f5dc5b7a700 1 -- 172.21.15.3:0/1989031528 wait complete. 2023-08-11T13:54:41.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:40 smithi003 ceph-mon[97195]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:41.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:40 smithi003 ceph-mon[97195]: Generating ssh key... 2023-08-11T13:54:41.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:40 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:41.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:40 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:41.326 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:54:41.326 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.647+0000 7fdf34333700 1 Processor -- start 2023-08-11T13:54:41.326 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.647+0000 7fdf34333700 1 -- start start 2023-08-11T13:54:41.326 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.648+0000 7fdf34333700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:41.326 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.648+0000 7fdf34333700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c07a910 con 0x7fdf2c07bee0 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.648+0000 7fdf320cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.648+0000 7fdf320cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48540/0 (socket says 172.21.15.3:48540) 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.648+0000 7fdf320cf700 1 -- 172.21.15.3:0/2147378737 learned_addr learned my addr 172.21.15.3:0/2147378737 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.649+0000 7fdf320cf700 1 -- 172.21.15.3:0/2147378737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c07aa50 con 0x7fdf2c07bee0 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.649+0000 7fdf320cf700 1 --2- 172.21.15.3:0/2147378737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098a0 tx=0x7fdf28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321dc859ba22aeb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.649+0000 7fdf310cd700 1 -- 172.21.15.3:0/2147378737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf28004160 con 0x7fdf2c07bee0 2023-08-11T13:54:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.649+0000 7fdf310cd700 1 -- 172.21.15.3:0/2147378737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf280042c0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.649+0000 7fdf310cd700 1 -- 172.21.15.3:0/2147378737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf28011a00 con 0x7fdf2c07bee0 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 -- 172.21.15.3:0/2147378737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 msgr2=0x7fdf2c07a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 --2- 172.21.15.3:0/2147378737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098a0 tx=0x7fdf28009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 -- 172.21.15.3:0/2147378737 shutdown_connections 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 --2- 172.21.15.3:0/2147378737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c07a340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 -- 172.21.15.3:0/2147378737 >> 172.21.15.3:0/2147378737 conn(0x7fdf2c1016b0 msgr2=0x7fdf2c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 -- 172.21.15.3:0/2147378737 shutdown_connections 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.650+0000 7fdf34333700 1 -- 172.21.15.3:0/2147378737 wait complete. 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.651+0000 7fdf34333700 1 Processor -- start 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.651+0000 7fdf34333700 1 -- start start 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.651+0000 7fdf34333700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf34333700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c19c5a0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf320cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf320cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48550/0 (socket says 172.21.15.3:48550) 2023-08-11T13:54:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf320cf700 1 -- 172.21.15.3:0/2531523639 learned_addr learned my addr 172.21.15.3:0/2531523639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf320cf700 1 -- 172.21.15.3:0/2531523639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009550 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.652+0000 7fdf320cf700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009130 tx=0x7fdf28011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.653+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf28003ab0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.653+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf28003c10 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.653+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf280247d0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.653+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c19c7a0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.653+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c19cbc0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.654+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c1960b0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.654+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fdf2801abb0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.654+0000 7fdf237fe700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 0x7fdf1803a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.654+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdf2804c0f0 con 0x7fdf2c07bee0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.655+0000 7fdf318ce700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 0x7fdf1803a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.655+0000 7fdf318ce700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 0x7fdf1803a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fd0 tx=0x7fdf1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.662+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fdf2807ec30 con 0x7fdf2c07bee0 2023-08-11T13:54:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.870+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf2c02cd20 con 0x7fdf18038160 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.870+0000 7fdf237fe700 1 -- 172.21.15.3:0/2531523639 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdf2c02cd20 con 0x7fdf18038160 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 msgr2=0x7fdf1803a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 0x7fdf1803a620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fd0 tx=0x7fdf1c006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 msgr2=0x7fdf2c19c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009130 tx=0x7fdf28011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 shutdown_connections 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fdf18038160 0x7fdf1803a620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 --2- 172.21.15.3:0/2531523639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf2c07bee0 0x7fdf2c19c060 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:41.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 >> 172.21.15.3:0/2531523639 conn(0x7fdf2c1016b0 msgr2=0x7fdf2c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:41.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 shutdown_connections 2023-08-11T13:54:41.333 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:40.873+0000 7fdf34333700 1 -- 172.21.15.3:0/2531523639 wait complete. 2023-08-11T13:54:41.333 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-08-11T13:54:41.333 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2023-08-11T13:54:41.334 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2023-08-11T13:54:42.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:41 smithi003 ceph-mon[97195]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:43.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:43 smithi003 ceph-mon[97195]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:44.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:44 smithi003 ceph-mon[97195]: Deploying cephadm binary to smithi003 2023-08-11T13:54:45.813 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2023-08-11T13:54:45.813 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.875+0000 7f162f278700 1 Processor -- start 2023-08-11T13:54:45.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.875+0000 7f162f278700 1 -- start start 2023-08-11T13:54:45.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162f278700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:45.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162f278700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628106ac0 con 0x7f16281060d0 2023-08-11T13:54:45.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162d014700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162d014700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53062/0 (socket says 172.21.15.3:53062) 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162d014700 1 -- 172.21.15.3:0/1913837398 learned_addr learned my addr 172.21.15.3:0/1913837398 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.876+0000 7f162d014700 1 -- 172.21.15.3:0/1913837398 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16281072d0 con 0x7f16281060d0 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f162d014700 1 --2- 172.21.15.3:0/1913837398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16240098a0 tx=0x7f1624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e9954c28ec583a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f161ffff700 1 -- 172.21.15.3:0/1913837398 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1624004160 con 0x7f16281060d0 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f161ffff700 1 -- 172.21.15.3:0/1913837398 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16240042c0 con 0x7f16281060d0 2023-08-11T13:54:45.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f161ffff700 1 -- 172.21.15.3:0/1913837398 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1624011a00 con 0x7f16281060d0 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f162f278700 1 -- 172.21.15.3:0/1913837398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 msgr2=0x7f16281064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.877+0000 7f162f278700 1 --2- 172.21.15.3:0/1913837398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16240098a0 tx=0x7f1624009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.878+0000 7f162f278700 1 -- 172.21.15.3:0/1913837398 shutdown_connections 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.878+0000 7f162f278700 1 --2- 172.21.15.3:0/1913837398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f16281064f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.878+0000 7f162f278700 1 -- 172.21.15.3:0/1913837398 >> 172.21.15.3:0/1913837398 conn(0x7f16281016b0 msgr2=0x7f1628103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.878+0000 7f162f278700 1 -- 172.21.15.3:0/1913837398 shutdown_connections 2023-08-11T13:54:45.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.878+0000 7f162f278700 1 -- 172.21.15.3:0/1913837398 wait complete. 2023-08-11T13:54:45.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162f278700 1 Processor -- start 2023-08-11T13:54:45.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162f278700 1 -- start start 2023-08-11T13:54:45.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162f278700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:45.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162f278700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162807c1b0 con 0x7f16281060d0 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162d014700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162d014700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53078/0 (socket says 172.21.15.3:53078) 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.879+0000 7f162d014700 1 -- 172.21.15.3:0/4223386150 learned_addr learned my addr 172.21.15.3:0/4223386150 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f162d014700 1 -- 172.21.15.3:0/4223386150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624009550 con 0x7f16281060d0 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f162d014700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1624009130 tx=0x7f1624011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:45.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1624003ab0 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1624003c10 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16240247d0 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162807a2c0 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.880+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162807a6e0 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.882+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1628061f50 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.882+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f162401abb0 con 0x7f16281060d0 2023-08-11T13:54:45.820 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.882+0000 7f161e7fc700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 0x7f161403a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.882+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f162404c0f0 con 0x7f16281060d0 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.883+0000 7f162c813700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 0x7f161403a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.883+0000 7f162c813700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 0x7f161403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1618006fd0 tx=0x7f1618006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:41.887+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f162407ec30 con 0x7f16281060d0 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:42.099+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}) v1 -- 0x7f162810a4d0 con 0x7f1614038160 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.372+0000 7f161e7fc700 1 -- 172.21.15.3:0/4223386150 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f162810a4d0 con 0x7f1614038160 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 msgr2=0x7f161403a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:45.821 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 0x7f161403a620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1618006fd0 tx=0x7f1618006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 msgr2=0x7f162807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1624009130 tx=0x7f1624011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 shutdown_connections 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f1614038160 0x7f161403a620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 --2- 172.21.15.3:0/4223386150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16281060d0 0x7f162807bc70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 >> 172.21.15.3:0/4223386150 conn(0x7f16281016b0 msgr2=0x7f1628102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:45.822 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 shutdown_connections 2023-08-11T13:54:45.823 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:45.375+0000 7f162f278700 1 -- 172.21.15.3:0/4223386150 wait complete. 2023-08-11T13:54:45.823 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2023-08-11T13:54:46.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:46 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:46.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:46 smithi003 ceph-mon[97195]: Added host smithi003 2023-08-11T13:54:46.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:46 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:54:47.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-08-11T13:54:47.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.389+0000 7fccf641f700 1 Processor -- start 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.389+0000 7fccf641f700 1 -- start start 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.389+0000 7fccf641f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.389+0000 7fccf641f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0106ac0 con 0x7fccf01060d0 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53088/0 (socket says 172.21.15.3:53088) 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceffff700 1 -- 172.21.15.3:0/457776596 learned_addr learned my addr 172.21.15.3:0/457776596 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceffff700 1 -- 172.21.15.3:0/457776596 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf01072d0 con 0x7fccf01060d0 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceffff700 1 --2- 172.21.15.3:0/457776596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcce00098a0 tx=0x7fcce0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5a3a4f73eeea8f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.390+0000 7fcceeffd700 1 -- 172.21.15.3:0/457776596 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce0004160 con 0x7fccf01060d0 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fcceeffd700 1 -- 172.21.15.3:0/457776596 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcce00042c0 con 0x7fccf01060d0 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fcceeffd700 1 -- 172.21.15.3:0/457776596 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce0011a00 con 0x7fccf01060d0 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 -- 172.21.15.3:0/457776596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 msgr2=0x7fccf01064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 --2- 172.21.15.3:0/457776596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcce00098a0 tx=0x7fcce0009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:47.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 -- 172.21.15.3:0/457776596 shutdown_connections 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 --2- 172.21.15.3:0/457776596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf01064f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 -- 172.21.15.3:0/457776596 >> 172.21.15.3:0/457776596 conn(0x7fccf01016b0 msgr2=0x7fccf0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 -- 172.21.15.3:0/457776596 shutdown_connections 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.391+0000 7fccf641f700 1 -- 172.21.15.3:0/457776596 wait complete. 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.392+0000 7fccf641f700 1 Processor -- start 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.392+0000 7fccf641f700 1 -- start start 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fccf641f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:47.297 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fccf641f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf019a370 con 0x7fccf01060d0 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fcceffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fcceffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53090/0 (socket says 172.21.15.3:53090) 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fcceffff700 1 -- 172.21.15.3:0/3871300803 learned_addr learned my addr 172.21.15.3:0/3871300803 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.393+0000 7fcceffff700 1 -- 172.21.15.3:0/3871300803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009550 con 0x7fccf01060d0 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fcceffff700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fb0 tx=0x7fcce0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce0003a40 con 0x7fccf01060d0 2023-08-11T13:54:47.298 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcce0003ba0 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce001a750 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf019a570 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.394+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf019a990 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.395+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fcce0024b60 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.395+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf0061f50 con 0x7fccf01060d0 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.396+0000 7fcced7fa700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 0x7fccd803a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:47.299 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.396+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcce004bfc0 con 0x7fccf01060d0 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.396+0000 7fccef7fe700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 0x7fccd803a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.396+0000 7fccef7fe700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 0x7fccd803a5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcce4006fd0 tx=0x7fcce4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.401+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fcce0024430 con 0x7fccf01060d0 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.629+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fccf0103a10 con 0x7fccd8038110 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.636+0000 7fcced7fa700 1 -- 172.21.15.3:0/3871300803 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fccf0103a10 con 0x7fccd8038110 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 msgr2=0x7fccd803a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 0x7fccd803a5d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcce4006fd0 tx=0x7fcce4006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:47.300 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 msgr2=0x7fccf0199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009fb0 tx=0x7fcce0011ad0 comp rx=0 tx=0).stop 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 shutdown_connections 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fccd8038110 0x7fccd803a5d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 --2- 172.21.15.3:0/3871300803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf01060d0 0x7fccf0199e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 >> 172.21.15.3:0/3871300803 conn(0x7fccf01016b0 msgr2=0x7fccf0102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 shutdown_connections 2023-08-11T13:54:47.301 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:46.639+0000 7fccf641f700 1 -- 172.21.15.3:0/3871300803 wait complete. 2023-08-11T13:54:47.302 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2023-08-11T13:54:47.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:47 smithi003 ceph-mon[97195]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:47.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:47 smithi003 ceph-mon[97195]: Saving service mon spec with placement count:5 2023-08-11T13:54:47.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:47 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:48.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.015+0000 7f360bcc4700 1 Processor -- start 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.015+0000 7f360bcc4700 1 -- start start 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.015+0000 7f360bcc4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.015+0000 7f360bcc4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360407a8d0 con 0x7f360407bea0 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.016+0000 7f3609a60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.016+0000 7f3609a60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53100/0 (socket says 172.21.15.3:53100) 2023-08-11T13:54:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.016+0000 7f3609a60700 1 -- 172.21.15.3:0/2695385085 learned_addr learned my addr 172.21.15.3:0/2695385085 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.016+0000 7f3609a60700 1 -- 172.21.15.3:0/2695385085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360407aa10 con 0x7f360407bea0 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.016+0000 7f3609a60700 1 --2- 172.21.15.3:0/2695385085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36000098a0 tx=0x7f3600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777bebc8f37bc5e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.017+0000 7f3608a5e700 1 -- 172.21.15.3:0/2695385085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3600004160 con 0x7f360407bea0 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.017+0000 7f3608a5e700 1 -- 172.21.15.3:0/2695385085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36000042c0 con 0x7f360407bea0 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.017+0000 7f3608a5e700 1 -- 172.21.15.3:0/2695385085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36000119b0 con 0x7f360407bea0 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 -- 172.21.15.3:0/2695385085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 msgr2=0x7f360407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2695385085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36000098a0 tx=0x7f3600009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 -- 172.21.15.3:0/2695385085 shutdown_connections 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2695385085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360407a300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 -- 172.21.15.3:0/2695385085 >> 172.21.15.3:0/2695385085 conn(0x7f36041016b0 msgr2=0x7f3604103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 -- 172.21.15.3:0/2695385085 shutdown_connections 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.018+0000 7f360bcc4700 1 -- 172.21.15.3:0/2695385085 wait complete. 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.019+0000 7f360bcc4700 1 Processor -- start 2023-08-11T13:54:48.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.019+0000 7f360bcc4700 1 -- start start 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.019+0000 7f360bcc4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.019+0000 7f360bcc4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360419c570 con 0x7f360407bea0 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.020+0000 7f3609a60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.020+0000 7f3609a60700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53102/0 (socket says 172.21.15.3:53102) 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.020+0000 7f3609a60700 1 -- 172.21.15.3:0/2747605427 learned_addr learned my addr 172.21.15.3:0/2747605427 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.020+0000 7f3609a60700 1 -- 172.21.15.3:0/2747605427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009550 con 0x7f360407bea0 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.020+0000 7f3609a60700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3600006af0 tx=0x7f3600011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:48.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.021+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3600003a40 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.021+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3600003ba0 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.021+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36000196c0 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.021+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360419c770 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.021+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360419cb90 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.022+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360410c510 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.023+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f360001aa80 con 0x7f360407bea0 2023-08-11T13:54:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.023+0000 7f35faffd700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 0x7f35f003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.023+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f360004c250 con 0x7f360407bea0 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.023+0000 7f360925f700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 0x7f35f003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.024+0000 7f360925f700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 0x7f35f003a5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006fd0 tx=0x7f35f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.029+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f360001a430 con 0x7f360407bea0 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.260+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f36041039b0 con 0x7f35f0038110 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.268+0000 7f35faffd700 1 -- 172.21.15.3:0/2747605427 <== mgr.14118 v2:172.21.15.3:6800/1168721553 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f36041039b0 con 0x7f35f0038110 2023-08-11T13:54:48.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 msgr2=0x7f35f003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 0x7f35f003a5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35f4006fd0 tx=0x7f35f4006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 msgr2=0x7f360419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3600006af0 tx=0x7f3600011a10 comp rx=0 tx=0).stop 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 shutdown_connections 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f35f0038110 0x7f35f003a5d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 --2- 172.21.15.3:0/2747605427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f360407bea0 0x7f360419c030 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 >> 172.21.15.3:0/2747605427 conn(0x7f36041016b0 msgr2=0x7f3604102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:48.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 shutdown_connections 2023-08-11T13:54:48.784 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:48.270+0000 7f360bcc4700 1 -- 172.21.15.3:0/2747605427 wait complete. 2023-08-11T13:54:49.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:48 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:49.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:48 smithi003 ceph-mon[97195]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:54:49.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:48 smithi003 ceph-mon[97195]: Saving service mgr spec with placement count:2 2023-08-11T13:54:49.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:48 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:49.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:48 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:50.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.377+0000 7facfbbce700 1 Processor -- start 2023-08-11T13:54:50.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.377+0000 7facfbbce700 1 -- start start 2023-08-11T13:54:50.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.377+0000 7facfbbce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:50.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.377+0000 7facfbbce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf407a910 con 0x7facf407bee0 2023-08-11T13:54:50.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.378+0000 7facf996a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:50.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.378+0000 7facf996a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53108/0 (socket says 172.21.15.3:53108) 2023-08-11T13:54:50.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.378+0000 7facf996a700 1 -- 172.21.15.3:0/4046674247 learned_addr learned my addr 172.21.15.3:0/4046674247 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:50.074 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.378+0000 7facf996a700 1 -- 172.21.15.3:0/4046674247 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf407aa50 con 0x7facf407bee0 2023-08-11T13:54:50.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.378+0000 7facf996a700 1 --2- 172.21.15.3:0/4046674247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facf00098a0 tx=0x7facf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2439c92c0ad919a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:50.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.379+0000 7facf8968700 1 -- 172.21.15.3:0/4046674247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facf0004160 con 0x7facf407bee0 2023-08-11T13:54:50.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.379+0000 7facf8968700 1 -- 172.21.15.3:0/4046674247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf00042c0 con 0x7facf407bee0 2023-08-11T13:54:50.075 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.379+0000 7facf8968700 1 -- 172.21.15.3:0/4046674247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facf00119b0 con 0x7facf407bee0 2023-08-11T13:54:50.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.379+0000 7facfbbce700 1 -- 172.21.15.3:0/4046674247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 msgr2=0x7facf407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:50.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.379+0000 7facfbbce700 1 --2- 172.21.15.3:0/4046674247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7facf00098a0 tx=0x7facf0009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:50.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.380+0000 7facfbbce700 1 -- 172.21.15.3:0/4046674247 shutdown_connections 2023-08-11T13:54:50.076 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.380+0000 7facfbbce700 1 --2- 172.21.15.3:0/4046674247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf407a340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:50.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.380+0000 7facfbbce700 1 -- 172.21.15.3:0/4046674247 >> 172.21.15.3:0/4046674247 conn(0x7facf41016b0 msgr2=0x7facf4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:50.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.380+0000 7facfbbce700 1 -- 172.21.15.3:0/4046674247 shutdown_connections 2023-08-11T13:54:50.077 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.380+0000 7facfbbce700 1 -- 172.21.15.3:0/4046674247 wait complete. 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.381+0000 7facfbbce700 1 Processor -- start 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.381+0000 7facfbbce700 1 -- start start 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.382+0000 7facfbbce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.382+0000 7facfbbce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf419c5c0 con 0x7facf407bee0 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.382+0000 7facf996a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:50.078 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.382+0000 7facf996a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53122/0 (socket says 172.21.15.3:53122) 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.382+0000 7facf996a700 1 -- 172.21.15.3:0/382114433 learned_addr learned my addr 172.21.15.3:0/382114433 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7facf996a700 1 -- 172.21.15.3:0/382114433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0009550 con 0x7facf407bee0 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7facf996a700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facf0003e60 tx=0x7facf0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facf0003a60 con 0x7facf407bee0 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf419c7c0 con 0x7facf407bee0 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf0003bc0 con 0x7facf407bee0 2023-08-11T13:54:50.079 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facf00196c0 con 0x7facf407bee0 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.383+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf419cbe0 con 0x7facf407bee0 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.385+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7facf0019980 con 0x7facf407bee0 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.386+0000 7faceaffd700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 0x7face003a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.386+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf41960d0 con 0x7facf407bee0 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.386+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7facf004c240 con 0x7facf407bee0 2023-08-11T13:54:50.080 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.386+0000 7facf9169700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 0x7face003a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.387+0000 7facf9169700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 0x7face003a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7face4006fd0 tx=0x7face4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.391+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7facf0019e60 con 0x7facf407bee0 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.625+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7facf4061f50 con 0x7facf407bee0 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.630+0000 7faceaffd700 1 -- 172.21.15.3:0/382114433 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7facf0010b70 con 0x7facf407bee0 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.632+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 msgr2=0x7face003a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.632+0000 7facfbbce700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 0x7face003a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7face4006fd0 tx=0x7face4006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:50.081 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 msgr2=0x7facf419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7facf0003e60 tx=0x7facf0011a30 comp rx=0 tx=0).stop 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 shutdown_connections 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7face0038140 0x7face003a600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 --2- 172.21.15.3:0/382114433 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf407bee0 0x7facf419c080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 >> 172.21.15.3:0/382114433 conn(0x7facf41016b0 msgr2=0x7facf4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 shutdown_connections 2023-08-11T13:54:50.082 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:49.633+0000 7facfbbce700 1 -- 172.21.15.3:0/382114433 wait complete. 2023-08-11T13:54:50.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/382114433' entity='client.admin' 2023-08-11T13:54:51.504 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.741+0000 7f6e3be79700 1 Processor -- start 2023-08-11T13:54:51.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.741+0000 7f6e3be79700 1 -- start start 2023-08-11T13:54:51.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e3be79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:51.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e3be79700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34108cc0 con 0x7f6e341082d0 2023-08-11T13:54:51.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e39c15700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e39c15700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53136/0 (socket says 172.21.15.3:53136) 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e39c15700 1 -- 172.21.15.3:0/612827748 learned_addr learned my addr 172.21.15.3:0/612827748 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.742+0000 7f6e39c15700 1 -- 172.21.15.3:0/612827748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e341094d0 con 0x7f6e341082d0 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e39c15700 1 --2- 172.21.15.3:0/612827748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e300098a0 tx=0x7f6e30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00b285a98e3c205 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e38c13700 1 -- 172.21.15.3:0/612827748 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30004160 con 0x7f6e341082d0 2023-08-11T13:54:51.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e38c13700 1 -- 172.21.15.3:0/612827748 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e300042c0 con 0x7f6e341082d0 2023-08-11T13:54:51.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e38c13700 1 -- 172.21.15.3:0/612827748 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e300119b0 con 0x7f6e341082d0 2023-08-11T13:54:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e3be79700 1 -- 172.21.15.3:0/612827748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 msgr2=0x7f6e341086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.743+0000 7f6e3be79700 1 --2- 172.21.15.3:0/612827748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e300098a0 tx=0x7f6e30009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.744+0000 7f6e3be79700 1 -- 172.21.15.3:0/612827748 shutdown_connections 2023-08-11T13:54:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.744+0000 7f6e3be79700 1 --2- 172.21.15.3:0/612827748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e341086f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:51.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.744+0000 7f6e3be79700 1 -- 172.21.15.3:0/612827748 >> 172.21.15.3:0/612827748 conn(0x7f6e3407b6c0 msgr2=0x7f6e3407bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:51.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.744+0000 7f6e3be79700 1 -- 172.21.15.3:0/612827748 shutdown_connections 2023-08-11T13:54:51.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.744+0000 7f6e3be79700 1 -- 172.21.15.3:0/612827748 wait complete. 2023-08-11T13:54:51.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.745+0000 7f6e3be79700 1 Processor -- start 2023-08-11T13:54:51.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.745+0000 7f6e3be79700 1 -- start start 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.745+0000 7f6e3be79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.745+0000 7f6e3be79700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3419c5b0 con 0x7f6e341082d0 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e39c15700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e39c15700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53144/0 (socket says 172.21.15.3:53144) 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e39c15700 1 -- 172.21.15.3:0/953406938 learned_addr learned my addr 172.21.15.3:0/953406938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e39c15700 1 -- 172.21.15.3:0/953406938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30009550 con 0x7f6e341082d0 2023-08-11T13:54:51.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e39c15700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006af0 tx=0x7f6e30011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:51.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30003a40 con 0x7f6e341082d0 2023-08-11T13:54:51.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e30003ba0 con 0x7f6e341082d0 2023-08-11T13:54:51.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3419c7b0 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.746+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e30019690 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.747+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3419cbd0 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.748+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7f6e3001aa60 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.748+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e34061f50 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.748+0000 7f6e2affd700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 0x7f6e2003a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.748+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6e3004bee0 con 0x7f6e341082d0 2023-08-11T13:54:51.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.748+0000 7f6e39414700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 0x7f6e2003a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.749+0000 7f6e39414700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 0x7f6e2003a5d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.753+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6e300237a0 con 0x7f6e341082d0 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.960+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6e3410c6c0 con 0x7f6e341082d0 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.966+0000 7f6e2affd700 1 -- 172.21.15.3:0/953406938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6e3000b3e0 con 0x7f6e341082d0 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 msgr2=0x7f6e2003a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:51.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 0x7f6e2003a5d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:51.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 msgr2=0x7f6e3419c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:51.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6e30006af0 tx=0x7f6e30011a10 comp rx=0 tx=0).stop 2023-08-11T13:54:51.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 shutdown_connections 2023-08-11T13:54:51.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f6e20038110 0x7f6e2003a5d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:51.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 --2- 172.21.15.3:0/953406938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e341082d0 0x7f6e3419c070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:51.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 >> 172.21.15.3:0/953406938 conn(0x7f6e3407b6c0 msgr2=0x7f6e34106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:51.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 shutdown_connections 2023-08-11T13:54:51.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:50.969+0000 7f6e3be79700 1 -- 172.21.15.3:0/953406938 wait complete. 2023-08-11T13:54:51.516 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2023-08-11T13:54:52.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:51 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:52.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/953406938' entity='client.admin' 2023-08-11T13:54:53.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/594223477' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-08-11T13:54:53.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-mon[97195]: from='mgr.14118 172.21.15.3:0/2918504989' entity='mgr.x' 2023-08-11T13:54:53.450 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: ignoring --setuser ceph since I am not root 2023-08-11T13:54:53.450 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: ignoring --setgroup ceph since I am not root 2023-08-11T13:54:53.450 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:53.447+0000 7ff21b979200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T13:54:53.631 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.064+0000 7fa1a95ae700 1 Processor -- start 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.064+0000 7fa1a95ae700 1 -- start start 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.064+0000 7fa1a95ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.064+0000 7fa1a95ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a407a930 con 0x7fa1a407bf00 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.064+0000 7fa1a2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54880/0 (socket says 172.21.15.3:54880) 2023-08-11T13:54:53.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a2ffd700 1 -- 172.21.15.3:0/2428483790 learned_addr learned my addr 172.21.15.3:0/2428483790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:53.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a2ffd700 1 -- 172.21.15.3:0/2428483790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a407aa70 con 0x7fa1a407bf00 2023-08-11T13:54:53.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a2ffd700 1 --2- 172.21.15.3:0/2428483790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098a0 tx=0x7fa194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1334045f2b72930 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:53.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a1ffb700 1 -- 172.21.15.3:0/2428483790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa194004160 con 0x7fa1a407bf00 2023-08-11T13:54:53.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a1ffb700 1 -- 172.21.15.3:0/2428483790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1940042c0 con 0x7fa1a407bf00 2023-08-11T13:54:53.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.065+0000 7fa1a1ffb700 1 -- 172.21.15.3:0/2428483790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1940119b0 con 0x7fa1a407bf00 2023-08-11T13:54:53.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 -- 172.21.15.3:0/2428483790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 msgr2=0x7fa1a407a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:53.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/2428483790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1940098a0 tx=0x7fa194009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:53.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 -- 172.21.15.3:0/2428483790 shutdown_connections 2023-08-11T13:54:53.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/2428483790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a407a360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:53.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 -- 172.21.15.3:0/2428483790 >> 172.21.15.3:0/2428483790 conn(0x7fa1a41016b0 msgr2=0x7fa1a4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.066+0000 7fa1a95ae700 1 -- 172.21.15.3:0/2428483790 shutdown_connections 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.067+0000 7fa1a95ae700 1 -- 172.21.15.3:0/2428483790 wait complete. 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a95ae700 1 Processor -- start 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a95ae700 1 -- start start 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a95ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:53.636 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a95ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a419c5c0 con 0x7fa1a407bf00 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54896/0 (socket says 172.21.15.3:54896) 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.068+0000 7fa1a2ffd700 1 -- 172.21.15.3:0/594223477 learned_addr learned my addr 172.21.15.3:0/594223477 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa1a2ffd700 1 -- 172.21.15.3:0/594223477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194009550 con 0x7fa1a407bf00 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa1a2ffd700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa194003e60 tx=0x7fa194011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa194003a60 con 0x7fa1a407bf00 2023-08-11T13:54:53.637 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa194003bc0 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1940196c0 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a419c7c0 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.069+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a419cbe0 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.070+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 45135+0+0 (secure 0 0 0) 0x7fa19401a430 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.071+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a41960d0 con 0x7fa1a407bf00 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.071+0000 7fa18bfff700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 0x7fa18c03a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:53.638 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.071+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa19404c110 con 0x7fa1a407bf00 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.071+0000 7fa1a27fc700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 0x7fa18c03a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.072+0000 7fa1a27fc700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 0x7fa18c03a600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa198006fd0 tx=0x7fa198006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.076+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa194010380 con 0x7fa1a407bf00 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:52.367+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fa1a4061f50 con 0x7fa1a407bf00 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.135+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7fa19401a6e0 con 0x7fa1a407bf00 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.138+0000 7fa18bfff700 1 -- 172.21.15.3:0/594223477 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fa19404e0e0 con 0x7fa1a407bf00 2023-08-11T13:54:53.639 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 msgr2=0x7fa18c03a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 0x7fa18c03a600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa198006fd0 tx=0x7fa198006e40 comp rx=0 tx=0).stop 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 msgr2=0x7fa1a419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa194003e60 tx=0x7fa194011a30 comp rx=0 tx=0).stop 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 shutdown_connections 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7fa18c038140 0x7fa18c03a600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 --2- 172.21.15.3:0/594223477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a407bf00 0x7fa1a419c080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:53.640 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 >> 172.21.15.3:0/594223477 conn(0x7fa1a41016b0 msgr2=0x7fa1a4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:53.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 shutdown_connections 2023-08-11T13:54:53.641 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:53.141+0000 7fa1a95ae700 1 -- 172.21.15.3:0/594223477 wait complete. 2023-08-11T13:54:53.766 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:53.560+0000 7ff21b979200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T13:54:53.766 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:53.685+0000 7ff21b979200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T13:54:54.037 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:53.831+0000 7ff21b979200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T13:54:54.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:54.094+0000 7ff21b979200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T13:54:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/594223477' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-08-11T13:54:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-mon[97195]: mgrmap e9: x(active, since 18s) 2023-08-11T13:54:54.761 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:54.588+0000 7ff21b979200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T13:54:55.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:54.759+0000 7ff21b979200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T13:54:55.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:54 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:54.878+0000 7ff21b979200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T13:54:55.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:55 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:55.009+0000 7ff21b979200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T13:54:55.367 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:54:55.368 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-08-11T13:54:55.368 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2023-08-11T13:54:55.368 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-08-11T13:54:55.368 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-08-11T13:54:55.368 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:54:55.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.661+0000 7f7119dc5700 1 Processor -- start 2023-08-11T13:54:55.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.661+0000 7f7119dc5700 1 -- start start 2023-08-11T13:54:55.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.661+0000 7f7119dc5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:55.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.661+0000 7f7119dc5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711407a8d0 con 0x7f711407bea0 2023-08-11T13:54:55.369 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.662+0000 7f71137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.662+0000 7f71137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54918/0 (socket says 172.21.15.3:54918) 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.662+0000 7f71137fe700 1 -- 172.21.15.3:0/1571372882 learned_addr learned my addr 172.21.15.3:0/1571372882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.662+0000 7f71137fe700 1 -- 172.21.15.3:0/1571372882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711407aa10 con 0x7f711407bea0 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f71137fe700 1 --2- 172.21.15.3:0/1571372882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71040098a0 tx=0x7f7104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621f9e658b3366bc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f71127fc700 1 -- 172.21.15.3:0/1571372882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7104004160 con 0x7f711407bea0 2023-08-11T13:54:55.370 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f71127fc700 1 -- 172.21.15.3:0/1571372882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71040042c0 con 0x7f711407bea0 2023-08-11T13:54:55.371 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f71127fc700 1 -- 172.21.15.3:0/1571372882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71040119b0 con 0x7f711407bea0 2023-08-11T13:54:55.371 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f7119dc5700 1 -- 172.21.15.3:0/1571372882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 msgr2=0x7f711407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:55.371 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.663+0000 7f7119dc5700 1 --2- 172.21.15.3:0/1571372882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f71040098a0 tx=0x7f7104009bb0 comp rx=0 tx=0).stop 2023-08-11T13:54:55.371 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.664+0000 7f7119dc5700 1 -- 172.21.15.3:0/1571372882 shutdown_connections 2023-08-11T13:54:55.372 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.664+0000 7f7119dc5700 1 --2- 172.21.15.3:0/1571372882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711407a300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:55.372 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.664+0000 7f7119dc5700 1 -- 172.21.15.3:0/1571372882 >> 172.21.15.3:0/1571372882 conn(0x7f71141016b0 msgr2=0x7f7114103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:55.372 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.664+0000 7f7119dc5700 1 -- 172.21.15.3:0/1571372882 shutdown_connections 2023-08-11T13:54:55.372 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.664+0000 7f7119dc5700 1 -- 172.21.15.3:0/1571372882 wait complete. 2023-08-11T13:54:55.372 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f7119dc5700 1 Processor -- start 2023-08-11T13:54:55.373 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f7119dc5700 1 -- start start 2023-08-11T13:54:55.373 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f7119dc5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:55.373 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f7119dc5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711419c570 con 0x7f711407bea0 2023-08-11T13:54:55.373 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f71137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f71137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54928/0 (socket says 172.21.15.3:54928) 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.665+0000 7f71137fe700 1 -- 172.21.15.3:0/555158874 learned_addr learned my addr 172.21.15.3:0/555158874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f71137fe700 1 -- 172.21.15.3:0/555158874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009550 con 0x7f711407bea0 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f71137fe700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7104006af0 tx=0x7f7104011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7104003a40 con 0x7f711407bea0 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7104003ba0 con 0x7f711407bea0 2023-08-11T13:54:55.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711419c770 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.666+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71040196c0 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.667+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711419cb90 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f710401a430 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114061f50 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7110ff9700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 0x7f70fc03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f710404bf60 con 0x7f711407bea0 2023-08-11T13:54:55.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7112ffd700 1 -- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 msgr2=0x7f70fc03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.668+0000 7f7112ffd700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 0x7f70fc03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.673+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f710401ac60 con 0x7f711407bea0 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.869+0000 7f7112ffd700 1 -- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 msgr2=0x7f70fc03a620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.869+0000 7f7112ffd700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 0x7f70fc03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.937+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f711419f7e0 con 0x7f711407bea0 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.937+0000 7f7110ff9700 1 -- 172.21.15.3:0/555158874 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7104023390 con 0x7f711407bea0 2023-08-11T13:54:55.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 msgr2=0x7f70fc03a620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 0x7f70fc03a620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 msgr2=0x7f711419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7104006af0 tx=0x7f7104011a10 comp rx=0 tx=0).stop 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 shutdown_connections 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f70fc038160 0x7f70fc03a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 --2- 172.21.15.3:0/555158874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f711407bea0 0x7f711419c030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 >> 172.21.15.3:0/555158874 conn(0x7f71141016b0 msgr2=0x7f7114102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:54:55.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 shutdown_connections 2023-08-11T13:54:55.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:54.940+0000 7f7119dc5700 1 -- 172.21.15.3:0/555158874 wait complete. 2023-08-11T13:54:55.382 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2023-08-11T13:54:55.382 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2023-08-11T13:54:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:54:55 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/555158874' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-08-11T13:54:56.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:55 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:55.582+0000 7ff21b979200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T13:54:56.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:56 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:56.476+0000 7ff21b979200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T13:54:56.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:56 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:56.585+0000 7ff21b979200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T13:54:57.926 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:57 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:57.570+0000 7ff21b979200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:57.926 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:57 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:57.693+0000 7ff21b979200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T13:54:58.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:57 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:57.924+0000 7ff21b979200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:54:58.579 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:58 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:58.297+0000 7ff21b979200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T13:54:59.041 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:58 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:58.578+0000 7ff21b979200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T13:54:59.042 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:58 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:58.778+0000 7ff21b979200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T13:54:59.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:54:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:54:59.137+0000 7ff21b979200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T13:55:00.537 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:00.098+0000 7ff21b979200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T13:55:00.537 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:00.279+0000 7ff21b979200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T13:55:00.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:00.536+0000 7ff21b979200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T13:55:00.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:00.686+0000 7ff21b979200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T13:55:01.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:00.974+0000 7ff21b979200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T13:55:02.541 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T13:55:02.076+0000 7ff21b979200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T13:55:02.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: Active manager daemon x restarted 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: Activating manager daemon x 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: osdmap e3: 0 total, 0 up, 0 in 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: mgrmap e10: x(active, starting, since 0.00427532s) 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-08-11T13:55:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-08-11T13:55:02.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-08-11T13:55:02.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: Manager daemon x is now available 2023-08-11T13:55:02.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:02.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-08-11T13:55:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-08-11T13:55:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:03 smithi003 ceph-mon[97195]: mgrmap e11: x(active, since 1.00757s) 2023-08-11T13:55:03.568 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f4402391700 1 Processor -- start 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f4402391700 1 -- start start 2023-08-11T13:55:03.570 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f4402391700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f4402391700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc079e50 con 0x7f43fc1049b0 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54942/0 (socket says 172.21.15.3:54942) 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.943+0000 7f43fbfff700 1 -- 172.21.15.3:0/1259714801 learned_addr learned my addr 172.21.15.3:0/1259714801 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.944+0000 7f43fbfff700 1 -- 172.21.15.3:0/1259714801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc105310 con 0x7f43fc1049b0 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.944+0000 7f43fbfff700 1 --2- 172.21.15.3:0/1259714801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0098a0 tx=0x7f43ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7010c7b5fb0d97ce server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.944+0000 7f43faffd700 1 -- 172.21.15.3:0/1259714801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43ec004160 con 0x7f43fc1049b0 2023-08-11T13:55:03.571 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.944+0000 7f43faffd700 1 -- 172.21.15.3:0/1259714801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43ec0042c0 con 0x7f43fc1049b0 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.944+0000 7f43faffd700 1 -- 172.21.15.3:0/1259714801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43ec0119b0 con 0x7f43fc1049b0 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 -- 172.21.15.3:0/1259714801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 msgr2=0x7f43fc104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 --2- 172.21.15.3:0/1259714801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0098a0 tx=0x7f43ec009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 -- 172.21.15.3:0/1259714801 shutdown_connections 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 --2- 172.21.15.3:0/1259714801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc104dd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 -- 172.21.15.3:0/1259714801 >> 172.21.15.3:0/1259714801 conn(0x7f43fc100590 msgr2=0x7f43fc1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:03.572 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 -- 172.21.15.3:0/1259714801 shutdown_connections 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.945+0000 7f4402391700 1 -- 172.21.15.3:0/1259714801 wait complete. 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.946+0000 7f4402391700 1 Processor -- start 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.946+0000 7f4402391700 1 -- start start 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f4402391700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f4402391700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc19c550 con 0x7f43fc1049b0 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:03.573 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54948/0 (socket says 172.21.15.3:54948) 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f43fbfff700 1 -- 172.21.15.3:0/3756759217 learned_addr learned my addr 172.21.15.3:0/3756759217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.947+0000 7f43fbfff700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec009550 con 0x7f43fc1049b0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f43fbfff700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f43ec006af0 tx=0x7f43ec011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43ec003a40 con 0x7f43fc1049b0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43ec003ba0 con 0x7f43fc1049b0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc19c750 con 0x7f43fc1049b0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43ec0196c0 con 0x7f43fc1049b0 2023-08-11T13:55:03.574 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.948+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc19cb70 con 0x7f43fc1049b0 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.949+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f43ec01aa80 con 0x7f43fc1049b0 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.950+0000 7f43f97fa700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.950+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43e403ace0 con 0x7f43e4038110 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.950+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f43ec04c0e0 con 0x7f43fc1049b0 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.950+0000 7f43fb7fe700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:55.950+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:55:03.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:56.151+0000 7f43fb7fe700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:56.151+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:56.552+0000 7f43fb7fe700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:56.552+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:57.353+0000 7f43fb7fe700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:57.353+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:58.954+0000 7f43fb7fe700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1168721553 2023-08-11T13:55:03.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:54:58.955+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:02.081+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 10) v1 ==== 44915+0+0 (secure 0 0 0) 0x7f43ec01a610 con 0x7f43fc1049b0 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:02.081+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 msgr2=0x7f43e403a5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:02.081+0000 7f43f97fa700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.083+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 11) v1 ==== 45042+0+0 (secure 0 0 0) 0x7f43ec017d90 con 0x7f43fc1049b0 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.083+0000 7f43f97fa700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 0x7f43e403dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.083+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43e403ace0 con 0x7f43e403bbc0 2023-08-11T13:55:03.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.084+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 0x7f43e403dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.085+0000 7f43fb7fe700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 0x7f43e403dfb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a10 tx=0x7f43f00092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.086+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f43e403ace0 con 0x7f43e403bbc0 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.093+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f43fc19d350 con 0x7f43e403bbc0 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.093+0000 7f43f97fa700 1 -- 172.21.15.3:0/3756759217 <== mgr.14150 v2:172.21.15.3:6800/2363526909 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f43fc19d350 con 0x7f43e403bbc0 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 msgr2=0x7f43e403dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 0x7f43e403dfb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003a10 tx=0x7f43f00092b0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 msgr2=0x7f43fc19c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f43ec006af0 tx=0x7f43ec011a10 comp rx=0 tx=0).stop 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 shutdown_connections 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e403bbc0 0x7f43e403dfb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43fc1049b0 0x7f43fc19c010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 --2- 172.21.15.3:0/3756759217 >> [v2:172.21.15.3:6800/1168721553,v1:172.21.15.3:6801/1168721553] conn(0x7f43e4038110 0x7f43e403a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:03.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 >> 172.21.15.3:0/3756759217 conn(0x7f43fc100590 msgr2=0x7f43fc190570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:03.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 shutdown_connections 2023-08-11T13:55:03.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:03.094+0000 7f4402391700 1 -- 172.21.15.3:0/3756759217 wait complete. 2023-08-11T13:55:03.580 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2023-08-11T13:55:03.580 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2023-08-11T13:55:04.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: [11/Aug/2023:13:55:02] ENGINE Bus STARTING 2023-08-11T13:55:04.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: [11/Aug/2023:13:55:02] ENGINE Serving on http://172.21.15.3:8765 2023-08-11T13:55:04.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: [11/Aug/2023:13:55:02] ENGINE Serving on https://172.21.15.3:7150 2023-08-11T13:55:04.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: [11/Aug/2023:13:55:02] ENGINE Bus STARTED 2023-08-11T13:55:04.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-08-11T13:55:04.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:04 smithi003 ceph-mon[97195]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-08-11T13:55:05.065 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.297+0000 7f15b71e0700 1 Processor -- start 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.297+0000 7f15b71e0700 1 -- start start 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.298+0000 7f15b71e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.298+0000 7f15b71e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b007a910 con 0x7f15b007bee0 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.298+0000 7f15b4f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:05.066 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.298+0000 7f15b4f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42718/0 (socket says 172.21.15.3:42718) 2023-08-11T13:55:05.067 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.298+0000 7f15b4f7c700 1 -- 172.21.15.3:0/889423841 learned_addr learned my addr 172.21.15.3:0/889423841 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:05.067 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.299+0000 7f15b4f7c700 1 -- 172.21.15.3:0/889423841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b007aa50 con 0x7f15b007bee0 2023-08-11T13:55:05.067 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.299+0000 7f15b4f7c700 1 --2- 172.21.15.3:0/889423841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfe9322c6e2e784 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:05.067 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.299+0000 7f15af7fe700 1 -- 172.21.15.3:0/889423841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a4004160 con 0x7f15b007bee0 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.299+0000 7f15af7fe700 1 -- 172.21.15.3:0/889423841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a40042c0 con 0x7f15b007bee0 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.299+0000 7f15af7fe700 1 -- 172.21.15.3:0/889423841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a40119b0 con 0x7f15b007bee0 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 -- 172.21.15.3:0/889423841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 msgr2=0x7f15b007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 --2- 172.21.15.3:0/889423841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15a40098a0 tx=0x7f15a4009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 -- 172.21.15.3:0/889423841 shutdown_connections 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 --2- 172.21.15.3:0/889423841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b007a340 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 -- 172.21.15.3:0/889423841 >> 172.21.15.3:0/889423841 conn(0x7f15b01016b0 msgr2=0x7f15b0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:05.068 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 -- 172.21.15.3:0/889423841 shutdown_connections 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.300+0000 7f15b71e0700 1 -- 172.21.15.3:0/889423841 wait complete. 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b71e0700 1 Processor -- start 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b71e0700 1 -- start start 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b71e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b71e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b019c5c0 con 0x7f15b007bee0 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b4f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b4f7c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42732/0 (socket says 172.21.15.3:42732) 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.302+0000 7f15b4f7c700 1 -- 172.21.15.3:0/2413513535 learned_addr learned my addr 172.21.15.3:0/2413513535 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15b4f7c700 1 -- 172.21.15.3:0/2413513535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4009550 con 0x7f15b007bee0 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15b4f7c700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15a4003e60 tx=0x7f15a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a4003a60 con 0x7f15b007bee0 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15a4003bc0 con 0x7f15b007bee0 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b019c7c0 con 0x7f15b007bee0 2023-08-11T13:55:05.070 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a40196c0 con 0x7f15b007bee0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.303+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b019cbe0 con 0x7f15b007bee0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.304+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f15a401a430 con 0x7f15b007bee0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.304+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b01960d0 con 0x7f15b007bee0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.305+0000 7f15adffb700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 0x7f15a003a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.305+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f15a404c2d0 con 0x7f15b007bee0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.305+0000 7f15affff700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 0x7f15a003a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.306+0000 7f15affff700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 0x7f15a003a650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:05.071 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.310+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f15a4010380 con 0x7f15b007bee0 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.536+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f15b002cc80 con 0x7f15a0038190 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.660+0000 7f15adffb700 1 -- 172.21.15.3:0/2413513535 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f15b002cc80 con 0x7f15a0038190 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 msgr2=0x7f15a003a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 0x7f15a003a650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 msgr2=0x7f15b019c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15a4003e60 tx=0x7f15a4011a30 comp rx=0 tx=0).stop 2023-08-11T13:55:05.072 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 shutdown_connections 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f15a0038190 0x7f15a003a650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 --2- 172.21.15.3:0/2413513535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15b007bee0 0x7f15b019c080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 >> 172.21.15.3:0/2413513535 conn(0x7f15b01016b0 msgr2=0x7f15b0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 shutdown_connections 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:04.663+0000 7f15b71e0700 1 -- 172.21.15.3:0/2413513535 wait complete. 2023-08-11T13:55:05.073 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2023-08-11T13:55:05.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:05 smithi003 ceph-mon[97195]: mgrmap e12: x(active, since 2s) 2023-08-11T13:55:05.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:05.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:05.364 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:06 smithi003 ceph-mon[97195]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:06.655 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DrWlo33oyTRJMXjxGJ7pzepRfCAp60ZXUFJwoTP1LVU6IGM/.ikly", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1691762106, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-08-11T13:55:06.655 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.642+0000 7f4e50e42700 1 Processor -- start 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.642+0000 7f4e50e42700 1 -- start start 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.642+0000 7f4e50e42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.642+0000 7f4e50e42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c07a8d0 con 0x7f4e4c07bea0 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.643+0000 7f4e4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.643+0000 7f4e4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42746/0 (socket says 172.21.15.3:42746) 2023-08-11T13:55:06.656 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.643+0000 7f4e4a59c700 1 -- 172.21.15.3:0/131990218 learned_addr learned my addr 172.21.15.3:0/131990218 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:06.657 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.643+0000 7f4e4a59c700 1 -- 172.21.15.3:0/131990218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c07aa10 con 0x7f4e4c07bea0 2023-08-11T13:55:06.657 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e4a59c700 1 --2- 172.21.15.3:0/131990218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098a0 tx=0x7f4e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf17de6ef3bbea4a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:06.657 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e4959a700 1 -- 172.21.15.3:0/131990218 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e3c004160 con 0x7f4e4c07bea0 2023-08-11T13:55:06.657 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e4959a700 1 -- 172.21.15.3:0/131990218 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3c0042c0 con 0x7f4e4c07bea0 2023-08-11T13:55:06.657 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e4959a700 1 -- 172.21.15.3:0/131990218 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e3c0119b0 con 0x7f4e4c07bea0 2023-08-11T13:55:06.660 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e50e42700 1 -- 172.21.15.3:0/131990218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 msgr2=0x7f4e4c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:06.661 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.644+0000 7f4e50e42700 1 --2- 172.21.15.3:0/131990218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098a0 tx=0x7f4e3c009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:06.661 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.645+0000 7f4e50e42700 1 -- 172.21.15.3:0/131990218 shutdown_connections 2023-08-11T13:55:06.661 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.645+0000 7f4e50e42700 1 --2- 172.21.15.3:0/131990218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c07a300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:06.661 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.645+0000 7f4e50e42700 1 -- 172.21.15.3:0/131990218 >> 172.21.15.3:0/131990218 conn(0x7f4e4c1016b0 msgr2=0x7f4e4c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:06.661 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.645+0000 7f4e50e42700 1 -- 172.21.15.3:0/131990218 shutdown_connections 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.645+0000 7f4e50e42700 1 -- 172.21.15.3:0/131990218 wait complete. 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.646+0000 7f4e50e42700 1 Processor -- start 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.646+0000 7f4e50e42700 1 -- start start 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.647+0000 7f4e50e42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.647+0000 7f4e50e42700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c19c520 con 0x7f4e4c07bea0 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.647+0000 7f4e4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:06.662 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.647+0000 7f4e4a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42758/0 (socket says 172.21.15.3:42758) 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.647+0000 7f4e4a59c700 1 -- 172.21.15.3:0/1896906425 learned_addr learned my addr 172.21.15.3:0/1896906425 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.649+0000 7f4e4a59c700 1 -- 172.21.15.3:0/1896906425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009550 con 0x7f4e4c07bea0 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e4a59c700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c003f00 tx=0x7f4e3c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e3c003b20 con 0x7f4e4c07bea0 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c19c720 con 0x7f4e4c07bea0 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3c003c80 con 0x7f4e4c07bea0 2023-08-11T13:55:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e3c019770 con 0x7f4e4c07bea0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.650+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c19cb40 con 0x7f4e4c07bea0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.651+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f4e3c019a60 con 0x7f4e4c07bea0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.651+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c061f50 con 0x7f4e4c07bea0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.652+0000 7f4e3b7fe700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 0x7f4e3403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.652+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4e3c055840 con 0x7f4e4c07bea0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.652+0000 7f4e49d9b700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 0x7f4e3403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:06.664 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.653+0000 7f4e49d9b700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 0x7f4e3403a670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006fd0 tx=0x7f4e40006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.659+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4e3c052060 con 0x7f4e4c07bea0 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:05.905+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4e4c19f580 con 0x7f4e340381b0 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.155+0000 7f4e3b7fe700 1 -- 172.21.15.3:0/1896906425 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f4e4c19f580 con 0x7f4e340381b0 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 msgr2=0x7f4e3403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 0x7f4e3403a670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006fd0 tx=0x7f4e40006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 msgr2=0x7f4e4c19bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:06.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c003f00 tx=0x7f4e3c011a30 comp rx=0 tx=0).stop 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 shutdown_connections 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4e340381b0 0x7f4e3403a670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 --2- 172.21.15.3:0/1896906425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e4c07bea0 0x7f4e4c19bfe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.157+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 >> 172.21.15.3:0/1896906425 conn(0x7f4e4c1016b0 msgr2=0x7f4e4c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.158+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 shutdown_connections 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:06.158+0000 7f4e50e42700 1 -- 172.21.15.3:0/1896906425 wait complete. 2023-08-11T13:55:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2023-08-11T13:55:07.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:07 smithi003 ceph-mon[97195]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:07.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:08.353 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2023-08-11T13:55:08.354 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.388+0000 7f7d6949b700 1 Processor -- start 2023-08-11T13:55:08.354 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.388+0000 7f7d6949b700 1 -- start start 2023-08-11T13:55:08.354 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.389+0000 7f7d6949b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.389+0000 7f7d6949b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64106ac0 con 0x7f7d641060d0 2023-08-11T13:55:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.389+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.389+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42770/0 (socket says 172.21.15.3:42770) 2023-08-11T13:55:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.389+0000 7f7d62ffd700 1 -- 172.21.15.3:0/2977600440 learned_addr learned my addr 172.21.15.3:0/2977600440 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.390+0000 7f7d62ffd700 1 -- 172.21.15.3:0/2977600440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d641072d0 con 0x7f7d641060d0 2023-08-11T13:55:08.356 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.390+0000 7f7d62ffd700 1 --2- 172.21.15.3:0/2977600440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098a0 tx=0x7f7d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb6af995cadc3cf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:08.356 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.390+0000 7f7d61ffb700 1 -- 172.21.15.3:0/2977600440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d54004160 con 0x7f7d641060d0 2023-08-11T13:55:08.356 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d61ffb700 1 -- 172.21.15.3:0/2977600440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d540042c0 con 0x7f7d641060d0 2023-08-11T13:55:08.356 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d61ffb700 1 -- 172.21.15.3:0/2977600440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d540119b0 con 0x7f7d641060d0 2023-08-11T13:55:08.357 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 -- 172.21.15.3:0/2977600440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 msgr2=0x7f7d641064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:08.357 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 --2- 172.21.15.3:0/2977600440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098a0 tx=0x7f7d54009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:08.357 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 -- 172.21.15.3:0/2977600440 shutdown_connections 2023-08-11T13:55:08.358 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 --2- 172.21.15.3:0/2977600440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d641064f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:08.358 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 -- 172.21.15.3:0/2977600440 >> 172.21.15.3:0/2977600440 conn(0x7f7d641016b0 msgr2=0x7f7d64103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:08.358 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 -- 172.21.15.3:0/2977600440 shutdown_connections 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.391+0000 7f7d6949b700 1 -- 172.21.15.3:0/2977600440 wait complete. 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.392+0000 7f7d6949b700 1 Processor -- start 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d6949b700 1 -- start start 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d6949b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d6949b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6419c590 con 0x7f7d641060d0 2023-08-11T13:55:08.359 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:08.360 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42782/0 (socket says 172.21.15.3:42782) 2023-08-11T13:55:08.360 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.393+0000 7f7d62ffd700 1 -- 172.21.15.3:0/1170454720 learned_addr learned my addr 172.21.15.3:0/1170454720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:08.360 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d62ffd700 1 -- 172.21.15.3:0/1170454720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54009550 con 0x7f7d641060d0 2023-08-11T13:55:08.360 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d62ffd700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d54003e60 tx=0x7f7d54011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:08.360 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d54003a60 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d54003bc0 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d540196c0 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.394+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6419c790 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.395+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6419cbb0 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.396+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6410a500 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.396+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) v1 ==== 45148+0+0 (secure 0 0 0) 0x7f7d5401ab60 con 0x7f7d641060d0 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.397+0000 7f7d4bfff700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 0x7f7d4c03a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.397+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d5404c4f0 con 0x7f7d641060d0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.397+0000 7f7d627fc700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 0x7f7d4c03a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.398+0000 7f7d627fc700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 0x7f7d4c03a650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.401+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7d5401a430 con 0x7f7d641060d0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.622+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f7d6410a740 con 0x7f7d641060d0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.622+0000 7f7d4bfff700 1 -- 172.21.15.3:0/1170454720 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f7d5402c3a0 con 0x7f7d641060d0 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 msgr2=0x7f7d4c03a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:08.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 0x7f7d4c03a650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 msgr2=0x7f7d6419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d54003e60 tx=0x7f7d54011a30 comp rx=0 tx=0).stop 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 shutdown_connections 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d4c038190 0x7f7d4c03a650 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 --2- 172.21.15.3:0/1170454720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d641060d0 0x7f7d6419c050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 >> 172.21.15.3:0/1170454720 conn(0x7f7d641016b0 msgr2=0x7f7d64102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 shutdown_connections 2023-08-11T13:55:08.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:07.625+0000 7f7d6949b700 1 -- 172.21.15.3:0/1170454720 wait complete. 2023-08-11T13:55:08.371 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-08-11T13:55:08.372 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2023-08-11T13:55:08.388 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-08-11T13:55:08.388 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2023-08-11T13:55:08.388 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2023-08-11T13:55:08.389 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-08-11T13:55:08.390 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2023-08-11T13:55:08.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:08.391 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2023-08-11T13:55:08.391 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2023-08-11T13:55:08.391 INFO:teuthology.orchestra.run.smithi003.stdout: Password: 3fqzx6tm3s 2023-08-11T13:55:08.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:08.392 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config directory 2023-08-11T13:55:08.392 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling autotune for osd_memory_target 2023-08-11T13:55:08.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:08 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1170454720' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-08-11T13:55:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:09 smithi003 ceph-mon[97195]: mgrmap e13: x(active, since 6s) 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.954+0000 7f479fcad700 1 Processor -- start 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.954+0000 7f479fcad700 1 -- start start 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.954+0000 7f479fcad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.954+0000 7f479fcad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479807a910 con 0x7f479807bee0 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479da49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479da49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42790/0 (socket says 172.21.15.3:42790) 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479da49700 1 -- 172.21.15.3:0/2821648746 learned_addr learned my addr 172.21.15.3:0/2821648746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479da49700 1 -- 172.21.15.3:0/2821648746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479807aa50 con 0x7f479807bee0 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479da49700 1 --2- 172.21.15.3:0/2821648746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47940098a0 tx=0x7f4794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a90d7a62c65f84 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.955+0000 7f479ca47700 1 -- 172.21.15.3:0/2821648746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4794004160 con 0x7f479807bee0 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479ca47700 1 -- 172.21.15.3:0/2821648746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47940042c0 con 0x7f479807bee0 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479ca47700 1 -- 172.21.15.3:0/2821648746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47940119b0 con 0x7f479807bee0 2023-08-11T13:55:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 -- 172.21.15.3:0/2821648746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 msgr2=0x7f479807a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 --2- 172.21.15.3:0/2821648746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47940098a0 tx=0x7f4794009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 -- 172.21.15.3:0/2821648746 shutdown_connections 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 --2- 172.21.15.3:0/2821648746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479807a340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 -- 172.21.15.3:0/2821648746 >> 172.21.15.3:0/2821648746 conn(0x7f47981016b0 msgr2=0x7f4798103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 -- 172.21.15.3:0/2821648746 shutdown_connections 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.956+0000 7f479fcad700 1 -- 172.21.15.3:0/2821648746 wait complete. 2023-08-11T13:55:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.957+0000 7f479fcad700 1 Processor -- start 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.957+0000 7f479fcad700 1 -- start start 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479fcad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479fcad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479819c5c0 con 0x7f479807bee0 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479da49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479da49700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42796/0 (socket says 172.21.15.3:42796) 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479da49700 1 -- 172.21.15.3:0/3239396716 learned_addr learned my addr 172.21.15.3:0/3239396716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.958+0000 7f479da49700 1 -- 172.21.15.3:0/3239396716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4794009550 con 0x7f479807bee0 2023-08-11T13:55:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f479da49700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4794006af0 tx=0x7f4794011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4794003a40 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4794003ba0 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4794019690 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479819c7c0 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.959+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479819cbe0 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.960+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f479401a430 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.960+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47981960d0 con 0x7f479807bee0 2023-08-11T13:55:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.960+0000 7f478effd700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 0x7f478403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.961+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f479404c4c0 con 0x7f479807bee0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.961+0000 7f479d248700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 0x7f478403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.961+0000 7f479d248700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 0x7f478403a6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4788006fd0 tx=0x7f4788006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:08.966+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4794023390 con 0x7f479807bee0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.175+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4798061f50 con 0x7f479807bee0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.175+0000 7f478effd700 1 -- 172.21.15.3:0/3239396716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4794023390 con 0x7f479807bee0 2023-08-11T13:55:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 msgr2=0x7f478403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 0x7f478403a6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4788006fd0 tx=0x7f4788006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 msgr2=0x7f479819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4794006af0 tx=0x7f4794011a10 comp rx=0 tx=0).stop 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 shutdown_connections 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4784038200 0x7f478403a6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 --2- 172.21.15.3:0/3239396716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479807bee0 0x7f479819c080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 >> 172.21.15.3:0/3239396716 conn(0x7f47981016b0 msgr2=0x7f4798102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 shutdown_connections 2023-08-11T13:55:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:09.178+0000 7f479fcad700 1 -- 172.21.15.3:0/3239396716 wait complete. 2023-08-11T13:55:10.854 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.095+0000 7f6679828700 1 Processor -- start 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6679828700 1 -- start start 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6679828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6679828700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667407a8d0 con 0x7f667407bea0 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6672ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6672ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42808/0 (socket says 172.21.15.3:42808) 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.096+0000 7f6672ffd700 1 -- 172.21.15.3:0/2763423202 learned_addr learned my addr 172.21.15.3:0/2763423202 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.097+0000 7f6672ffd700 1 -- 172.21.15.3:0/2763423202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667407aa10 con 0x7f667407bea0 2023-08-11T13:55:10.855 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.097+0000 7f6672ffd700 1 --2- 172.21.15.3:0/2763423202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66640098a0 tx=0x7f6664009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de7d249a59fbc41 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.097+0000 7f6671ffb700 1 -- 172.21.15.3:0/2763423202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664004160 con 0x7f667407bea0 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.097+0000 7f6671ffb700 1 -- 172.21.15.3:0/2763423202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66640042c0 con 0x7f667407bea0 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.097+0000 7f6671ffb700 1 -- 172.21.15.3:0/2763423202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66640119b0 con 0x7f667407bea0 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 -- 172.21.15.3:0/2763423202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 msgr2=0x7f667407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 --2- 172.21.15.3:0/2763423202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66640098a0 tx=0x7f6664009bb0 comp rx=0 tx=0).stop 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 -- 172.21.15.3:0/2763423202 shutdown_connections 2023-08-11T13:55:10.856 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 --2- 172.21.15.3:0/2763423202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667407a300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 -- 172.21.15.3:0/2763423202 >> 172.21.15.3:0/2763423202 conn(0x7f66741016b0 msgr2=0x7f6674103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 -- 172.21.15.3:0/2763423202 shutdown_connections 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.098+0000 7f6679828700 1 -- 172.21.15.3:0/2763423202 wait complete. 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.099+0000 7f6679828700 1 Processor -- start 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.099+0000 7f6679828700 1 -- start start 2023-08-11T13:55:10.857 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6679828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:10.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6679828700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419c570 con 0x7f667407bea0 2023-08-11T13:55:10.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6672ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:10.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6672ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42814/0 (socket says 172.21.15.3:42814) 2023-08-11T13:55:10.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6672ffd700 1 -- 172.21.15.3:0/3380913563 learned_addr learned my addr 172.21.15.3:0/3380913563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:10.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6672ffd700 1 -- 172.21.15.3:0/3380913563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664009550 con 0x7f667407bea0 2023-08-11T13:55:10.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.100+0000 7f6672ffd700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6664003f00 tx=0x7f6664011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:10.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.101+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664003b20 con 0x7f667407bea0 2023-08-11T13:55:10.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.101+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6664003c80 con 0x7f667407bea0 2023-08-11T13:55:10.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.101+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667419c770 con 0x7f667407bea0 2023-08-11T13:55:10.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.101+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6664019770 con 0x7f667407bea0 2023-08-11T13:55:10.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.101+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667419cb90 con 0x7f667407bea0 2023-08-11T13:55:10.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.102+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f666401ab60 con 0x7f667407bea0 2023-08-11T13:55:10.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.102+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6674061f50 con 0x7f667407bea0 2023-08-11T13:55:10.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.102+0000 7f6678826700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 0x7f665c03a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:10.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.102+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f666404c470 con 0x7f667407bea0 2023-08-11T13:55:10.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.103+0000 7f66727fc700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 0x7f665c03a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:10.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.103+0000 7f66727fc700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 0x7f665c03a710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:10.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.107+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f666401a430 con 0x7f667407bea0 2023-08-11T13:55:10.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.428+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f667410c510 con 0x7f667407bea0 2023-08-11T13:55:10.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.430+0000 7f6678826700 1 -- 172.21.15.3:0/3380913563 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f666402db40 con 0x7f667407bea0 2023-08-11T13:55:10.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 msgr2=0x7f665c03a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:10.864 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 0x7f665c03a710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:10.865 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 msgr2=0x7f667419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:10.865 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6664003f00 tx=0x7f6664011a30 comp rx=0 tx=0).stop 2023-08-11T13:55:10.865 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 shutdown_connections 2023-08-11T13:55:10.865 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c038250 0x7f665c03a710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:10.865 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 --2- 172.21.15.3:0/3380913563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667407bea0 0x7f667419c030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:10.866 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 >> 172.21.15.3:0/3380913563 conn(0x7f66741016b0 msgr2=0x7f6674102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:10.866 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 shutdown_connections 2023-08-11T13:55:10.866 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2023-08-11T13:55:10.433+0000 7f6679828700 1 -- 172.21.15.3:0/3380913563 wait complete. 2023-08-11T13:55:10.866 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-08-11T13:55:10.866 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-08-11T13:55:10.867 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.867 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:10.867 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.868 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2023-08-11T13:55:10.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.868 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-08-11T13:55:10.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-08-11T13:55:10.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:10.870 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2023-08-11T13:55:10.916 INFO:tasks.cephadm:Fetching config... 2023-08-11T13:55:10.917 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:55:10.917 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-08-11T13:55:10.960 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-08-11T13:55:10.961 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:55:10.961 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-08-11T13:55:11.045 INFO:tasks.cephadm:Fetching mon keyring... 2023-08-11T13:55:11.045 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:55:11.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-08-11T13:55:11.157 INFO:tasks.cephadm:Fetching pub ssh key... 2023-08-11T13:55:11.158 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:55:11.158 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-08-11T13:55:11.242 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-08-11T13:55:11.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T13:55:11.386 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:11.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T13:55:11.464 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:11.476 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-08-11T13:55:11.521 INFO:teuthology.orchestra.run.smithi187.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCXY752LabmJ2f+2Dnf/erCVxS6AsPUtKTwvsGW5Bk5epuMdLHar7pYQt91emSTBC9h0RKIp5Kqf5TF5X7v5E4vCPutmBN9MYOWc/4tWPJIdxbmSlTBDt4rodtriXwlWG1MMqYnizCYN4ein38DR+RP3/2ijwwamBelOw3+Y7mheStqh6fXqfpKNkst4iJ2COyLSdZgJQcRsJI8W23w3ruTzS9vcYiKm6kzBHpBB+Ga0rA1t2yw5xnl2H4bfAailLhmnWXvMfG+q2LouV2qLitnGraMtAoRNPEs1SSzqUvwJsS6GIWqXB20p8NazdHIgBUXE99BSz59SGTlX822TcjPTGbxg96+6/59PbUtBHYzZiXoaBCbmthCWX1I7+V2iqfOdVwl4WCAwssS0lfvDxrfhqj9GGMtuLQEzZFW01N3bsa0YuLz5llQU8gROC+R96REfCbQADX+tdg4aqLk+/4bfv9TOuBYIL3MecdCimi4dFZMpPDF75B1Mvmyqtnzdk= ceph-69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:11.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-08-11T13:55:11.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:11 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3380913563' entity='client.admin' 2023-08-11T13:55:12.576 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:13.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:13.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:13.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:13.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:13.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:13.980 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:14.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 -- 172.21.15.3:0/480134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 msgr2=0x7f73e8102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 --2- 172.21.15.3:0/480134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8102a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009910 tx=0x7f73e4009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 -- 172.21.15.3:0/480134983 shutdown_connections 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 --2- 172.21.15.3:0/480134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8102a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 -- 172.21.15.3:0/480134983 >> 172.21.15.3:0/480134983 conn(0x7f73e80fdba0 msgr2=0x7f73e8100000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.227+0000 7f73efb81700 1 -- 172.21.15.3:0/480134983 shutdown_connections 2023-08-11T13:55:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.228+0000 7f73efb81700 1 -- 172.21.15.3:0/480134983 wait complete. 2023-08-11T13:55:14.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.229+0000 7f73efb81700 1 Processor -- start 2023-08-11T13:55:14.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.229+0000 7f73efb81700 1 -- start start 2023-08-11T13:55:14.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.229+0000 7f73efb81700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:14.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.229+0000 7f73efb81700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8197c70 con 0x7f73e8102620 2023-08-11T13:55:14.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.230+0000 7f73ed91d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:14.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.230+0000 7f73ed91d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47382/0 (socket says 172.21.15.3:47382) 2023-08-11T13:55:14.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.230+0000 7f73ed91d700 1 -- 172.21.15.3:0/603969917 learned_addr learned my addr 172.21.15.3:0/603969917 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:14.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.230+0000 7f73ed91d700 1 -- 172.21.15.3:0/603969917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40095f0 con 0x7f73e8102620 2023-08-11T13:55:14.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.230+0000 7f73ed91d700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004f40 tx=0x7f73e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:14.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.231+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e401d070 con 0x7f73e8102620 2023-08-11T13:55:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.231+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e400bc10 con 0x7f73e8102620 2023-08-11T13:55:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.231+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e4003a70 con 0x7f73e8102620 2023-08-11T13:55:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.231+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e8197e70 con 0x7f73e8102620 2023-08-11T13:55:14.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.231+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e8198310 con 0x7f73e8102620 2023-08-11T13:55:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.232+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8191130 con 0x7f73e8102620 2023-08-11T13:55:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.232+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f73e4003ce0 con 0x7f73e8102620 2023-08-11T13:55:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.233+0000 7f73deffd700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 0x7f73d403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.233+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73e404bdf0 con 0x7f73e8102620 2023-08-11T13:55:14.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.233+0000 7f73ed11c700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 0x7f73d403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:14.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.234+0000 7f73ed11c700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 0x7f73d403a6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:14.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.238+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f73e402abf0 con 0x7f73e8102620 2023-08-11T13:55:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.487+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f73e8108370 con 0x7f73e8102620 2023-08-11T13:55:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.491+0000 7f73deffd700 1 -- 172.21.15.3:0/603969917 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f73e402abf0 con 0x7f73e8102620 2023-08-11T13:55:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 msgr2=0x7f73d403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 0x7f73d403a6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 msgr2=0x7f73e8197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004f40 tx=0x7f73e40057c0 comp rx=0 tx=0).stop 2023-08-11T13:55:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 shutdown_connections 2023-08-11T13:55:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f73d4038200 0x7f73d403a6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 --2- 172.21.15.3:0/603969917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e8102620 0x7f73e8197730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 >> 172.21.15.3:0/603969917 conn(0x7f73e80fdba0 msgr2=0x7f73e80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 shutdown_connections 2023-08-11T13:55:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:14.494+0000 7f73efb81700 1 -- 172.21.15.3:0/603969917 wait complete. 2023-08-11T13:55:15.173 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-08-11T13:55:15.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-08-11T13:55:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:15 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/603969917' entity='client.admin' 2023-08-11T13:55:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:15.900 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 -- 172.21.15.3:0/1603956257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 msgr2=0x7fa2ec100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/1603956257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec100850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:17.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 -- 172.21.15.3:0/1603956257 shutdown_connections 2023-08-11T13:55:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/1603956257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec100850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 -- 172.21.15.3:0/1603956257 >> 172.21.15.3:0/1603956257 conn(0x7fa2ec0fb9b0 msgr2=0x7fa2ec0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 -- 172.21.15.3:0/1603956257 shutdown_connections 2023-08-11T13:55:17.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.458+0000 7fa2f08a7700 1 -- 172.21.15.3:0/1603956257 wait complete. 2023-08-11T13:55:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.460+0000 7fa2f08a7700 1 Processor -- start 2023-08-11T13:55:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.460+0000 7fa2f08a7700 1 -- start start 2023-08-11T13:55:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.460+0000 7fa2f08a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.460+0000 7fa2f08a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec074c00 con 0x7fa2ec100430 2023-08-11T13:55:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.461+0000 7fa2ea59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.461+0000 7fa2ea59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47398/0 (socket says 172.21.15.3:47398) 2023-08-11T13:55:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.461+0000 7fa2ea59c700 1 -- 172.21.15.3:0/2293485190 learned_addr learned my addr 172.21.15.3:0/2293485190 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.461+0000 7fa2ea59c700 1 -- 172.21.15.3:0/2293485190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0095f0 con 0x7fa2ec100430 2023-08-11T13:55:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.461+0000 7fa2ea59c700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006010 tx=0x7fa2dc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.462+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2dc01d070 con 0x7fa2ec100430 2023-08-11T13:55:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.462+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2dc0044e0 con 0x7fa2ec100430 2023-08-11T13:55:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.462+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec072d10 con 0x7fa2ec100430 2023-08-11T13:55:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.462+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2dc0039b0 con 0x7fa2ec100430 2023-08-11T13:55:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.462+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec073150 con 0x7fa2ec100430 2023-08-11T13:55:17.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.463+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fa2dc003c20 con 0x7fa2ec100430 2023-08-11T13:55:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.463+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec061f50 con 0x7fa2ec100430 2023-08-11T13:55:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.463+0000 7fa2db7fe700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 0x7fa2d403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.463+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2dc04c0e0 con 0x7fa2ec100430 2023-08-11T13:55:17.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.463+0000 7fa2e9d9b700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 0x7fa2d403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:17.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.464+0000 7fa2e9d9b700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 0x7fa2d403a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006fd0 tx=0x7fa2e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:17.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.469+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa2dc07ec30 con 0x7fa2ec100430 2023-08-11T13:55:17.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.681+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2ec18ece0 con 0x7fa2d4038200 2023-08-11T13:55:17.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.684+0000 7fa2db7fe700 1 -- 172.21.15.3:0/2293485190 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa2ec18ece0 con 0x7fa2d4038200 2023-08-11T13:55:17.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 msgr2=0x7fa2d403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:17.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 0x7fa2d403a6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006fd0 tx=0x7fa2e0006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 msgr2=0x7fa2ec0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006010 tx=0x7fa2dc0057c0 comp rx=0 tx=0).stop 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 shutdown_connections 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2d4038200 0x7fa2d403a6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 --2- 172.21.15.3:0/2293485190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2ec100430 0x7fa2ec0746c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 >> 172.21.15.3:0/2293485190 conn(0x7fa2ec0fb9b0 msgr2=0x7fa2ec0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 shutdown_connections 2023-08-11T13:55:17.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:17.687+0000 7fa2f08a7700 1 -- 172.21.15.3:0/2293485190 wait complete. 2023-08-11T13:55:18.278 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2023-08-11T13:55:18.279 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:55:18.279 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2023-08-11T13:55:18.300 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:55:18.301 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:18.364 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2023-08-11T13:55:18.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch host add smithi103 2023-08-11T13:55:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:19.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:19.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:19.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:19.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:19.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:19.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: Updating smithi003:/etc/ceph/ceph.conf 2023-08-11T13:55:19.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:18 smithi003 ceph-mon[97195]: Updating smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:19.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:19 smithi003 ceph-mon[97195]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 -- 172.21.15.3:0/1118096358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 msgr2=0x7f6ee01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 --2- 172.21.15.3:0/1118096358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee01009f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009910 tx=0x7f6ed0009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 -- 172.21.15.3:0/1118096358 shutdown_connections 2023-08-11T13:55:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 --2- 172.21.15.3:0/1118096358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee01009f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 -- 172.21.15.3:0/1118096358 >> 172.21.15.3:0/1118096358 conn(0x7f6ee00fbee0 msgr2=0x7f6ee00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:20.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 -- 172.21.15.3:0/1118096358 shutdown_connections 2023-08-11T13:55:20.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.405+0000 7f6ee659a700 1 -- 172.21.15.3:0/1118096358 wait complete. 2023-08-11T13:55:20.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.407+0000 7f6ee659a700 1 Processor -- start 2023-08-11T13:55:20.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.407+0000 7f6ee659a700 1 -- start start 2023-08-11T13:55:20.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.407+0000 7f6ee659a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.407+0000 7f6ee659a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee019bb60 con 0x7f6ee0100610 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.408+0000 7f6edffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.408+0000 7f6edffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47422/0 (socket says 172.21.15.3:47422) 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.408+0000 7f6edffff700 1 -- 172.21.15.3:0/115226697 learned_addr learned my addr 172.21.15.3:0/115226697 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.408+0000 7f6edffff700 1 -- 172.21.15.3:0/115226697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00095f0 con 0x7f6ee0100610 2023-08-11T13:55:20.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.408+0000 7f6edffff700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0006010 tx=0x7f6ed0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.409+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed001d070 con 0x7f6ee0100610 2023-08-11T13:55:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.409+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed000bc10 con 0x7f6ee0100610 2023-08-11T13:55:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.409+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee019bd60 con 0x7f6ee0100610 2023-08-11T13:55:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.409+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ed0003910 con 0x7f6ee0100610 2023-08-11T13:55:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.409+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee019c180 con 0x7f6ee0100610 2023-08-11T13:55:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.410+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f6ed000bd80 con 0x7f6ee0100610 2023-08-11T13:55:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.410+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee0061f50 con 0x7f6ee0100610 2023-08-11T13:55:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.410+0000 7f6eddffb700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 0x7f6ecc03a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.410+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ed004bdc0 con 0x7f6ee0100610 2023-08-11T13:55:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.411+0000 7f6ed7fff700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 0x7f6ecc03a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.411+0000 7f6ed7fff700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 0x7f6ecc03a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006fd0 tx=0x7f6ec8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:20.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.415+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6ed0029360 con 0x7f6ee0100610 2023-08-11T13:55:20.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:20.626+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ee00008d0 con 0x7f6ecc038200 2023-08-11T13:55:21.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:20 smithi003 ceph-mon[97195]: Updating smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:21.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:20 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:21.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:20 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:21.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:20 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:22.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:21 smithi003 ceph-mon[97195]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:22.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:21 smithi003 ceph-mon[97195]: Deploying cephadm binary to smithi103 2023-08-11T13:55:22.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.987+0000 7f6eddffb700 1 -- 172.21.15.3:0/115226697 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6ee00008d0 con 0x7f6ecc038200 2023-08-11T13:55:22.989 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi103' with addr '172.21.15.103' 2023-08-11T13:55:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 msgr2=0x7f6ecc03a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 0x7f6ecc03a6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006fd0 tx=0x7f6ec8006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 msgr2=0x7f6ee019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0006010 tx=0x7f6ed0004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 shutdown_connections 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ecc038200 0x7f6ecc03a6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 --2- 172.21.15.3:0/115226697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ee0100610 0x7f6ee019b620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.989+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 >> 172.21.15.3:0/115226697 conn(0x7f6ee00fbee0 msgr2=0x7f6ee0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.990+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 shutdown_connections 2023-08-11T13:55:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:22.990+0000 7f6ee659a700 1 -- 172.21.15.3:0/115226697 wait complete. 2023-08-11T13:55:23.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:23 smithi003 ceph-mon[97195]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:23.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:23.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:23.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch host ls --format=json 2023-08-11T13:55:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:24.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:24 smithi003 ceph-mon[97195]: Added host smithi103 2023-08-11T13:55:24.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:24.406 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:25.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:25 smithi003 ceph-mon[97195]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:25.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 -- 172.21.15.3:0/1462761967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 msgr2=0x7f8cf81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 --2- 172.21.15.3:0/1462761967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf81009f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009910 tx=0x7f8cf4009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 -- 172.21.15.3:0/1462761967 shutdown_connections 2023-08-11T13:55:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 --2- 172.21.15.3:0/1462761967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf81009f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 -- 172.21.15.3:0/1462761967 >> 172.21.15.3:0/1462761967 conn(0x7f8cf80fbee0 msgr2=0x7f8cf80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 -- 172.21.15.3:0/1462761967 shutdown_connections 2023-08-11T13:55:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.663+0000 7f8d00245700 1 -- 172.21.15.3:0/1462761967 wait complete. 2023-08-11T13:55:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.664+0000 7f8d00245700 1 Processor -- start 2023-08-11T13:55:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.664+0000 7f8d00245700 1 -- start start 2023-08-11T13:55:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8d00245700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8d00245700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf819bb60 con 0x7f8cf8100610 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8cfdfe1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8cfdfe1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39764/0 (socket says 172.21.15.3:39764) 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8cfdfe1700 1 -- 172.21.15.3:0/2223924894 learned_addr learned my addr 172.21.15.3:0/2223924894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.665+0000 7f8cfdfe1700 1 -- 172.21.15.3:0/2223924894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40095f0 con 0x7f8cf8100610 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8cfdfe1700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006010 tx=0x7f8cf4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cf401d070 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf819bd60 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cf400bc10 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cf4003910 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.666+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf819c180 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.667+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f8cf400bd80 con 0x7f8cf8100610 2023-08-11T13:55:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.667+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf8061f50 con 0x7f8cf8100610 2023-08-11T13:55:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.668+0000 7f8ceeffd700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 0x7f8ce403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.668+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8cf404bd60 con 0x7f8cf8100610 2023-08-11T13:55:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.668+0000 7f8cfd7e0700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 0x7f8ce403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.668+0000 7f8cfd7e0700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 0x7f8ce403a670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8006fd0 tx=0x7f8ce8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:25.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.673+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8cf4029360 con 0x7f8cf8100610 2023-08-11T13:55:25.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.889+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cf80008d0 con 0x7f8ce40381b0 2023-08-11T13:55:25.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.890+0000 7f8ceeffd700 1 -- 172.21.15.3:0/2223924894 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f8cf80008d0 con 0x7f8ce40381b0 2023-08-11T13:55:25.891 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2023-08-11T13:55:25.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 msgr2=0x7f8ce403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 0x7f8ce403a670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8006fd0 tx=0x7f8ce8006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 msgr2=0x7f8cf819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4006010 tx=0x7f8cf4004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 shutdown_connections 2023-08-11T13:55:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8ce40381b0 0x7f8ce403a670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 --2- 172.21.15.3:0/2223924894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cf8100610 0x7f8cf819b620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:25.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 >> 172.21.15.3:0/2223924894 conn(0x7f8cf80fbee0 msgr2=0x7f8cf8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 shutdown_connections 2023-08-11T13:55:25.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:25.893+0000 7f8d00245700 1 -- 172.21.15.3:0/2223924894 wait complete. 2023-08-11T13:55:26.327 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi187 2023-08-11T13:55:26.327 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:55:26.328 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.conf 2023-08-11T13:55:26.348 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:55:26.348 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:26.408 INFO:tasks.cephadm:Adding host smithi187 to orchestrator... 2023-08-11T13:55:26.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch host add smithi187 2023-08-11T13:55:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:27.455 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:27 smithi003 ceph-mon[97195]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:27.455 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:27 smithi003 ceph-mon[97195]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.687+0000 7f22125fe700 1 -- 172.21.15.3:0/291377850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 msgr2=0x7f220c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.687+0000 7f22125fe700 1 --2- 172.21.15.3:0/291377850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c108a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009910 tx=0x7f21fc009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.688+0000 7f22125fe700 1 -- 172.21.15.3:0/291377850 shutdown_connections 2023-08-11T13:55:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.688+0000 7f22125fe700 1 --2- 172.21.15.3:0/291377850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c108a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:28.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.688+0000 7f22125fe700 1 -- 172.21.15.3:0/291377850 >> 172.21.15.3:0/291377850 conn(0x7f220c0fe0b0 msgr2=0x7f220c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.688+0000 7f22125fe700 1 -- 172.21.15.3:0/291377850 shutdown_connections 2023-08-11T13:55:28.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.688+0000 7f22125fe700 1 -- 172.21.15.3:0/291377850 wait complete. 2023-08-11T13:55:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.689+0000 7f22125fe700 1 Processor -- start 2023-08-11T13:55:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.689+0000 7f22125fe700 1 -- start start 2023-08-11T13:55:28.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f22125fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f22125fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c19dc30 con 0x7f220c108670 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f220bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f220bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39784/0 (socket says 172.21.15.3:39784) 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f220bfff700 1 -- 172.21.15.3:0/4275627078 learned_addr learned my addr 172.21.15.3:0/4275627078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f220bfff700 1 -- 172.21.15.3:0/4275627078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0095f0 con 0x7f220c108670 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.690+0000 7f220bfff700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006010 tx=0x7f21fc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:28.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.691+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21fc01d070 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.691+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21fc00bc10 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.691+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c19de30 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.691+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21fc003910 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.691+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c197770 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.692+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c061f50 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.692+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f21fc00bd80 con 0x7f220c108670 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.692+0000 7f22097fa700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 0x7f21f403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:28.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.692+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21fc04bdc0 con 0x7f220c108670 2023-08-11T13:55:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.693+0000 7f220b7fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 0x7f21f403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:28.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.693+0000 7f220b7fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 0x7f21f403a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2200006fd0 tx=0x7f2200006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.698+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f21fc029360 con 0x7f220c108670 2023-08-11T13:55:28.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:28.909+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}) v1 -- 0x7f220c10c8b0 con 0x7f21f4038200 2023-08-11T13:55:29.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:29 smithi003 ceph-mon[97195]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:30.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:30 smithi003 ceph-mon[97195]: Deploying cephadm binary to smithi187 2023-08-11T13:55:30.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:30 smithi003 ceph-mon[97195]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:31.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.311+0000 7f22097fa700 1 -- 172.21.15.3:0/4275627078 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f220c10c8b0 con 0x7f21f4038200 2023-08-11T13:55:31.312 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi187' with addr '172.21.15.187' 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 msgr2=0x7f21f403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 0x7f21f403a6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2200006fd0 tx=0x7f2200006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 msgr2=0x7f220c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006010 tx=0x7f21fc004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 shutdown_connections 2023-08-11T13:55:31.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f21f4038200 0x7f21f403a6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:31.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 --2- 172.21.15.3:0/4275627078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f220c108670 0x7f220c19d6f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:31.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 >> 172.21.15.3:0/4275627078 conn(0x7f220c0fe0b0 msgr2=0x7f220c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:31.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.314+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 shutdown_connections 2023-08-11T13:55:31.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:31.315+0000 7f22125fe700 1 -- 172.21.15.3:0/4275627078 wait complete. 2023-08-11T13:55:31.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch host ls --format=json 2023-08-11T13:55:32.469 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:32.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:32 smithi003 ceph-mon[97195]: Added host smithi187 2023-08-11T13:55:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:32 smithi003 ceph-mon[97195]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:33.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:33 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:33.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 -- 172.21.15.3:0/202435444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 msgr2=0x7f91201009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:33.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 --2- 172.21.15.3:0/202435444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f91201009f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f911c009910 tx=0x7f911c009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:33.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 -- 172.21.15.3:0/202435444 shutdown_connections 2023-08-11T13:55:33.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 --2- 172.21.15.3:0/202435444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f91201009f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:33.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 -- 172.21.15.3:0/202435444 >> 172.21.15.3:0/202435444 conn(0x7f91200fbee0 msgr2=0x7f91200fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:33.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 -- 172.21.15.3:0/202435444 shutdown_connections 2023-08-11T13:55:33.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.978+0000 7f912833d700 1 -- 172.21.15.3:0/202435444 wait complete. 2023-08-11T13:55:33.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.979+0000 7f912833d700 1 Processor -- start 2023-08-11T13:55:33.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.979+0000 7f912833d700 1 -- start start 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f912833d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f912833d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912019bb60 con 0x7f9120100610 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f91260d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f91260d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46634/0 (socket says 172.21.15.3:46634) 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f91260d9700 1 -- 172.21.15.3:0/711378419 learned_addr learned my addr 172.21.15.3:0/711378419 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.980+0000 7f91260d9700 1 -- 172.21.15.3:0/711378419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0095f0 con 0x7f9120100610 2023-08-11T13:55:33.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f91260d9700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:33.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f911c01d070 con 0x7f9120100610 2023-08-11T13:55:33.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911c00bc10 con 0x7f9120100610 2023-08-11T13:55:33.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912019bd60 con 0x7f9120100610 2023-08-11T13:55:33.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f911c003910 con 0x7f9120100610 2023-08-11T13:55:33.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.981+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912019c180 con 0x7f9120100610 2023-08-11T13:55:33.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.982+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f911c00bd80 con 0x7f9120100610 2023-08-11T13:55:33.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.982+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9120061f50 con 0x7f9120100610 2023-08-11T13:55:33.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.983+0000 7f91177fe700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 0x7f910c03a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:33.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.983+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f911c04be00 con 0x7f9120100610 2023-08-11T13:55:33.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.983+0000 7f91258d8700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 0x7f910c03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:33.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.983+0000 7f91258d8700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 0x7f910c03a670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9110006fd0 tx=0x7f9110006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:33.988+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f911c029360 con 0x7f9120100610 2023-08-11T13:55:34.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.203+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f912010a890 con 0x7f910c0381b0 2023-08-11T13:55:34.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.205+0000 7f91177fe700 1 -- 172.21.15.3:0/711378419 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f912010a890 con 0x7f910c0381b0 2023-08-11T13:55:34.206 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T13:55:34.206 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}] 2023-08-11T13:55:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 msgr2=0x7f910c03a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 0x7f910c03a670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9110006fd0 tx=0x7f9110006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:34.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 msgr2=0x7f912019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f911c006010 tx=0x7f911c004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 shutdown_connections 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f910c0381b0 0x7f910c03a670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 --2- 172.21.15.3:0/711378419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9120100610 0x7f912019b620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 >> 172.21.15.3:0/711378419 conn(0x7f91200fbee0 msgr2=0x7f9120106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 shutdown_connections 2023-08-11T13:55:34.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:34.207+0000 7f912833d700 1 -- 172.21.15.3:0/711378419 wait complete. 2023-08-11T13:55:34.760 INFO:tasks.cephadm:Setting crush tunables to default 2023-08-11T13:55:34.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd crush tunables default 2023-08-11T13:55:35.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:35.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:34 smithi003 ceph-mon[97195]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:34 smithi003 ceph-mon[97195]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:35.488 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:55:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 -- 172.21.15.3:0/1406071953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 msgr2=0x7f78000730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 --2- 172.21.15.3:0/1406071953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f78000730a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009910 tx=0x7f77f0009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 -- 172.21.15.3:0/1406071953 shutdown_connections 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 --2- 172.21.15.3:0/1406071953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f78000730a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 -- 172.21.15.3:0/1406071953 >> 172.21.15.3:0/1406071953 conn(0x7f78000fbee0 msgr2=0x7f78000fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 -- 172.21.15.3:0/1406071953 shutdown_connections 2023-08-11T13:55:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.012+0000 7f780634c700 1 -- 172.21.15.3:0/1406071953 wait complete. 2023-08-11T13:55:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.013+0000 7f780634c700 1 Processor -- start 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.013+0000 7f780634c700 1 -- start start 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.014+0000 7f780634c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.014+0000 7f780634c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780019c400 con 0x7f7800072cc0 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.014+0000 7f77fffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.014+0000 7f77fffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46658/0 (socket says 172.21.15.3:46658) 2023-08-11T13:55:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.014+0000 7f77fffff700 1 -- 172.21.15.3:0/3360283583 learned_addr learned my addr 172.21.15.3:0/3360283583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f77fffff700 1 -- 172.21.15.3:0/3360283583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00095f0 con 0x7f7800072cc0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f77fffff700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005850 tx=0x7f77f0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77f001d070 con 0x7f7800072cc0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780019c660 con 0x7f7800072cc0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77f0004370 con 0x7f7800072cc0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77f00039f0 con 0x7f7800072cc0 2023-08-11T13:55:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.015+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780019cae0 con 0x7f7800072cc0 2023-08-11T13:55:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.016+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f77f0003c60 con 0x7f7800072cc0 2023-08-11T13:55:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.016+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800061f50 con 0x7f7800072cc0 2023-08-11T13:55:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.017+0000 7f77fd7fa700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 0x7f77e803a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.017+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f77f004bdd0 con 0x7f7800072cc0 2023-08-11T13:55:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.017+0000 7f77ff7fe700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 0x7f77e803a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.017+0000 7f77ff7fe700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 0x7f77e803a6c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77f4006fd0 tx=0x7f77f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.022+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f77f00044e0 con 0x7f7800072cc0 2023-08-11T13:55:37.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:37.237+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7800110db0 con 0x7f7800072cc0 2023-08-11T13:55:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:37.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.150+0000 7f77fd7fa700 1 -- 172.21.15.3:0/3360283583 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f77f0026070 con 0x7f7800072cc0 2023-08-11T13:55:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 msgr2=0x7f77e803a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 0x7f77e803a6c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77f4006fd0 tx=0x7f77f4006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 msgr2=0x7f780019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77f0005850 tx=0x7f77f0004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 shutdown_connections 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77e8038200 0x7f77e803a6c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 --2- 172.21.15.3:0/3360283583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7800072cc0 0x7f780019bec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.153+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 >> 172.21.15.3:0/3360283583 conn(0x7f78000fbee0 msgr2=0x7f78000fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.154+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 shutdown_connections 2023-08-11T13:55:38.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:55:38.154+0000 7f780634c700 1 -- 172.21.15.3:0/3360283583 wait complete. 2023-08-11T13:55:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2023-08-11T13:55:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:55:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T13:55:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:38.674 INFO:tasks.cephadm:Adding mon.a on smithi003 2023-08-11T13:55:38.674 INFO:tasks.cephadm:Adding mon.b on smithi103 2023-08-11T13:55:38.674 INFO:tasks.cephadm:Adding mon.c on smithi187 2023-08-11T13:55:38.675 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c' 2023-08-11T13:55:39.264 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:39 smithi003 ceph-mon[97195]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T13:55:39.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:39 smithi003 ceph-mon[97195]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:40.459 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 -- 172.21.15.187:0/306443002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 msgr2=0x7f31900feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 --2- 172.21.15.187:0/306443002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f31900feef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3184009910 tx=0x7f3184009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 -- 172.21.15.187:0/306443002 shutdown_connections 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 --2- 172.21.15.187:0/306443002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f31900feef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 -- 172.21.15.187:0/306443002 >> 172.21.15.187:0/306443002 conn(0x7f3190075980 msgr2=0x7f3190075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 -- 172.21.15.187:0/306443002 shutdown_connections 2023-08-11T13:55:40.460 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.457+0000 7f3196ec7700 1 -- 172.21.15.187:0/306443002 wait complete. 2023-08-11T13:55:40.461 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3196ec7700 1 Processor -- start 2023-08-11T13:55:40.461 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3196ec7700 1 -- start start 2023-08-11T13:55:40.461 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3196ec7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:40.461 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3196ec7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319019bb80 con 0x7f31900feb10 2023-08-11T13:55:40.462 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3194c63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:40.462 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3194c63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.187:40006/0 (socket says 172.21.15.187:40006) 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.458+0000 7f3194c63700 1 -- 172.21.15.187:0/1036368322 learned_addr learned my addr 172.21.15.187:0/1036368322 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f3194c63700 1 -- 172.21.15.187:0/1036368322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840095f0 con 0x7f31900feb10 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f3194c63700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3184006010 tx=0x7f3184004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f318401d070 con 0x7f31900feb10 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319019bd20 con 0x7f31900feb10 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f318400bc10 con 0x7f31900feb10 2023-08-11T13:55:40.463 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3184003910 con 0x7f31900feb10 2023-08-11T13:55:40.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.459+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319019c140 con 0x7f31900feb10 2023-08-11T13:55:40.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.460+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3190061f50 con 0x7f31900feb10 2023-08-11T13:55:40.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.460+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f318400bd80 con 0x7f31900feb10 2023-08-11T13:55:40.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.461+0000 7f318dffb700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 0x7f318003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:40.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.461+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f318404bdc0 con 0x7f31900feb10 2023-08-11T13:55:40.465 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.461+0000 7f318ffff700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 0x7f318003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:40.465 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.462+0000 7f318ffff700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 0x7f318003a6c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f317c006fd0 tx=0x7f317c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:40.468 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.466+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3184029360 con 0x7f31900feb10 2023-08-11T13:55:40.667 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.665+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f31901956c0 con 0x7f3180038200 2023-08-11T13:55:40.674 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.673+0000 7f318dffb700 1 -- 172.21.15.187:0/1036368322 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f31901956c0 con 0x7f3180038200 2023-08-11T13:55:40.675 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mon update... 2023-08-11T13:55:40.678 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 msgr2=0x7f318003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:40.678 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 0x7f318003a6c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f317c006fd0 tx=0x7f317c006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:40.678 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 msgr2=0x7f319019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:40.678 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3184006010 tx=0x7f3184004ff0 comp rx=0 tx=0).stop 2023-08-11T13:55:40.679 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 shutdown_connections 2023-08-11T13:55:40.679 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3180038200 0x7f318003a6c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:40.679 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 --2- 172.21.15.187:0/1036368322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31900feb10 0x7f319019b640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:40.679 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 >> 172.21.15.187:0/1036368322 conn(0x7f3190075980 msgr2=0x7f31901048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:40.679 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 shutdown_connections 2023-08-11T13:55:40.680 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:40.674+0000 7f3196ec7700 1 -- 172.21.15.187:0/1036368322 wait complete. 2023-08-11T13:55:41.148 DEBUG:teuthology.orchestra.run.smithi103:mon.b> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.b.service 2023-08-11T13:55:41.151 DEBUG:teuthology.orchestra.run.smithi187:mon.c> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.c.service 2023-08-11T13:55:41.155 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T13:55:41.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T13:55:41.186 INFO:journalctl@ceph.mon.b.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T13:55:41.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T13:55:41.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:41.989 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:42.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:42 smithi003 ceph-mon[97195]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:42.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:42 smithi003 ceph-mon[97195]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c;count:3 2023-08-11T13:55:42.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:42 smithi003 ceph-mon[97195]: Deploying daemon mon.c on smithi187 2023-08-11T13:55:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:43 smithi003 ceph-mon[97195]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:43.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 -- 172.21.15.187:0/1070858064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 msgr2=0x7f6d241009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:43.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 --2- 172.21.15.187:0/1070858064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d241009f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009910 tx=0x7f6d18009c20 comp rx=0 tx=0).stop 2023-08-11T13:55:43.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 -- 172.21.15.187:0/1070858064 shutdown_connections 2023-08-11T13:55:43.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 --2- 172.21.15.187:0/1070858064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d241009f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:43.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 -- 172.21.15.187:0/1070858064 >> 172.21.15.187:0/1070858064 conn(0x7f6d240fbee0 msgr2=0x7f6d240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:43.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 -- 172.21.15.187:0/1070858064 shutdown_connections 2023-08-11T13:55:43.751 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.747+0000 7f6d2b233700 1 -- 172.21.15.187:0/1070858064 wait complete. 2023-08-11T13:55:43.752 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.748+0000 7f6d2b233700 1 Processor -- start 2023-08-11T13:55:43.752 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.748+0000 7f6d2b233700 1 -- start start 2023-08-11T13:55:43.752 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d2b233700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:43.752 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d2b233700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2419bb60 con 0x7f6d24100610 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d28fcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d28fcf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.187:40012/0 (socket says 172.21.15.187:40012) 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d28fcf700 1 -- 172.21.15.187:0/2543978402 learned_addr learned my addr 172.21.15.187:0/2543978402 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d28fcf700 1 -- 172.21.15.187:0/2543978402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180095f0 con 0x7f6d24100610 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.749+0000 7f6d28fcf700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d18004f40 tx=0x7f6d18005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.750+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d1801d070 con 0x7f6d24100610 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.750+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2419bd60 con 0x7f6d24100610 2023-08-11T13:55:43.753 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.750+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d18004370 con 0x7f6d24100610 2023-08-11T13:55:43.754 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.750+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d180039b0 con 0x7f6d24100610 2023-08-11T13:55:43.754 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.750+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2419c180 con 0x7f6d24100610 2023-08-11T13:55:43.754 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.751+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d24061f50 con 0x7f6d24100610 2023-08-11T13:55:43.755 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.751+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f6d18003c70 con 0x7f6d24100610 2023-08-11T13:55:43.755 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.751+0000 7f6d21ffb700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 0x7f6d1403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:43.755 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.751+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d1804be30 con 0x7f6d24100610 2023-08-11T13:55:43.756 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.751+0000 7f6d23fff700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 0x7f6d1403a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:43.756 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.752+0000 7f6d23fff700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 0x7f6d1403a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:43.758 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:43.757+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6d1802a980 con 0x7f6d24100610 2023-08-11T13:55:44.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:43 smithi187 systemd[1]: Starting Ceph mon.c for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:55:44.068 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.065+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d24196bb0 con 0x7f6d24100610 2023-08-11T13:55:44.069 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:55:44.069 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":1,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","modified":"2023-08-11T13:53:56.953867Z","created":"2023-08-11T13:53:56.953867Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-08-11T13:55:44.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.066+0000 7f6d21ffb700 1 -- 172.21.15.187:0/2543978402 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f6d18026030 con 0x7f6d24100610 2023-08-11T13:55:44.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 msgr2=0x7f6d1403a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:44.070 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 0x7f6d1403a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).stop 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 msgr2=0x7f6d2419b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d18004f40 tx=0x7f6d18005020 comp rx=0 tx=0).stop 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 shutdown_connections 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d140381b0 0x7f6d1403a670 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 --2- 172.21.15.187:0/2543978402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d24100610 0x7f6d2419b620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 >> 172.21.15.187:0/2543978402 conn(0x7f6d240fbee0 msgr2=0x7f6d24106170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:44.071 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 shutdown_connections 2023-08-11T13:55:44.072 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:44.069+0000 7f6d2b233700 1 -- 172.21.15.187:0/2543978402 wait complete. 2023-08-11T13:55:44.072 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 1 2023-08-11T13:55:44.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:44 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/2543978402' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:44.693 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 podman[130931]: 2023-08-11T13:55:45.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T13:55:45.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T13:55:45.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pidfile_write: ignore empty --pid-file 2023-08-11T13:55:45.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: load: jerasure load: lrc 2023-08-11T13:55:45.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: RocksDB version: 7.9.2 2023-08-11T13:55:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Git sha 0 2023-08-11T13:55:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T13:55:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: DB SUMMARY 2023-08-11T13:55:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: DB Session ID: R29BAELM2JQW5LS0UWO0 2023-08-11T13:55:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: CURRENT file: CURRENT 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.error_if_exists: 0 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.create_if_missing: 0 2023-08-11T13:55:45.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.paranoid_checks: 1 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.env: 0x55b40c33bda0 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.info_log: 0x55b40ccbb460 2023-08-11T13:55:45.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.statistics: (nil) 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.use_fsync: 0 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_log_file_size: 0 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T13:55:45.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_fallocate: 1 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.use_direct_reads: 0 2023-08-11T13:55:45.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.db_log_dir: 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.wal_dir: 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T13:55:45.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T13:55:45.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T13:55:45.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T13:55:45.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T13:55:45.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T13:55:45.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.write_buffer_manager: 0x55b40cc743c0 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T13:55:45.054 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T13:55:45.055 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T13:55:45.055 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T13:55:45.055 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.unordered_write: 0 2023-08-11T13:55:45.055 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T13:55:45.056 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T13:55:45.056 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T13:55:45.056 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T13:55:45.056 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.row_cache: None 2023-08-11T13:55:45.056 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.wal_filter: None 2023-08-11T13:55:45.057 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T13:55:45.057 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T13:55:45.057 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.two_write_queues: 0 2023-08-11T13:55:45.057 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T13:55:45.058 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.wal_compression: 0 2023-08-11T13:55:45.058 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.atomic_flush: 0 2023-08-11T13:55:45.058 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T13:55:45.058 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T13:55:45.059 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T13:55:45.059 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.log_readahead_size: 0 2023-08-11T13:55:45.059 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T13:55:45.059 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T13:55:45.059 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_background_jobs: 2 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_background_compactions: -1 2023-08-11T13:55:45.060 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_subcompactions: 1 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T13:55:45.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_open_files: -1 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_background_flushes: -1 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Compression algorithms supported: 2023-08-11T13:55:45.062 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kZSTD supported: 0 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kXpressCompression supported: 0 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kZlibCompression supported: 1 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kSnappyCompression supported: 1 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kLZ4Compression supported: 1 2023-08-11T13:55:45.063 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: kBZip2Compression supported: 0 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.merge_operator: 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_filter: None 2023-08-11T13:55:45.064 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b40ccbb560) 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout: cache_index_and_filter_blocks: 1 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T13:55:45.065 INFO:journalctl@ceph.mon.c.smithi187.stdout: pin_top_level_index_and_filter: 1 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_type: 0 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: data_block_index_type: 0 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_shortening: 1 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: checksum: 4 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: no_block_cache: 0 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache: 0x55b40cd131f0 2023-08-11T13:55:45.066 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_name: BinnedLRUCache 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_options: 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: capacity : 536870912 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: num_shard_bits : 4 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: strict_capacity_limit : 0 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: high_pri_pool_ratio: 0.000 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_compressed: (nil) 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: persistent_cache: (nil) 2023-08-11T13:55:45.067 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_size: 4096 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_size_deviation: 10 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_restart_interval: 16 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_block_restart_interval: 1 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: metadata_block_size: 4096 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: partition_filters: 0 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: use_delta_encoding: 1 2023-08-11T13:55:45.068 INFO:journalctl@ceph.mon.c.smithi187.stdout: filter_policy: bloomfilter 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: whole_key_filtering: 1 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: verify_compression: 0 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: read_amp_bytes_per_bit: 0 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: format_version: 5 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: enable_index_compression: 1 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_align: 0 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: max_auto_readahead_size: 262144 2023-08-11T13:55:45.069 INFO:journalctl@ceph.mon.c.smithi187.stdout: prepopulate_block_cache: 0 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout: initial_auto_readahead_size: 8192 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression: NoCompression 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T13:55:45.070 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.num_levels: 7 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T13:55:45.071 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T13:55:45.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:55:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T13:55:45.074 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T13:55:45.075 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T13:55:45.076 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T13:55:45.077 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T13:55:45.078 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.table_properties_collectors: 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.inplace_update_support: 0 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T13:55:45.079 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.bloom_locality: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.max_successive_merges: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.ttl: 2592000 2023-08-11T13:55:45.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enable_blob_files: false 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.min_blob_size: 0 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T13:55:45.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80b339e9-2995-40f4-b4af-c2b9dbb1ef8a 2023-08-11T13:55:45.082 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762144789374, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762144789802, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691762144, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80b339e9-2995-40f4-b4af-c2b9dbb1ef8a", "db_session_id": "R29BAELM2JQW5LS0UWO0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762144789869, "job": 1, "event": "recovery_finished"} 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b40cdb0000 2023-08-11T13:55:45.083 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: DB pointer 0x55b40cd9c000 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** DB Stats ** 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:55:45.084 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** Compaction Stats [default] ** 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T13:55:45.085 INFO:journalctl@ceph.mon.c.smithi187.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** Compaction Stats [default] ** 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-08-11T13:55:45.086 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T13:55:45.087 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: Block cache BinnedLRUCache@0x55b40cd131f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T13:55:45.088 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: using public_addr v2:172.21.15.187:0/0 -> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: starting mon.c rank -1 at public addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] at bind addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(???) e0 preinit fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 bash[130931]: b9de1bdbe0e4d8ee98460f5faefd6fc1a592ba4021fb728d2b2620321153d174 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).mds e1 new map 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).mds e1 print_map 2023-08-11T13:55:45.089 INFO:journalctl@ceph.mon.c.smithi187.stdout: e1 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout: legacy client fscid: -1 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout: No filesystems configured 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-11T13:55:45.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:45.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:45.093 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Deploying cephadm binary to smithi187 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.094 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Added host smithi187 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:45.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:45.096 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.097 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:45.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c;count:3 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: Deploying daemon mon.c on smithi187 2023-08-11T13:55:45.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:45.100 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/2543978402' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expand map: {default=false} 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta from 'false' to 'false' 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expanded map: {default=false} 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expand map: {default=info} 2023-08-11T13:55:45.101 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta from 'info' to 'info' 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expanded map: {default=info} 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expand map: {default=daemon} 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expand map: {default=debug} 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 ceph-mon[130973]: expand_channel_meta expanded map: {default=debug} 2023-08-11T13:55:45.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:44 smithi187 systemd[1]: Started Ceph mon.c for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:55:45.448 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T13:55:45.448 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T13:55:46.000 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:55:50.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 -- 172.21.15.187:0/4290799896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 msgr2=0x7f8d34005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:50.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/4290799896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d34005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d480ff300 tx=0x7f8d3800feb0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 -- 172.21.15.187:0/4290799896 shutdown_connections 2023-08-11T13:55:50.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/4290799896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d34005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 -- 172.21.15.187:0/4290799896 >> 172.21.15.187:0/4290799896 conn(0x7f8d480faee0 msgr2=0x7f8d480fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:50.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 -- 172.21.15.187:0/4290799896 shutdown_connections 2023-08-11T13:55:50.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.202+0000 7f8d4fb63700 1 -- 172.21.15.187:0/4290799896 wait complete. 2023-08-11T13:55:50.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 Processor -- start 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 -- start start 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d481967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4819f850 con 0x7f8d340035c0 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4fb63700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4819f990 con 0x7f8d48196d20 2023-08-11T13:55:50.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:54858/0 (socket says 172.21.15.187:54858) 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 -- 172.21.15.187:0/1721873675 learned_addr learned my addr 172.21.15.187:0/1721873675 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 -- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 msgr2=0x7f8d481971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 -- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 msgr2=0x7f8d481971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.203+0000 7f8d4d0fe700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:55:50.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4d8ff700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d481967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:50.208 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4d8ff700 1 -- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 msgr2=0x7f8d481971a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:55:50.208 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4d8ff700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.208 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4d8ff700 1 -- 172.21.15.187:0/1721873675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3800f900 con 0x7f8d340035c0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4d8ff700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d481967e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800f800 tx=0x7f8d38005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8d38013b00 con 0x7f8d340035c0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d38004b70 con 0x7f8d340035c0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8d38003b30 con 0x7f8d340035c0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.204+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4819fb90 con 0x7f8d340035c0 2023-08-11T13:55:50.209 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.205+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481a00e0 con 0x7f8d340035c0 2023-08-11T13:55:50.210 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.205+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f8d38004030 con 0x7f8d340035c0 2023-08-11T13:55:50.210 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.206+0000 7f8d3effd700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 0x7f8d2c03a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:50.210 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.206+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d3804d540 con 0x7f8d340035c0 2023-08-11T13:55:50.211 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.206+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d48061f50 con 0x7f8d340035c0 2023-08-11T13:55:50.211 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.206+0000 7f8d4d0fe700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 0x7f8d2c03a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:50.211 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.207+0000 7f8d4d0fe700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 0x7f8d2c03a4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009180 tx=0x7f8d44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:50.212 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.211+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8d38013c60 con 0x7f8d340035c0 2023-08-11T13:55:50.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:50.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: mon.a calling monitor election 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: mon.c calling monitor election 2023-08-11T13:55:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: fsmap 2023-08-11T13:55:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: mgrmap e13: x(active, since 47s) 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: overall HEALTH_OK 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:50.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:50.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:50.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: mon.a calling monitor election 2023-08-11T13:55:50.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:50.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: mon.c calling monitor election 2023-08-11T13:55:50.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:50.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:50.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:55:50.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:55:50.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: fsmap 2023-08-11T13:55:50.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:50.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: mgrmap e13: x(active, since 47s) 2023-08-11T13:55:50.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: overall HEALTH_OK 2023-08-11T13:55:50.301 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.301 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.301 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:50.302 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:50.302 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:50.481 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.480+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d48068f60 con 0x7f8d340035c0 2023-08-11T13:55:50.481 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.481+0000 7f8d3effd700 1 -- 172.21.15.187:0/1721873675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f8d3804a020 con 0x7f8d340035c0 2023-08-11T13:55:50.482 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:55:50.482 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":2,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","modified":"2023-08-11T13:55:44.821933Z","created":"2023-08-11T13:53:56.953867Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 msgr2=0x7f8d2c03a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 0x7f8d2c03a4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009180 tx=0x7f8d44008040 comp rx=0 tx=0).stop 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 msgr2=0x7f8d481967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d481967e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800f800 tx=0x7f8d38005250 comp rx=0 tx=0).stop 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 shutdown_connections 2023-08-11T13:55:50.484 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8d2c037fe0 0x7f8d2c03a4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d340035c0 0x7f8d481967e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 --2- 172.21.15.187:0/1721873675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d48196d20 0x7f8d481971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 >> 172.21.15.187:0/1721873675 conn(0x7f8d480faee0 msgr2=0x7f8d4818d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 shutdown_connections 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:50.483+0000 7f8d4fb63700 1 -- 172.21.15.187:0/1721873675 wait complete. 2023-08-11T13:55:50.485 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 2 2023-08-11T13:55:51.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:50 smithi003 ceph-mon[97195]: Deploying daemon mon.b on smithi103 2023-08-11T13:55:51.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:50 smithi003 ceph-mon[97195]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:51.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/1721873675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:51.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:51.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:50 smithi187 ceph-mon[130973]: Deploying daemon mon.b on smithi103 2023-08-11T13:55:51.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:50 smithi187 ceph-mon[130973]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:51.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/1721873675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:51.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:51.862 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-08-11T13:55:51.863 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph mon dump -f json 2023-08-11T13:55:52.420 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:55:52.595 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 systemd[1]: Starting Ceph mon.b for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:55:52.882 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 podman[130394]: 2023-08-11T13:55:52.882 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: set uid:gid to 167:167 (ceph:ceph) 2023-08-11T13:55:52.882 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev), process ceph-mon, pid 7 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 bash[130394]: e2ad3ee33b2827eb6d463ec3b461544580068ce075b13c8e93774a1086a33a58 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pidfile_write: ignore empty --pid-file 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: load: jerasure load: lrc 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: RocksDB version: 7.9.2 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Git sha 0 2023-08-11T13:55:52.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Compile date 2023-08-10 15:31:53 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: DB SUMMARY 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: DB Session ID: 4ETTJR8EXBEQMYAS8H12 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: CURRENT file: CURRENT 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: IDENTITY file: IDENTITY 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-08-11T13:55:52.884 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-08-11T13:55:52.885 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.error_if_exists: 0 2023-08-11T13:55:52.885 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.create_if_missing: 0 2023-08-11T13:55:52.885 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.paranoid_checks: 1 2023-08-11T13:55:52.885 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.flush_verify_memtable_count: 1 2023-08-11T13:55:52.885 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-08-11T13:55:52.886 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-08-11T13:55:52.886 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.env: 0x556e03469da0 2023-08-11T13:55:52.886 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.fs: PosixFileSystem 2023-08-11T13:55:52.886 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.info_log: 0x556e0538b460 2023-08-11T13:55:52.886 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_file_opening_threads: 16 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.statistics: (nil) 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.use_fsync: 0 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_log_file_size: 0 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.log_file_time_to_roll: 0 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.keep_log_file_num: 1000 2023-08-11T13:55:52.887 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.recycle_log_file_num: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_fallocate: 1 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_mmap_reads: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_mmap_writes: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.use_direct_reads: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.create_missing_column_families: 0 2023-08-11T13:55:52.888 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.db_log_dir: 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.wal_dir: 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.table_cache_numshardbits: 6 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.WAL_ttl_seconds: 0 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.WAL_size_limit_MB: 0 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-08-11T13:55:52.889 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.is_fd_close_on_exec: 1 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.advise_random_on_open: 1 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.db_write_buffer_size: 0 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.write_buffer_manager: 0x556e053443c0 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-08-11T13:55:52.890 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.use_adaptive_mutex: 0 2023-08-11T13:55:52.891 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.rate_limiter: (nil) 2023-08-11T13:55:52.891 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-08-11T13:55:52.891 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.wal_recovery_mode: 2 2023-08-11T13:55:52.892 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enable_thread_tracking: 0 2023-08-11T13:55:52.892 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enable_pipelined_write: 0 2023-08-11T13:55:52.892 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.unordered_write: 0 2023-08-11T13:55:52.892 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.row_cache: None 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.wal_filter: None 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-08-11T13:55:52.893 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_ingest_behind: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.two_write_queues: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.manual_wal_flush: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.wal_compression: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.atomic_flush: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.persist_stats_to_disk: 0 2023-08-11T13:55:52.894 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.write_dbid_to_manifest: 0 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.log_readahead_size: 0 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.best_efforts_recovery: 0 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.allow_data_in_errors: 0 2023-08-11T13:55:52.895 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.db_host_id: __hostname__ 2023-08-11T13:55:52.896 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enforce_single_del_contracts: true 2023-08-11T13:55:52.896 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_background_jobs: 2 2023-08-11T13:55:52.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_background_compactions: -1 2023-08-11T13:55:52.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_subcompactions: 1 2023-08-11T13:55:52.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-08-11T13:55:52.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-08-11T13:55:52.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.delayed_write_rate : 16777216 2023-08-11T13:55:52.898 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_total_wal_size: 0 2023-08-11T13:55:52.898 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-08-11T13:55:52.898 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.stats_dump_period_sec: 600 2023-08-11T13:55:52.898 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.stats_persist_period_sec: 600 2023-08-11T13:55:52.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-08-11T13:55:52.899 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_open_files: -1 2023-08-11T13:55:52.900 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bytes_per_sync: 0 2023-08-11T13:55:52.900 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.wal_bytes_per_sync: 0 2023-08-11T13:55:52.900 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.strict_bytes_per_sync: 0 2023-08-11T13:55:52.900 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_readahead_size: 0 2023-08-11T13:55:52.900 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_background_flushes: -1 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Compression algorithms supported: 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kZSTD supported: 0 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kXpressCompression supported: 0 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kLZ4HCCompression supported: 1 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kZlibCompression supported: 1 2023-08-11T13:55:52.901 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kSnappyCompression supported: 1 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kLZ4Compression supported: 1 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: kBZip2Compression supported: 0 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Fast CRC32 supported: Supported on x86 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: DMutex implementation: pthread_mutex_t 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-08-11T13:55:52.902 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.merge_operator: 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_filter: None 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_filter_factory: None 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.sst_partitioner_factory: None 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.memtable_factory: SkipListFactory 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.table_factory: BlockBasedTable 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e0538b560) 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout: cache_index_and_filter_blocks: 1 2023-08-11T13:55:52.903 INFO:journalctl@ceph.mon.b.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: pin_top_level_index_and_filter: 1 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_type: 0 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: data_block_index_type: 0 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_shortening: 1 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2023-08-11T13:55:52.904 INFO:journalctl@ceph.mon.b.smithi103.stdout: checksum: 4 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: no_block_cache: 0 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache: 0x556e053e31f0 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_name: BinnedLRUCache 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_options: 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: capacity : 536870912 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: num_shard_bits : 4 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: strict_capacity_limit : 0 2023-08-11T13:55:52.905 INFO:journalctl@ceph.mon.b.smithi103.stdout: high_pri_pool_ratio: 0.000 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_cache_compressed: (nil) 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: persistent_cache: (nil) 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_size: 4096 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_size_deviation: 10 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_restart_interval: 16 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: index_block_restart_interval: 1 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: metadata_block_size: 4096 2023-08-11T13:55:52.906 INFO:journalctl@ceph.mon.b.smithi103.stdout: partition_filters: 0 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: use_delta_encoding: 1 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: filter_policy: bloomfilter 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: whole_key_filtering: 1 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: verify_compression: 0 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: read_amp_bytes_per_bit: 0 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: format_version: 5 2023-08-11T13:55:52.907 INFO:journalctl@ceph.mon.b.smithi103.stdout: enable_index_compression: 1 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout: block_align: 0 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout: max_auto_readahead_size: 262144 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout: prepopulate_block_cache: 0 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout: initial_auto_readahead_size: 8192 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.write_buffer_size: 33554432 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_write_buffer_number: 2 2023-08-11T13:55:52.908 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression: NoCompression 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression: Disabled 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.prefix_extractor: nullptr 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.num_levels: 7 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-08-11T13:55:52.909 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-08-11T13:55:52.910 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.window_bits: -14 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.level: 32767 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.strategy: 0 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-08-11T13:55:52.911 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.enabled: false 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.target_file_size_base: 67108864 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.target_file_size_multiplier: 1 2023-08-11T13:55:52.912 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-08-11T13:55:52.913 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.arena_block_size: 1048576 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-08-11T13:55:52.914 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.disable_auto_compactions: 0 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-08-11T13:55:52.915 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.table_properties_collectors: 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.inplace_update_support: 0 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.inplace_update_num_locks: 10000 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-08-11T13:55:52.916 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.memtable_huge_page_size: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.bloom_locality: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.max_successive_merges: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.optimize_filters_for_hits: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.paranoid_file_checks: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.force_consistency_checks: 1 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.report_bg_io_stats: 0 2023-08-11T13:55:52.917 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.ttl: 2592000 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.periodic_compaction_seconds: 0 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enable_blob_files: false 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.min_blob_size: 0 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_file_size: 268435456 2023-08-11T13:55:52.918 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_compression_type: NoCompression 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.enable_blob_garbage_collection: false 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.blob_file_starting_level: 0 2023-08-11T13:55:52.919 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-08-11T13:55:52.920 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d5ef5e8c-4ca4-4889-92ea-18bbc058789b 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762152798136, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762152798696, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1691762152, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d5ef5e8c-4ca4-4889-92ea-18bbc058789b", "db_session_id": "4ETTJR8EXBEQMYAS8H12", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: EVENT_LOG_v1 {"time_micros": 1691762152798785, "job": 1, "event": "recovery_finished"} 2023-08-11T13:55:52.921 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e05480000 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: DB pointer 0x556e0546c000 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** DB Stats ** 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:55:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** Compaction Stats [default] ** 2023-08-11T13:55:52.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-08-11T13:55:52.925 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-08-11T13:55:52.926 INFO:journalctl@ceph.mon.b.smithi103.stdout: Block cache BinnedLRUCache@0x556e053e31f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: using public_addr v2:172.21.15.103:0/0 -> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: starting mon.b rank -1 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(???) e0 preinit fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:55:52.927 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).mds e1 new map 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).mds e1 print_map 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: e1 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: legacy client fscid: -1 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: 2023-08-11T13:55:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout: No filesystems configured 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-08-11T13:55:52.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:52.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Deploying cephadm binary to smithi187 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Added host smithi187 2023-08-11T13:55:52.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:55:52.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.client.admin.keyring 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.936 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3360283583' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:55:52.937 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c;count:3 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Deploying daemon mon.c on smithi187 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/2543978402' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:52.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.a calling monitor election 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.c calling monitor election 2023-08-11T13:55:52.939 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: monmap e2: 2 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: fsmap 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mgrmap e13: x(active, since 47s) 2023-08-11T13:55:52.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: overall HEALTH_OK 2023-08-11T13:55:52.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:52.941 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: Deploying daemon mon.b on smithi103 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/1721873675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-08-11T13:55:52.942 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expand map: {default=false} 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta from 'false' to 'false' 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expanded map: {default=false} 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expand map: {default=info} 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta from 'info' to 'info' 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expanded map: {default=info} 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expand map: {default=daemon} 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta from 'daemon' to 'daemon' 2023-08-11T13:55:52.943 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expanded map: {default=daemon} 2023-08-11T13:55:52.944 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expand map: {default=debug} 2023-08-11T13:55:52.944 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta from 'debug' to 'debug' 2023-08-11T13:55:52.944 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: expand_channel_meta expanded map: {default=debug} 2023-08-11T13:55:52.944 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 ceph-mon[130434]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-08-11T13:55:52.944 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:52 smithi103 systemd[1]: Started Ceph mon.b for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:55:57.998 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 -- 172.21.15.187:0/2882105595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec0106580 msgr2=0x7f9eac005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:57.998 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/2882105595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec0106580 0x7f9eac005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005fa0 tx=0x7f9eb000ff70 comp rx=0 tx=0).stop 2023-08-11T13:55:57.998 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 -- 172.21.15.187:0/2882105595 shutdown_connections 2023-08-11T13:55:57.999 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/2882105595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec0106580 0x7f9eac005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:57.999 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 -- 172.21.15.187:0/2882105595 >> 172.21.15.187:0/2882105595 conn(0x7f9ec0074ed0 msgr2=0x7f9ec00752e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:57.999 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 -- 172.21.15.187:0/2882105595 shutdown_connections 2023-08-11T13:55:57.999 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.997+0000 7f9ec7a54700 1 -- 172.21.15.187:0/2882105595 wait complete. 2023-08-11T13:55:58.002 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 Processor -- start 2023-08-11T13:55:58.002 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 -- start start 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 0x7f9ec0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec019e540 con 0x7f9ec019dee0 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec01a47c0 con 0x7f9ec0197f60 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec7a54700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec01a4950 con 0x7f9ec0106580 2023-08-11T13:55:58.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec57f0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec57f0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:54882/0 (socket says 172.21.15.187:54882) 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec57f0700 1 -- 172.21.15.187:0/3527927118 learned_addr learned my addr 172.21.15.187:0/3527927118 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.998+0000 7f9ec4fef700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 0x7f9ec0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec57f0700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 msgr2=0x7f9ec019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec57f0700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 msgr2=0x7f9ec019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T13:55:58.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec57f0700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec57f0700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec5ff1700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec5ff1700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 msgr2=0x7f9ec0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec5ff1700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 msgr2=0x7f9ec0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec5ff1700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec5ff1700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 msgr2=0x7f9ec019d9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:55:58.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 msgr2=0x7f9ec0198410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 -- 172.21.15.187:0/3527927118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb000f970 con 0x7f9ec019dee0 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9ec4fef700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 0x7f9ec0197a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b550 tx=0x7f9ebc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:57.999+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc004230 con 0x7f9ec019dee0 2023-08-11T13:55:58.006 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.000+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ebc004390 con 0x7f9ec019dee0 2023-08-11T13:55:58.007 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.000+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc011af0 con 0x7f9ec019dee0 2023-08-11T13:55:58.007 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.000+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec01a4c30 con 0x7f9ec019dee0 2023-08-11T13:55:58.007 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.000+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec01a50e0 con 0x7f9ec019dee0 2023-08-11T13:55:58.007 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.001+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f9ebc011c50 con 0x7f9ec019dee0 2023-08-11T13:55:58.008 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.001+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec0061f50 con 0x7f9ec019dee0 2023-08-11T13:55:58.008 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.001+0000 7f9eb67fc700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 0x7f9ea403ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:55:58.008 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.001+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ebc04a580 con 0x7f9ec019dee0 2023-08-11T13:55:58.008 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.001+0000 7f9ec57f0700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 0x7f9ea403ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:55:58.008 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.002+0000 7f9ec57f0700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 0x7f9ea403ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005850 tx=0x7f9eb000f3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:55:58.009 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.007+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9ebc0214d0 con 0x7f9ec019dee0 2023-08-11T13:55:58.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: mon.c calling monitor election 2023-08-11T13:55:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: mon.a calling monitor election 2023-08-11T13:55:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: fsmap 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: mgrmap e13: x(active, since 55s) 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T13:55:58.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: mon.b (rank 2) addr [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] is down (out of quorum) 2023-08-11T13:55:58.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:58.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:58.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: mon.c calling monitor election 2023-08-11T13:55:58.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:55:58.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: mon.a calling monitor election 2023-08-11T13:55:58.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:55:58.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: fsmap 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: mgrmap e13: x(active, since 55s) 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T13:55:58.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T13:55:58.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T13:55:58.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: mon.b (rank 2) addr [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] is down (out of quorum) 2023-08-11T13:55:58.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:58.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:58.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.301+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ec0198f30 con 0x7f9ec019dee0 2023-08-11T13:55:58.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.302+0000 7f9eb67fc700 1 -- 172.21.15.187:0/3527927118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f9ebc0212f0 con 0x7f9ec019dee0 2023-08-11T13:55:58.303 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-08-11T13:55:58.303 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":3,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","modified":"2023-08-11T13:55:52.836336Z","created":"2023-08-11T13:53:56.953867Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-08-11T13:55:58.304 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 msgr2=0x7f9ea403ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 0x7f9ea403ab40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005850 tx=0x7f9eb000f3e0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 msgr2=0x7f9ec0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 0x7f9ec0197a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b550 tx=0x7f9ebc00b910 comp rx=0 tx=0).stop 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 shutdown_connections 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ea4038680 0x7f9ea403ab40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.305 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ec0197f60 0x7f9ec0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec0106580 0x7f9ec019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 --2- 172.21.15.187:0/3527927118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ec019dee0 0x7f9ec0197a20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 >> 172.21.15.187:0/3527927118 conn(0x7f9ec0074ed0 msgr2=0x7f9ec00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 shutdown_connections 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:55:58.304+0000 7f9ec7a54700 1 -- 172.21.15.187:0/3527927118 wait complete. 2023-08-11T13:55:58.306 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 3 2023-08-11T13:55:58.671 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-08-11T13:55:58.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph config generate-minimal-conf 2023-08-11T13:55:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:59.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:59.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/3527927118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:59.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:55:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:59.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:59.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:55:59.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:55:59.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:55:59.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/3527927118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:55:59.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:55:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:55:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:56:00.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: mon.c calling monitor election 2023-08-11T13:56:00.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: mon.a calling monitor election 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:56:00.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: fsmap 2023-08-11T13:56:00.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:56:00.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: mgrmap e13: x(active, since 55s) 2023-08-11T13:56:00.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-08-11T13:56:00.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-08-11T13:56:00.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-08-11T13:56:00.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: mon.b (rank 2) addr [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] is down (out of quorum) 2023-08-11T13:56:00.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:00.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:00.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:00.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:00.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:00.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:00.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/3527927118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-08-11T13:56:00.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:55:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 -- 172.21.15.3:0/2098471331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40100610 msgr2=0x7f5c401009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:00.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 --2- 172.21.15.3:0/2098471331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40100610 0x7f5c401009f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009910 tx=0x7f5c3c009c20 comp rx=0 tx=0).stop 2023-08-11T13:56:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 -- 172.21.15.3:0/2098471331 shutdown_connections 2023-08-11T13:56:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 --2- 172.21.15.3:0/2098471331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40100610 0x7f5c401009f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 -- 172.21.15.3:0/2098471331 >> 172.21.15.3:0/2098471331 conn(0x7f5c400fbee0 msgr2=0x7f5c400fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:00.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 -- 172.21.15.3:0/2098471331 shutdown_connections 2023-08-11T13:56:00.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.995+0000 7f5c48764700 1 -- 172.21.15.3:0/2098471331 wait complete. 2023-08-11T13:56:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.996+0000 7f5c48764700 1 Processor -- start 2023-08-11T13:56:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.996+0000 7f5c48764700 1 -- start start 2023-08-11T13:56:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c40100610 0x7f5c4019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:01.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4019df10 0x7f5c40197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4019e570 con 0x7f5c40197f90 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401a47f0 con 0x7f5c40100610 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c48764700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c401a4980 con 0x7f5c4019df10 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c46d01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c46d01700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57120/0 (socket says 172.21.15.3:57120) 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c46d01700 1 -- 172.21.15.3:0/3577889391 learned_addr learned my addr 172.21.15.3:0/3577889391 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:56:01.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c45cff700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4019df10 0x7f5c40197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.997+0000 7f5c46500700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c40100610 0x7f5c4019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 -- 172.21.15.3:0/3577889391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4019df10 msgr2=0x7f5c40197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4019df10 0x7f5c40197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 -- 172.21.15.3:0/3577889391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c40100610 msgr2=0x7f5c4019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:01.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c40100610 0x7f5c4019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 -- 172.21.15.3:0/3577889391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c0095f0 con 0x7f5c40197f90 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c46d01700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800ba50 tx=0x7f5c3800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800c930 con 0x7f5c40197f90 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c38014440 con 0x7f5c40197f90 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380075b0 con 0x7f5c40197f90 2023-08-11T13:56:01.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c401a4c60 con 0x7f5c40197f90 2023-08-11T13:56:01.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.998+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c401a5110 con 0x7f5c40197f90 2023-08-11T13:56:01.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.999+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f5c380145b0 con 0x7f5c40197f90 2023-08-11T13:56:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:00.999+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c40061f50 con 0x7f5c40197f90 2023-08-11T13:56:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.000+0000 7f5c377fe700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 0x7f5c2c03ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.000+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c3804a280 con 0x7f5c40197f90 2023-08-11T13:56:01.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.000+0000 7f5c46500700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 0x7f5c2c03ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.001+0000 7f5c46500700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 0x7f5c2c03ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fd0 tx=0x7f5c3c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:01.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.005+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5c3800cc20 con 0x7f5c40197f90 2023-08-11T13:56:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mon.b calling monitor election 2023-08-11T13:56:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Updating smithi003:/etc/ceph/ceph.conf 2023-08-11T13:56:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:56:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:56:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mon.b calling monitor election 2023-08-11T13:56:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mon.c calling monitor election 2023-08-11T13:56:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mon.a calling monitor election 2023-08-11T13:56:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T13:56:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:56:01.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: fsmap 2023-08-11T13:56:01.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:56:01.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: mgrmap e13: x(active, since 57s) 2023-08-11T13:56:01.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Cluster is now healthy 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: overall HEALTH_OK 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:01.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mon.b calling monitor election 2023-08-11T13:56:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Updating smithi003:/etc/ceph/ceph.conf 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mon.b calling monitor election 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mon.c calling monitor election 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mon.a calling monitor election 2023-08-11T13:56:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: fsmap 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: mgrmap e13: x(active, since 57s) 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T13:56:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Cluster is now healthy 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: overall HEALTH_OK 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.217+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5c40198f60 con 0x7f5c40197f90 2023-08-11T13:56:01.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.217+0000 7f5c377fe700 1 -- 172.21.15.3:0/3577889391 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f5c38012340 con 0x7f5c40197f90 2023-08-11T13:56:01.220 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:56:01.220 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-08-11T13:56:01.220 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T13:56:01.220 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2023-08-11T13:56:01.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 msgr2=0x7f5c2c03ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:01.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 0x7f5c2c03ab40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fd0 tx=0x7f5c3c0058e0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 msgr2=0x7f5c40198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800ba50 tx=0x7f5c3800bd60 comp rx=0 tx=0).stop 2023-08-11T13:56:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 shutdown_connections 2023-08-11T13:56:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c40197f90 0x7f5c40198440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c2c038680 0x7f5c2c03ab40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c40100610 0x7f5c4019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 --2- 172.21.15.3:0/3577889391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c4019df10 0x7f5c40197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 >> 172.21.15.3:0/3577889391 conn(0x7f5c400fbee0 msgr2=0x7f5c40103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 shutdown_connections 2023-08-11T13:56:01.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:01.220+0000 7f5c48764700 1 -- 172.21.15.3:0/3577889391 wait complete. 2023-08-11T13:56:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mon.b calling monitor election 2023-08-11T13:56:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Updating smithi003:/etc/ceph/ceph.conf 2023-08-11T13:56:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Updating smithi103:/etc/ceph/ceph.conf 2023-08-11T13:56:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Updating smithi187:/etc/ceph/ceph.conf 2023-08-11T13:56:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mon.b calling monitor election 2023-08-11T13:56:01.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mon.c calling monitor election 2023-08-11T13:56:01.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mon.a calling monitor election 2023-08-11T13:56:01.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-08-11T13:56:01.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: monmap e3: 3 mons at {a=[v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0],b=[v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: fsmap 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: osdmap e4: 0 total, 0 up, 0 in 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: mgrmap e13: x(active, since 57s) 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Cluster is now healthy 2023-08-11T13:56:01.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: overall HEALTH_OK 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Updating smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: Updating smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.613 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-08-11T13:56:01.613 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:56:01.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T13:56:01.689 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:56:01.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:56:01.805 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:56:01.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T13:56:01.841 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:56:01.842 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:56:01.912 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:56:01.912 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.conf 2023-08-11T13:56:01.948 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:56:01.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-08-11T13:56:01.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: Updating smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:01.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:01.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:01.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:01.991 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:01.991 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: Reconfiguring daemon mon.a on smithi003 2023-08-11T13:56:01.991 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:01 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3577889391' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:02.021 INFO:tasks.cephadm:Adding mgr.x on smithi003 2023-08-11T13:56:02.022 INFO:tasks.cephadm:Adding mgr.y on smithi103 2023-08-11T13:56:02.022 INFO:tasks.cephadm:Adding mgr.z on smithi187 2023-08-11T13:56:02.023 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch apply mgr '3;smithi003=x;smithi103=y;smithi187=z' 2023-08-11T13:56:02.088 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: Updating smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.089 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:02.090 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: Reconfiguring daemon mon.a on smithi003 2023-08-11T13:56:02.091 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:01 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3577889391' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:02.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: Updating smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/config/ceph.conf 2023-08-11T13:56:02.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: Reconfiguring mon.a (unknown last config time)... 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: Reconfiguring daemon mon.a on smithi003 2023-08-11T13:56:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:01 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3577889391' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:02.616 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:56:03.397 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:03.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:03.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:03.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:03.782 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 -- 172.21.15.187:0/2643281124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820100ac0 msgr2=0x7f1820100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:03.782 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 --2- 172.21.15.187:0/2643281124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820100ac0 0x7f1820100ee0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1810009910 tx=0x7f1810009c20 comp rx=0 tx=0).stop 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 -- 172.21.15.187:0/2643281124 shutdown_connections 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 --2- 172.21.15.187:0/2643281124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820100ac0 0x7f1820100ee0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 -- 172.21.15.187:0/2643281124 >> 172.21.15.187:0/2643281124 conn(0x7f18200fa160 msgr2=0x7f18200fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 -- 172.21.15.187:0/2643281124 shutdown_connections 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.781+0000 7f1828767700 1 -- 172.21.15.187:0/2643281124 wait complete. 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 Processor -- start 2023-08-11T13:56:03.783 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 -- start start 2023-08-11T13:56:03.784 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:03.784 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820198540 0x7f182019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:03.785 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f182019aec0 0x7f182019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:03.785 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182019daf0 con 0x7f1820198540 2023-08-11T13:56:03.785 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182019dc60 con 0x7f182019aec0 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1828767700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182019de20 con 0x7f1820100ac0 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:54892/0 (socket says 172.21.15.187:54892) 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826d04700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f182019aec0 0x7f182019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 -- 172.21.15.187:0/2330564386 learned_addr learned my addr 172.21.15.187:0/2330564386 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1825d02700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820198540 0x7f182019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:03.786 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 -- 172.21.15.187:0/2330564386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f182019aec0 msgr2=0x7f182019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f182019aec0 0x7f182019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 -- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820198540 msgr2=0x7f182019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820198540 0x7f182019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.782+0000 7f1826503700 1 -- 172.21.15.187:0/2330564386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18100095f0 con 0x7f1820100ac0 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f1826503700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1810004cb0 tx=0x7f1810004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181001d070 con 0x7f1820100ac0 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182019e0a0 con 0x7f1820100ac0 2023-08-11T13:56:03.787 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f181000be20 con 0x7f1820100ac0 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810022e70 con 0x7f1820100ac0 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.783+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182019e610 con 0x7f1820100ac0 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.784+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f1810022570 con 0x7f1820100ac0 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.784+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182018dfb0 con 0x7f1820100ac0 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.784+0000 7f18177fe700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 0x7f180c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:03.788 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.784+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f181004beb0 con 0x7f1820100ac0 2023-08-11T13:56:03.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.784+0000 7f1825d02700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 0x7f180c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:03.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.785+0000 7f1825d02700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 0x7f180c03aaf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18200ff660 tx=0x7f181c009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:03.789 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.789+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f181002a540 con 0x7f1820100ac0 2023-08-11T13:56:03.996 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:03.993+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi103=y;smithi187=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f182002cce0 con 0x7f180c038630 2023-08-11T13:56:04.001 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.000+0000 7f18177fe700 1 -- 172.21.15.187:0/2330564386 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f182002cce0 con 0x7f180c038630 2023-08-11T13:56:04.002 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mgr update... 2023-08-11T13:56:04.003 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 msgr2=0x7f180c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 0x7f180c03aaf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f18200ff660 tx=0x7f181c009210 comp rx=0 tx=0).stop 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 msgr2=0x7f18200ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1810004cb0 tx=0x7f1810004ce0 comp rx=0 tx=0).stop 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 shutdown_connections 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f180c038630 0x7f180c03aaf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:04.004 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f182019aec0 0x7f182019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:04.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1820100ac0 0x7f18200ff0e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:04.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 --2- 172.21.15.187:0/2330564386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1820198540 0x7f182019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:04.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 >> 172.21.15.187:0/2330564386 conn(0x7f18200fa160 msgr2=0x7f18200faea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:04.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 shutdown_connections 2023-08-11T13:56:04.005 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:04.003+0000 7f1828767700 1 -- 172.21.15.187:0/2330564386 wait complete. 2023-08-11T13:56:04.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: Reconfiguring mon.b (monmap changed)... 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: Reconfiguring daemon mon.b on smithi103 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:04.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.359 DEBUG:teuthology.orchestra.run.smithi103:mgr.y> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.y.service 2023-08-11T13:56:04.363 DEBUG:teuthology.orchestra.run.smithi187:mgr.z> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.z.service 2023-08-11T13:56:04.393 INFO:journalctl@ceph.mgr.y.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T13:56:04.402 INFO:tasks.cephadm:Deploying OSDs... 2023-08-11T13:56:04.403 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T13:56:04.403 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T13:56:04.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: Reconfiguring mon.b (monmap changed)... 2023-08-11T13:56:04.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: Reconfiguring daemon mon.b on smithi103 2023-08-11T13:56:04.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:04.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:04.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:04.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.431 INFO:journalctl@ceph.mgr.z.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T13:56:04.453 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T13:56:04.453 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2023-08-11T13:56:04.501 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T13:56:04.502 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:04.502 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-08-11T13:56:04.502 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:04.503 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:04.503 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-08-11 13:55:14.149251154 +0000 2023-08-11T13:56:04.503 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-08-11 13:41:19.371815805 +0000 2023-08-11T13:56:04.503 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-08-11 13:41:19.371815805 +0000 2023-08-11T13:56:04.504 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2023-08-11 13:41:19.344816142 +0000 2023-08-11T13:56:04.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T13:56:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: Reconfiguring mon.b (monmap changed)... 2023-08-11T13:56:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: Reconfiguring daemon mon.b on smithi103 2023-08-11T13:56:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-08-11T13:56:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-08-11T13:56:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:04.573 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-08-11T13:56:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-08-11T13:56:04.574 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00031243 s, 1.6 MB/s 2023-08-11T13:56:04.576 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T13:56:04.662 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2023-08-11T13:56:04.746 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T13:56:04.746 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:04.746 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-08-11T13:56:04.746 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:04.747 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:04.747 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-08-11 13:55:14.149251154 +0000 2023-08-11T13:56:04.747 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-08-11 13:41:20.224805157 +0000 2023-08-11T13:56:04.747 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-08-11 13:41:20.224805157 +0000 2023-08-11T13:56:04.747 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2023-08-11 13:41:20.198805481 +0000 2023-08-11T13:56:04.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T13:56:04.851 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-08-11T13:56:04.852 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-08-11T13:56:04.852 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000280106 s, 1.8 MB/s 2023-08-11T13:56:04.854 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T13:56:04.901 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:04.945 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-08-11 13:55:14.149251154 +0000 2023-08-11T13:56:04.946 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-08-11 13:41:21.142793697 +0000 2023-08-11T13:56:04.946 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-08-11 13:41:21.142793697 +0000 2023-08-11T13:56:04.946 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2023-08-11 13:41:21.114794046 +0000 2023-08-11T13:56:04.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T13:56:05.052 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-08-11T13:56:05.052 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-08-11T13:56:05.052 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00029722 s, 1.7 MB/s 2023-08-11T13:56:05.055 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T13:56:05.101 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2023-08-11T13:56:05.191 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T13:56:05.191 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:05.192 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-08-11T13:56:05.192 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:05.192 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:05.192 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-08-11 13:55:14.149251154 +0000 2023-08-11T13:56:05.192 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-08-11 13:41:22.041782474 +0000 2023-08-11T13:56:05.193 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-08-11 13:41:22.041782474 +0000 2023-08-11T13:56:05.193 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2023-08-11 13:41:22.013782823 +0000 2023-08-11T13:56:05.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T13:56:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-08-11T13:56:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-08-11T13:56:05.263 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000297989 s, 1.7 MB/s 2023-08-11T13:56:05.266 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T13:56:05.313 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-08-11T13:56:05.314 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T13:56:05.335 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T13:56:05.335 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2023-08-11T13:56:05.397 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T13:56:05.398 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:05.398 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 380794 Links: 1 2023-08-11T13:56:05.398 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:05.398 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:05.398 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-08-11 13:55:52.250881222 +0000 2023-08-11T13:56:05.399 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-08-11 13:38:43.530451914 +0000 2023-08-11T13:56:05.399 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-08-11 13:38:43.530451914 +0000 2023-08-11T13:56:05.399 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-08-11T13:56:05.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T13:56:05.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:05 smithi103 ceph-mon[130434]: Reconfiguring mon.c (monmap changed)... 2023-08-11T13:56:05.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:05 smithi103 ceph-mon[130434]: Reconfiguring daemon mon.c on smithi187 2023-08-11T13:56:05.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:05 smithi103 ceph-mon[130434]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi103=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:05.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:05 smithi103 ceph-mon[130434]: Saving service mgr spec with placement smithi003=x;smithi103=y;smithi187=z;count:3 2023-08-11T13:56:05.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:05 smithi103 ceph-mon[130434]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:05.460 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-08-11T13:56:05.461 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-08-11T13:56:05.461 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000265653 s, 1.9 MB/s 2023-08-11T13:56:05.462 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T13:56:05.518 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2023-08-11T13:56:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:05 smithi003 ceph-mon[97195]: Reconfiguring mon.c (monmap changed)... 2023-08-11T13:56:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:05 smithi003 ceph-mon[97195]: Reconfiguring daemon mon.c on smithi187 2023-08-11T13:56:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:05 smithi003 ceph-mon[97195]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi103=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:05 smithi003 ceph-mon[97195]: Saving service mgr spec with placement smithi003=x;smithi103=y;smithi187=z;count:3 2023-08-11T13:56:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:05 smithi003 ceph-mon[97195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:05 smithi187 ceph-mon[130973]: Reconfiguring mon.c (monmap changed)... 2023-08-11T13:56:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:05 smithi187 ceph-mon[130973]: Reconfiguring daemon mon.c on smithi187 2023-08-11T13:56:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:05 smithi187 ceph-mon[130973]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi103=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:05 smithi187 ceph-mon[130973]: Saving service mgr spec with placement smithi003=x;smithi103=y;smithi187=z;count:3 2023-08-11T13:56:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:05 smithi187 ceph-mon[130973]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:05.577 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T13:56:05.578 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:05.578 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 387479 Links: 1 2023-08-11T13:56:05.578 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:05.578 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:05.578 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-08-11 13:55:52.250881222 +0000 2023-08-11T13:56:05.579 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-08-11 13:38:44.075441715 +0000 2023-08-11T13:56:05.579 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-08-11 13:38:44.075441715 +0000 2023-08-11T13:56:05.579 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-08-11T13:56:05.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T13:56:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-08-11T13:56:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-08-11T13:56:05.646 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000251655 s, 2.0 MB/s 2023-08-11T13:56:05.647 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T13:56:05.709 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2023-08-11T13:56:05.768 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T13:56:05.768 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:05.768 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 386472 Links: 1 2023-08-11T13:56:05.768 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:05.769 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:05.769 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-08-11 13:55:52.250881222 +0000 2023-08-11T13:56:05.769 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-08-11 13:38:44.620431515 +0000 2023-08-11T13:56:05.769 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-08-11 13:38:44.620431515 +0000 2023-08-11T13:56:05.769 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-08-11T13:56:05.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T13:56:05.836 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-08-11T13:56:05.837 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-08-11T13:56:05.837 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000213603 s, 2.4 MB/s 2023-08-11T13:56:05.838 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T13:56:05.896 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2023-08-11T13:56:05.954 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T13:56:05.955 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:05.955 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 386539 Links: 1 2023-08-11T13:56:05.955 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:05.955 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:05.955 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-08-11 13:55:52.251881203 +0000 2023-08-11T13:56:05.956 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-08-11 13:38:45.149421615 +0000 2023-08-11T13:56:05.956 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-08-11 13:38:45.149421615 +0000 2023-08-11T13:56:05.956 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-08-11T13:56:05.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T13:56:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-08-11T13:56:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-08-11T13:56:06.021 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000232577 s, 2.2 MB/s 2023-08-11T13:56:06.022 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T13:56:06.081 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-08-11T13:56:06.081 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2023-08-11T13:56:06.098 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-11T13:56:06.098 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2023-08-11T13:56:06.154 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-11T13:56:06.154 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 386307 Links: 1 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-08-11 13:55:43.949414576 +0000 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-08-11 13:38:41.827768495 +0000 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-08-11 13:38:41.827768495 +0000 2023-08-11T13:56:06.155 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-08-11T13:56:06.156 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-11T13:56:06.224 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-08-11T13:56:06.224 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-08-11T13:56:06.224 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.00137225 s, 373 kB/s 2023-08-11T13:56:06.225 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-11T13:56:06.284 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2023-08-11T13:56:06.341 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 382900 Links: 1 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-08-11 13:55:43.949414576 +0000 2023-08-11T13:56:06.342 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-08-11 13:38:42.386759874 +0000 2023-08-11T13:56:06.343 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-08-11 13:38:42.386759874 +0000 2023-08-11T13:56:06.343 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-08-11T13:56:06.343 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-11T13:56:06.404 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.405 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:06.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:06.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:06.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:06 smithi187 ceph-mon[130973]: Deploying daemon mgr.z on smithi187 2023-08-11T13:56:06.406 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-08-11T13:56:06.407 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-08-11T13:56:06.407 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000194603 s, 2.6 MB/s 2023-08-11T13:56:06.408 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-11T13:56:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:06.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:06 smithi103 ceph-mon[130434]: Deploying daemon mgr.z on smithi187 2023-08-11T13:56:06.472 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2023-08-11T13:56:06.534 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-11T13:56:06.534 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 381429 Links: 1 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-08-11 13:56:06.495059288 +0000 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-08-11 13:38:42.900751947 +0000 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-08-11 13:38:42.900751947 +0000 2023-08-11T13:56:06.535 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-08-11T13:56:06.536 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-11T13:56:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:06.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:06 smithi003 ceph-mon[97195]: Deploying daemon mgr.z on smithi187 2023-08-11T13:56:06.672 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-08-11T13:56:06.673 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-08-11T13:56:06.673 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000205805 s, 2.5 MB/s 2023-08-11T13:56:06.674 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-11T13:56:06.693 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2023-08-11T13:56:06.754 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-11T13:56:06.754 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-11T13:56:06.755 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 390199 Links: 1 2023-08-11T13:56:06.755 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-11T13:56:06.755 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-08-11T13:56:06.756 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-08-11 13:56:06.623057271 +0000 2023-08-11T13:56:06.756 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-08-11 13:38:43.442743588 +0000 2023-08-11T13:56:06.756 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-08-11 13:38:43.442743588 +0000 2023-08-11T13:56:06.757 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-08-11T13:56:06.757 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-11T13:56:06.824 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-08-11T13:56:06.825 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-08-11T13:56:06.825 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000176458 s, 2.9 MB/s 2023-08-11T13:56:06.826 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-11T13:56:06.884 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2023-08-11T13:56:06.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T13:56:06.947 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:06 smithi187 systemd[1]: Starting Ceph mgr.z for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:56:07.223 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:06 smithi187 podman[133048]: 2023-08-11T13:56:07.223 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-mgr[133110]: -- 172.21.15.187:0/2415378018 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55923dbdf1e0 con 0x55923dbb4800 2023-08-11T13:56:07.223 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 bash[133048]: b2fbdd1461f72078a131ff747d22f5102e5dfe56a4b54237a87495b508a39669 2023-08-11T13:56:07.485 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-mon[130973]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:07.486 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 systemd[1]: Started Ceph mgr.z for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:56:07.486 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:07.314+0000 7f93c588d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T13:56:07.486 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:07.397+0000 7f93c588d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T13:56:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:07 smithi003 ceph-mon[97195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:07.622 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:56:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:07 smithi103 ceph-mon[130434]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:07.746 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:07.484+0000 7f93c588d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T13:56:07.747 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:07.577+0000 7f93c588d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T13:56:08.044 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:07.746+0000 7f93c588d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T13:56:08.419 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:08.107+0000 7f93c588d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T13:56:08.419 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:08.236+0000 7f93c588d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T13:56:08.420 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:08.324+0000 7f93c588d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T13:56:08.420 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:08.418+0000 7f93c588d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T13:56:08.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:08.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:08.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:08.422 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:08.422 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: Deploying daemon mgr.y on smithi103 2023-08-11T13:56:08.422 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-mon[130973]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:08.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:08.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:08.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: Deploying daemon mgr.y on smithi103 2023-08-11T13:56:08.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:08 smithi003 ceph-mon[97195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:08.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: Deploying daemon mgr.y on smithi103 2023-08-11T13:56:08.542 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:08 smithi103 ceph-mon[130434]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:09.178 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 systemd[1]: Starting Ceph mgr.y for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T13:56:09.293 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:08.836+0000 7f93c588d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T13:56:09.632 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 podman[131870]: 2023-08-11T13:56:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T13:56:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T13:56:09.645 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2023-08-11T13:56:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2023-08-11T13:56:09.645 INFO:teuthology.orchestra.run.smithi003.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0406865 s, 258 MB/s 2023-08-11T13:56:09.646 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 2023-08-11T13:56:09.646 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2023-08-11T13:56:09.794 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:09.497+0000 7f93c588d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T13:56:09.794 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:09.579+0000 7f93c588d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T13:56:09.928 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 bash[131870]: 10a2895cfcbc8bb82cd21177f88c091f646d25c19fbd5afaef4a31b730b51ba1 2023-08-11T13:56:09.928 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 ceph-mgr[131910]: -- 172.21.15.103:0/2617820596 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626a48e71e0 con 0x5626a48bc800 2023-08-11T13:56:09.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 systemd[1]: Started Ceph mgr.y for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T13:56:09.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:09.755+0000 7f7c7fc6e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-11T13:56:09.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:09.835+0000 7f7c7fc6e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-11T13:56:10.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2023-08-11T13:56:10.216 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:09.930+0000 7f7c7fc6e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-11T13:56:10.217 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.035+0000 7f7c7fc6e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-11T13:56:10.615 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.215+0000 7f7c7fc6e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-11T13:56:10.644 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:10.367+0000 7f93c588d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:56:10.644 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:10.461+0000 7f93c588d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T13:56:10.843 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:56:10.928 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.614+0000 7f7c7fc6e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-11T13:56:10.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.753+0000 7f7c7fc6e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-11T13:56:10.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.845+0000 7f7c7fc6e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-11T13:56:10.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:10.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-mon[130434]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:10.934 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:10.640+0000 7f93c588d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:56:10.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:10.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:10.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-mon[130973]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:11.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:11.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:11.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:11.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:11.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:11.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:10 smithi003 ceph-mon[97195]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:11.294 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:10.932+0000 7f93c588d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T13:56:11.294 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:11.176+0000 7f93c588d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T13:56:11.405 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:10.947+0000 7f7c7fc6e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-11T13:56:11.601 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:11.329+0000 7f93c588d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T13:56:11.601 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:11.599+0000 7f93c588d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T13:56:11.678 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:11 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:11.404+0000 7f7c7fc6e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-11T13:56:12.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 -- 172.21.15.3:0/2478164756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd4101bb0 msgr2=0x7f1dd4102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:12.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 --2- 172.21.15.3:0/2478164756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4102010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80099c0 tx=0x7f1dc8009cd0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 -- 172.21.15.3:0/2478164756 shutdown_connections 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 --2- 172.21.15.3:0/2478164756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4102550 0x7f1dd4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 --2- 172.21.15.3:0/2478164756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd41009b0 0x7f1dd4100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 --2- 172.21.15.3:0/2478164756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4102010 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 -- 172.21.15.3:0/2478164756 >> 172.21.15.3:0/2478164756 conn(0x7f1dd40fbf10 msgr2=0x7f1dd40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 -- 172.21.15.3:0/2478164756 shutdown_connections 2023-08-11T13:56:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.412+0000 7f1dd984e700 1 -- 172.21.15.3:0/2478164756 wait complete. 2023-08-11T13:56:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.413+0000 7f1dd984e700 1 Processor -- start 2023-08-11T13:56:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.413+0000 7f1dd984e700 1 -- start start 2023-08-11T13:56:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:12.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 0x7f1dd4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd40739d0 con 0x7f1dd4102550 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4073b40 con 0x7f1dd41009b0 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd984e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41a0710 con 0x7f1dd4101bb0 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd2ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47910/0 (socket says 172.21.15.3:47910) 2023-08-11T13:56:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd2ffd700 1 -- 172.21.15.3:0/1380703007 learned_addr learned my addr 172.21.15.3:0/1380703007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:56:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd37fe700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 0x7f1dd4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 -- 172.21.15.3:0/1380703007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 msgr2=0x7f1dd40746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 -- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 msgr2=0x7f1dd4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 0x7f1dd4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd27fc700 1 -- 172.21.15.3:0/1380703007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80095f0 con 0x7f1dd4101bb0 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd37fe700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 0x7f1dd4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.414+0000 7f1dd2ffd700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd27fc700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4072d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc801d070 con 0x7f1dd4101bb0 2023-08-11T13:56:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc8005510 con 0x7f1dd4101bb0 2023-08-11T13:56:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8004b30 con 0x7f1dd4101bb0 2023-08-11T13:56:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd41a0990 con 0x7f1dd4101bb0 2023-08-11T13:56:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.415+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a0d50 con 0x7f1dd4101bb0 2023-08-11T13:56:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.416+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f1dc8004c90 con 0x7f1dd4101bb0 2023-08-11T13:56:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.416+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd410c200 con 0x7f1dd4101bb0 2023-08-11T13:56:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.417+0000 7f1dd884c700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 0x7f1dbc03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.417+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dc804b960 con 0x7f1dd4101bb0 2023-08-11T13:56:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.417+0000 7f1dd2ffd700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 0x7f1dbc03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:12.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.417+0000 7f1dd2ffd700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 0x7f1dbc03a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005950 tx=0x7f1dc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.422+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1dc8016b50 con 0x7f1dd4101bb0 2023-08-11T13:56:12.428 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:12.079+0000 7f7c7fc6e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-11T13:56:12.428 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:12.161+0000 7f7c7fc6e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-11T13:56:12.628 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:12.303+0000 7f93c588d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T13:56:12.628 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:12.433+0000 7f93c588d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T13:56:12.628 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.629 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T13:56:12.630 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:12.630 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:12.630 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.630 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: Reconfiguring daemon mgr.x on smithi003 2023-08-11T13:56:12.630 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:12.631 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:12.651+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd4060d60 con 0x7f1dbc0383e0 2023-08-11T13:56:12.756 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: Reconfiguring daemon mgr.x on smithi003 2023-08-11T13:56:12.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:12.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:12.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: Reconfiguring mgr.x (unknown last config time)... 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: Reconfiguring daemon mgr.x on smithi003 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:12.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:12.954 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:12.626+0000 7f93c588d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T13:56:12.955 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:12.737+0000 7f93c588d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T13:56:13.193 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:12.931+0000 7f7c7fc6e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:56:13.193 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:13.021+0000 7f7c7fc6e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-11T13:56:13.293 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:12.952+0000 7f93c588d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T13:56:13.472 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:13.191+0000 7f7c7fc6e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-11T13:56:13.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:13.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:13.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:13.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:13.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:13.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:13.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:13.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:13.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.847 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:13.471+0000 7f7c7fc6e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-11T13:56:13.847 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:13.697+0000 7f7c7fc6e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.848 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:13.849 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:13.849 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:13.849 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:13.849 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:14.111 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:13.846+0000 7f7c7fc6e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-11T13:56:14.294 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 13:56:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z[133086]: 2023-08-11T13:56:13.811+0000 7f93c588d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T13:56:14.428 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:14.110+0000 7f7c7fc6e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-11T13:56:14.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:14.519+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mgrmap(e 14) v1 ==== 89902+0+0 (secure 0 0 0) 0x7f1dc800fb80 con 0x7f1dd4101bb0 2023-08-11T13:56:14.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: Standby manager daemon z started 2023-08-11T13:56:14.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T13:56:14.735 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:14.735 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T13:56:14.735 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:14.735 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:14 smithi003 ceph-mon[97195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:14.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: Standby manager daemon z started 2023-08-11T13:56:14.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T13:56:14.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:14.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T13:56:14.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:14.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:14 smithi187 ceph-mon[130973]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:14.813 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: Standby manager daemon z started 2023-08-11T13:56:14.813 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-08-11T13:56:14.813 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:14.813 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-08-11T13:56:14.814 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.187:0/3496679693' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:14.814 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-mon[130434]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:15.133 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:14.812+0000 7f7c7fc6e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-11T13:56:15.133 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:14.941+0000 7f7c7fc6e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-11T13:56:15.428 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:15.132+0000 7f7c7fc6e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-11T13:56:15.428 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:15.245+0000 7f7c7fc6e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-11T13:56:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:15 smithi003 ceph-mon[97195]: mgrmap e14: x(active, since 72s), standbys: z 2023-08-11T13:56:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T13:56:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:15 smithi187 ceph-mon[130973]: mgrmap e14: x(active, since 72s), standbys: z 2023-08-11T13:56:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T13:56:15.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:15 smithi103 ceph-mon[130434]: mgrmap e14: x(active, since 72s), standbys: z 2023-08-11T13:56:15.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-08-11T13:56:15.929 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:15.463+0000 7f7c7fc6e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-11T13:56:16.678 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y[131886]: 2023-08-11T13:56:16.322+0000 7f7c7fc6e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-11T13:56:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3996445957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]': finished 2023-08-11T13:56:16.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T13:56:16.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:16.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:16.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: Standby manager daemon y started 2023-08-11T13:56:16.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T13:56:16.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:16.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T13:56:16.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:16.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:16 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4095440033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:16.735+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mon.1 v2:172.21.15.187:3300/0 8 ==== mgrmap(e 15) v1 ==== 134614+0+0 (secure 0 0 0) 0x7f1dc8029bc0 con 0x7f1dd4101bb0 2023-08-11T13:56:16.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3996445957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]': finished 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: Standby manager daemon y started 2023-08-11T13:56:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T13:56:16.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:16.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T13:56:16.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:16.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:16 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4095440033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3996445957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]: dispatch 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f155a1aa-25d5-4785-9e30-63ab3dc892ba"}]': finished 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: osdmap e5: 1 total, 0 up, 1 in 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:16.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: Standby manager daemon y started 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='mgr.? 172.21.15.103:0/3912951530' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-08-11T13:56:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:16 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4095440033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:18.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:17 smithi003 ceph-mon[97195]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-08-11T13:56:18.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T13:56:18.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:17 smithi187 ceph-mon[130973]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-08-11T13:56:18.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T13:56:18.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:17 smithi103 ceph-mon[130434]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-08-11T13:56:18.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-08-11T13:56:19.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:18 smithi003 ceph-mon[97195]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:19.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:18 smithi187 ceph-mon[130973]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:19.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:18 smithi103 ceph-mon[130434]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:21.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:21 smithi103 ceph-mon[130434]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:21.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:21 smithi003 ceph-mon[97195]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:21.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:21 smithi187 ceph-mon[130973]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:23.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:23 smithi103 ceph-mon[130434]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:23.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T13:56:23.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:23 smithi003 ceph-mon[97195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T13:56:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:23.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:23 smithi187 ceph-mon[130973]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:23.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-08-11T13:56:23.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:24.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:24 smithi003 ceph-mon[97195]: Deploying daemon osd.0 on smithi003 2023-08-11T13:56:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:24 smithi103 ceph-mon[130434]: Deploying daemon osd.0 on smithi003 2023-08-11T13:56:24.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:24 smithi187 ceph-mon[130973]: Deploying daemon osd.0 on smithi003 2023-08-11T13:56:25.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:25 smithi003 ceph-mon[97195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:25.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:25 smithi103 ceph-mon[130434]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:25.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:25 smithi187 ceph-mon[130973]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:27.419 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:27 smithi003 ceph-mon[97195]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:27.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:27 smithi103 ceph-mon[130434]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:27.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:27 smithi187 ceph-mon[130973]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:29.420 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:29 smithi003 ceph-mon[97195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:29.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:29 smithi103 ceph-mon[130434]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:29.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:29 smithi187 ceph-mon[130973]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:30.838 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:30.838 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:30.838 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:30.839 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:30 smithi003 ceph-mon[97195]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:30.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:30.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:30.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:30.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:30 smithi103 ceph-mon[130434]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:31.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:31.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:31.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:31.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:30 smithi187 ceph-mon[130973]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:33.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.026+0000 7f1dd884c700 1 -- 172.21.15.3:0/1380703007 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1dd4060d60 con 0x7f1dbc0383e0 2023-08-11T13:56:33.029 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2023-08-11T13:56:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 msgr2=0x7f1dbc03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 0x7f1dbc03a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005950 tx=0x7f1dc4009260 comp rx=0 tx=0).stop 2023-08-11T13:56:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 msgr2=0x7f1dd4072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4072d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc800f690 comp rx=0 tx=0).stop 2023-08-11T13:56:33.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 shutdown_connections 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd4102550 0x7f1dd4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1dbc0383e0 0x7f1dbc03a8a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dd41009b0 0x7f1dd40746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 --2- 172.21.15.3:0/1380703007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dd4101bb0 0x7f1dd4072d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 >> 172.21.15.3:0/1380703007 conn(0x7f1dd40fbf10 msgr2=0x7f1dd40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.030+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 shutdown_connections 2023-08-11T13:56:33.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:56:33.031+0000 7f1dd984e700 1 -- 172.21.15.3:0/1380703007 wait complete. 2023-08-11T13:56:33.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:33.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:33.201 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.202 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:32 smithi003 ceph-mon[97195]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:32 smithi187 ceph-mon[130973]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:33.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:33.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:33.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:33.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:32 smithi103 ceph-mon[130434]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:33.471 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.0.service 2023-08-11T13:56:33.484 INFO:tasks.cephadm:Deploying osd.1 on smithi103 with /dev/vg_nvme/lv_4... 2023-08-11T13:56:33.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T13:56:33.570 INFO:journalctl@ceph.osd.0.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T13:56:33.791 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:56:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T13:56:33.574+0000 7fdfa1978640 -1 osd.0 0 log_to_monitors true 2023-08-11T13:56:34.048 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:56:34.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:34.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:34 smithi103 ceph-mon[130434]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:34.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:34 smithi103 ceph-mon[130434]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:34.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:34 smithi003 ceph-mon[97195]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:34 smithi003 ceph-mon[97195]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:34.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:34.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:34.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:34 smithi187 ceph-mon[130973]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:34.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:34 smithi187 ceph-mon[130973]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-08-11T13:56:35.030 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T13:56:35.025+0000 7fdf96ff0700 -1 osd.0 0 waiting for initial osdmap 2023-08-11T13:56:35.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T13:56:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T13:56:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-mon[97195]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:35.161 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T13:56:35.161 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T13:56:35.161 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:35.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:35 smithi103 ceph-mon[130434]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:35.291 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:56:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T13:56:35.028+0000 7fdf8fddf700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:56:35.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-08-11T13:56:35.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: osdmap e6: 1 total, 0 up, 1 in 2023-08-11T13:56:35.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:35.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: from='osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:56:35.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:35 smithi187 ceph-mon[130973]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:35.642 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T13:56:35.642 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T13:56:35.642 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2023-08-11T13:56:35.642 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2023-08-11T13:56:35.642 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01114 s, 941 MB/s 2023-08-11T13:56:35.643 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2023-08-11T13:56:36.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2023-08-11T13:56:36.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:36 smithi003 ceph-mon[97195]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:56:36.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:36 smithi003 ceph-mon[97195]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T13:56:36.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:36 smithi187 ceph-mon[130973]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:56:36.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:36 smithi187 ceph-mon[130973]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T13:56:36.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:36 smithi103 ceph-mon[130434]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:56:36.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:36 smithi103 ceph-mon[130434]: osdmap e7: 1 total, 0 up, 1 in 2023-08-11T13:56:36.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:36.648 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:56:37.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:56:37.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:56:37.164 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] boot 2023-08-11T13:56:37.164 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T13:56:37.164 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:37.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:37.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.165 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:37.166 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.166 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:37.166 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:37.166 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:56:37.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:56:37.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] boot 2023-08-11T13:56:37.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:37.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:37.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: osd.0 [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] boot 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: osdmap e8: 1 total, 1 up, 1 in 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:37.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:37.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:37.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:37.819 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 -- 172.21.15.103:0/2268903727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 msgr2=0x7fe0840ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:37.819 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 --2- 172.21.15.103:0/2268903727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe0840ffcd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0740099c0 tx=0x7fe074009cd0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.819 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 -- 172.21.15.103:0/2268903727 shutdown_connections 2023-08-11T13:56:37.820 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 --2- 172.21.15.103:0/2268903727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe084107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.820 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 --2- 172.21.15.103:0/2268903727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe0840ffcd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.820 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 --2- 172.21.15.103:0/2268903727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.821 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 -- 172.21.15.103:0/2268903727 >> 172.21.15.103:0/2268903727 conn(0x7fe0840fae30 msgr2=0x7fe0840fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:37.821 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 -- 172.21.15.103:0/2268903727 shutdown_connections 2023-08-11T13:56:37.821 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.815+0000 7fe08a69d700 1 -- 172.21.15.103:0/2268903727 wait complete. 2023-08-11T13:56:37.821 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 Processor -- start 2023-08-11T13:56:37.822 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 -- start start 2023-08-11T13:56:37.822 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:37.822 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:37.822 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe0841023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:37.823 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084102b50 con 0x7fe084100ab0 2023-08-11T13:56:37.823 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084102cc0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.823 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe08a69d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841a0840 con 0x7fe084101450 2023-08-11T13:56:37.823 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe083fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:37.824 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe083fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58642/0 (socket says 172.21.15.103:58642) 2023-08-11T13:56:37.824 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.816+0000 7fe083fff700 1 -- 172.21.15.103:0/291445270 learned_addr learned my addr 172.21.15.103:0/291445270 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-08-11T13:56:37.824 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe088c3a700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe0841023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:37.824 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 -- 172.21.15.103:0/291445270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 msgr2=0x7fe0841023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe0841023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe0837fe700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 -- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 msgr2=0x7fe084101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:37.825 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:37.826 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 -- 172.21.15.103:0/291445270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740095f0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.826 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe088c3a700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe0841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T13:56:37.826 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe0837fe700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T13:56:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe083fff700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0740036f0 tx=0x7fe0740038c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401d070 con 0x7fe0840ff8b0 2023-08-11T13:56:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe074022470 con 0x7fe0840ff8b0 2023-08-11T13:56:37.827 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0841a0ac0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400f6c0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.817+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0841a0f50 con 0x7fe0840ff8b0 2023-08-11T13:56:37.828 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.818+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08410b200 con 0x7fe0840ff8b0 2023-08-11T13:56:37.829 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.819+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) v1 ==== 134614+0+0 (secure 0 0 0) 0x7fe0740225e0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.830 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.820+0000 7fe0817fa700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 0x7fe06c0a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:37.830 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.820+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fe0740cb5c0 con 0x7fe0840ff8b0 2023-08-11T13:56:37.830 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.820+0000 7fe0837fe700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 0x7fe06c0a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:37.831 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.821+0000 7fe0837fe700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 0x7fe06c0a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe078007900 tx=0x7fe078008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:37.831 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:37.823+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe074096db0 con 0x7fe0840ff8b0 2023-08-11T13:56:38.019 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:38.017+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0840fbe90 con 0x7fe06c09fec0 2023-08-11T13:56:38.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi003 2023-08-11T13:56:38.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 17272M 2023-08-11T13:56:38.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T13:56:38.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:38.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:38.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi003 2023-08-11T13:56:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 17272M 2023-08-11T13:56:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T13:56:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:38.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi003 2023-08-11T13:56:38.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 17272M 2023-08-11T13:56:38.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: osdmap e9: 1 total, 1 up, 1 in 2023-08-11T13:56:38.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:38.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:38.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:39 smithi103 ceph-mon[130434]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:39 smithi103 ceph-mon[130434]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:39 smithi003 ceph-mon[97195]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:39 smithi003 ceph-mon[97195]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:39.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:39 smithi187 ceph-mon[130973]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:39.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:39 smithi187 ceph-mon[130973]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/4052343830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]': finished 2023-08-11T13:56:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T13:56:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/480821594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:41.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/4052343830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]': finished 2023-08-11T13:56:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T13:56:41.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:41.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/480821594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:41.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/4052343830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]: dispatch 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cb5b3f-ad6a-47d8-8241-18b6a6889961"}]': finished 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: osdmap e10: 2 total, 1 up, 2 in 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/480821594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:56:43.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:43 smithi103 ceph-mon[130434]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:43 smithi003 ceph-mon[97195]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:43.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:43 smithi187 ceph-mon[130973]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:45.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:45 smithi103 ceph-mon[130434]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:45 smithi003 ceph-mon[97195]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:45 smithi187 ceph-mon[130973]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:47 smithi103 ceph-mon[130434]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:47 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T13:56:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:47 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:47 smithi003 ceph-mon[97195]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:47 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T13:56:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:47 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:47.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:47 smithi187 ceph-mon[130973]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:47.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:47 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-08-11T13:56:47.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:47 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:48.426 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:48 smithi103 ceph-mon[130434]: Deploying daemon osd.1 on smithi103 2023-08-11T13:56:48.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:48 smithi003 ceph-mon[97195]: Deploying daemon osd.1 on smithi103 2023-08-11T13:56:48.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:48 smithi187 ceph-mon[130973]: Deploying daemon osd.1 on smithi103 2023-08-11T13:56:49.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:49 smithi103 ceph-mon[130434]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:49 smithi003 ceph-mon[97195]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:49.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:49 smithi187 ceph-mon[130973]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:51.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:51 smithi103 ceph-mon[130434]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:51.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:51 smithi003 ceph-mon[97195]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:51.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:51 smithi187 ceph-mon[130973]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:52.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:52.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:52 smithi103 ceph-mon[130434]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:52 smithi003 ceph-mon[97195]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:52 smithi187 ceph-mon[130973]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:54.287 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 1 on host 'smithi103' 2023-08-11T13:56:54.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.285+0000 7fe0817fa700 1 -- 172.21.15.103:0/291445270 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe0840fbe90 con 0x7fe06c09fec0 2023-08-11T13:56:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 msgr2=0x7fe06c0a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 0x7fe06c0a2380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe078007900 tx=0x7fe078008040 comp rx=0 tx=0).stop 2023-08-11T13:56:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 msgr2=0x7fe084103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0740036f0 tx=0x7fe0740038c0 comp rx=0 tx=0).stop 2023-08-11T13:56:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 shutdown_connections 2023-08-11T13:56:54.291 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe084101450 0x7fe0841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe06c09fec0 0x7fe06c0a2380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0840ff8b0 0x7fe084103840 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:54.292 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 --2- 172.21.15.103:0/291445270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe084100ab0 0x7fe084101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 >> 172.21.15.103:0/291445270 conn(0x7fe0840fae30 msgr2=0x7fe0840fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 shutdown_connections 2023-08-11T13:56:54.293 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:56:54.288+0000 7fe08a69d700 1 -- 172.21.15.103:0/291445270 wait complete. 2023-08-11T13:56:54.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:54.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:54.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:54.737 DEBUG:teuthology.orchestra.run.smithi103:osd.1> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.1.service 2023-08-11T13:56:54.739 INFO:tasks.cephadm:Deploying osd.2 on smithi187 with /dev/vg_nvme/lv_4... 2023-08-11T13:56:54.739 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-08-11T13:56:54.772 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.773 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.773 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:54.773 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:54.773 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.773 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:54.774 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.774 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.774 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:54.774 INFO:journalctl@ceph.osd.1.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T13:56:54.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:54.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:54.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:54.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:54.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:56:55.302 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:56:55.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:55 smithi103 ceph-mon[130434]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:55.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:55 smithi103 ceph-mon[130434]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:55.469 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:56:55 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T13:56:55.308+0000 7faff41bd640 -1 osd.1 0 log_to_monitors true 2023-08-11T13:56:55.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:55 smithi187 ceph-mon[130973]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:55.707 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:55 smithi187 ceph-mon[130973]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:55 smithi003 ceph-mon[97195]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:55 smithi003 ceph-mon[97195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-08-11T13:56:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T13:56:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T13:56:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:56 smithi187 ceph-mon[130973]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:56.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T13:56:56.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T13:56:56.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:56.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.670 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-mon[130434]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:56.670 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T13:56:56.381+0000 7fafe9835700 -1 osd.1 0 waiting for initial osdmap 2023-08-11T13:56:56.670 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:56:56 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T13:56:56.387+0000 7fafe3e27700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:56:56.786 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-08-11T13:56:56.786 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-08-11T13:56:56.786 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-08-11T13:56:56.786 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-08-11T13:56:56.786 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125056 s, 838 MB/s 2023-08-11T13:56:56.787 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-08-11T13:56:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-08-11T13:56:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: osdmap e11: 2 total, 1 up, 2 in 2023-08-11T13:56:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:56.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: from='osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:56:56.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:56 smithi003 ceph-mon[97195]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:57.157 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi187:vg_nvme/lv_4 2023-08-11T13:56:57.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:56:57.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T13:56:57.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi103 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:57.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 17168M 2023-08-11T13:56:57.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:57.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:57.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:56:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T13:56:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi103 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 17168M 2023-08-11T13:56:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:57.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:57.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.746 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:56:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:56:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: osdmap e12: 2 total, 1 up, 2 in 2023-08-11T13:56:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi103 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 17168M 2023-08-11T13:56:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:57.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:57.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:56:57.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:56:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:56:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:56:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] boot 2023-08-11T13:56:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T13:56:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:58 smithi103 ceph-mon[130434]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:56:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:56:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] boot 2023-08-11T13:56:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T13:56:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:58 smithi003 ceph-mon[97195]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: osd.1 [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] boot 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: osdmap e13: 2 total, 2 up, 2 in 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-08-11T13:56:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:58 smithi187 ceph-mon[130973]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-08-11T13:56:59.027 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 -- 172.21.15.187:0/508183015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 msgr2=0x7f70f4103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:59.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 --2- 172.21.15.187:0/508183015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4103890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009990 tx=0x7f70e4009ca0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 -- 172.21.15.187:0/508183015 shutdown_connections 2023-08-11T13:56:59.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 --2- 172.21.15.187:0/508183015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4106700 0x7f70f4108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 --2- 172.21.15.187:0/508183015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4103890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 --2- 172.21.15.187:0/508183015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4103dd0 0x7f70f41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 -- 172.21.15.187:0/508183015 >> 172.21.15.187:0/508183015 conn(0x7f70f40fae30 msgr2=0x7f70f40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 -- 172.21.15.187:0/508183015 shutdown_connections 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 -- 172.21.15.187:0/508183015 wait complete. 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.025+0000 7f70fa823700 1 Processor -- start 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 -- start start 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103dd0 0x7f70f4196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:59.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4106700 0x7f70f419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419e2d0 con 0x7f70f4103dd0 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419e410 con 0x7f70f4106700 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70fa823700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f419e5a0 con 0x7f70f41014a0 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:54922/0 (socket says 172.21.15.187:54922) 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 -- 172.21.15.187:0/2683251764 learned_addr learned my addr 172.21.15.187:0/2683251764 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:56:59.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 -- 172.21.15.187:0/2683251764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4106700 msgr2=0x7f70f419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4106700 0x7f70f419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 -- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103dd0 msgr2=0x7f70f4196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103dd0 0x7f70f4196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 -- 172.21.15.187:0/2683251764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40095f0 con 0x7f70f41014a0 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.026+0000 7f70f3fff700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005910 tx=0x7f70e400f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.027+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e401d070 con 0x7f70f41014a0 2023-08-11T13:56:59.031 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.027+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e400fd10 con 0x7f70f41014a0 2023-08-11T13:56:59.032 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.027+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f419e820 con 0x7f70f41014a0 2023-08-11T13:56:59.032 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.027+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e40176e0 con 0x7f70f41014a0 2023-08-11T13:56:59.032 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.027+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f419ed90 con 0x7f70f41014a0 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.028+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f40fc9b0 con 0x7f70f41014a0 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.028+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 15) v1 ==== 134614+0+0 (secure 0 0 0) 0x7f70e4022470 con 0x7f70f41014a0 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.029+0000 7f70f17fa700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 0x7f70dc0a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.029+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f70e40cad60 con 0x7f70f41014a0 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.029+0000 7f70f37fe700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 0x7f70dc0a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:56:59.033 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.030+0000 7f70f37fe700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 0x7f70dc0a2400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70e80097b0 tx=0x7f70e8006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:56:59.034 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.034+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f70e405cc30 con 0x7f70f41014a0 2023-08-11T13:56:59.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:56:59.230+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f70f402ced0 con 0x7f70dc09ff40 2023-08-11T13:56:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:59 smithi103 ceph-mon[130434]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T13:56:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:59 smithi103 ceph-mon[130434]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:56:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:59 smithi003 ceph-mon[97195]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T13:56:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:59 smithi003 ceph-mon[97195]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:59.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:59.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:56:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:56:59.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:59 smithi187 ceph-mon[130973]: osdmap e14: 2 total, 2 up, 2 in 2023-08-11T13:56:59.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:59 smithi187 ceph-mon[130973]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:56:59.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:56:59.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:56:59.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:56:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:00 smithi103 ceph-mon[130434]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:00 smithi003 ceph-mon[97195]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:00 smithi187 ceph-mon[130973]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:01 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/2694660540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:01 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:01 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]': finished 2023-08-11T13:57:01.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:01 smithi103 ceph-mon[130434]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T13:57:01.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:02.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:01 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/2694660540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:02.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:01 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:01 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]': finished 2023-08-11T13:57:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:01 smithi003 ceph-mon[97195]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T13:57:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:02.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:01 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/2694660540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:02.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:01 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]: dispatch 2023-08-11T13:57:02.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:01 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d3838c3-0422-4ead-946c-f9dd572bff9d"}]': finished 2023-08-11T13:57:02.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:01 smithi187 ceph-mon[130973]: osdmap e15: 3 total, 2 up, 3 in 2023-08-11T13:57:02.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:03.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:03 smithi103 ceph-mon[130434]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:03.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:03 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/710397068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:03 smithi003 ceph-mon[97195]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:03 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/710397068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:03.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:03 smithi187 ceph-mon[130973]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:03.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:03 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/710397068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:05.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:05 smithi103 ceph-mon[130434]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:05 smithi003 ceph-mon[97195]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:05.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:05 smithi187 ceph-mon[130973]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:07.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:07 smithi103 ceph-mon[130434]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:07 smithi003 ceph-mon[97195]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:07 smithi187 ceph-mon[130973]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:08.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T13:57:08.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:08.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T13:57:08.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:08.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-08-11T13:57:08.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:09.526 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:09 smithi187 ceph-mon[130973]: Deploying daemon osd.2 on smithi187 2023-08-11T13:57:09.526 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:09 smithi187 ceph-mon[130973]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:09 smithi003 ceph-mon[97195]: Deploying daemon osd.2 on smithi187 2023-08-11T13:57:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:09 smithi003 ceph-mon[97195]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:09 smithi103 ceph-mon[130434]: Deploying daemon osd.2 on smithi187 2023-08-11T13:57:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:09 smithi103 ceph-mon[130434]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:11.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:11 smithi187 ceph-mon[130973]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:11.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:11 smithi003 ceph-mon[97195]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:11 smithi103 ceph-mon[130434]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:13.502 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:13 smithi187 ceph-mon[130973]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:13.503 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.503 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.503 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:13 smithi003 ceph-mon[97195]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:13 smithi103 ceph-mon[130434]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:13.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:14.848 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 2 on host 'smithi187' 2023-08-11T13:57:14.848 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.846+0000 7f70f17fa700 1 -- 172.21.15.187:0/2683251764 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f70f402ced0 con 0x7f70dc09ff40 2023-08-11T13:57:14.850 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 msgr2=0x7f70dc0a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:14.850 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 0x7f70dc0a2400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f70e80097b0 tx=0x7f70e8006d20 comp rx=0 tx=0).stop 2023-08-11T13:57:14.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 msgr2=0x7f70f4196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:14.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005910 tx=0x7f70e400f830 comp rx=0 tx=0).stop 2023-08-11T13:57:14.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 shutdown_connections 2023-08-11T13:57:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70dc09ff40 0x7f70dc0a2400 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:14.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4106700 0x7f70f419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:14.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f41014a0 0x7f70f4196740 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:14.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 --2- 172.21.15.187:0/2683251764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70f4103dd0 0x7f70f4196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:14.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 >> 172.21.15.187:0/2683251764 conn(0x7f70f40fae30 msgr2=0x7f70f4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:57:14.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 shutdown_connections 2023-08-11T13:57:14.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:57:14.848+0000 7f70fa823700 1 -- 172.21.15.187:0/2683251764 wait complete. 2023-08-11T13:57:15.230 DEBUG:teuthology.orchestra.run.smithi187:osd.2> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.2.service 2023-08-11T13:57:15.233 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2023-08-11T13:57:15.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T13:57:15.265 INFO:journalctl@ceph.osd.2.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T13:57:15.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:15.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:15.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:15.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:57:16.794 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T13:57:16.375+0000 7fb6bc910640 -1 osd.2 0 log_to_monitors true 2023-08-11T13:57:17.423 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:17 smithi187 ceph-mon[130973]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:17.423 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:17 smithi187 ceph-mon[130973]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:17.423 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:17 smithi187 ceph-mon[130973]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:17 smithi003 ceph-mon[97195]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:17 smithi003 ceph-mon[97195]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:17 smithi003 ceph-mon[97195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:17 smithi103 ceph-mon[130434]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:17 smithi103 ceph-mon[130434]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:17 smithi103 ceph-mon[130434]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-08-11T13:57:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T13:57:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T13:57:18.019 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2023-08-11T13:57:18.019 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2023-08-11T13:57:18.020 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0404487 s, 259 MB/s 2023-08-11T13:57:18.020 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2023-08-11T13:57:18.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T13:57:18.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T13:57:18.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:18.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:18.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:18.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2023-08-11T13:57:18.546 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T13:57:18.218+0000 7fb6b1f88700 -1 osd.2 0 waiting for initial osdmap 2023-08-11T13:57:18.547 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T13:57:18.222+0000 7fb6ac57a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:57:18.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T13:57:18.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T13:57:18.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:18.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:18.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:18.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:18.551 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-08-11T13:57:18.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: osdmap e16: 3 total, 2 up, 3 in 2023-08-11T13:57:18.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:18.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:18.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:18.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:19.178 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:57:19.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi187 2023-08-11T13:57:19.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 17168M 2023-08-11T13:57:19.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:19.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:57:19.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T13:57:19.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:19.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi187 2023-08-11T13:57:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 17168M 2023-08-11T13:57:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:57:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T13:57:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi187 2023-08-11T13:57:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 17168M 2023-08-11T13:57:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:57:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: osdmap e17: 3 total, 2 up, 3 in 2023-08-11T13:57:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/1333582868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0108880 msgr2=0x7f3fb010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:20.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/1333582868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0108880 0x7f3fb010ad10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800b0a0 tx=0x7f3fa800b3b0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/1333582868 shutdown_connections 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/1333582868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0108880 0x7f3fb010ad10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/1333582868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0074950 0x7f3fb0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/1333582868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb0073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/1333582868 >> 172.21.15.3:0/1333582868 conn(0x7f3fb00fbf10 msgr2=0x7f3fb00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/1333582868 shutdown_connections 2023-08-11T13:57:20.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.748+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/1333582868 wait complete. 2023-08-11T13:57:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 Processor -- start 2023-08-11T13:57:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 -- start start 2023-08-11T13:57:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 0x7f3fb0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0108880 0x7f3fb019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01a0550 con 0x7f3fb0073380 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01a0690 con 0x7f3fb0074950 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3fb5fd7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01a0820 con 0x7f3fb0108880 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3faf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3faf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52874/0 (socket says 172.21.15.3:52874) 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.749+0000 7f3faf7fe700 1 -- 172.21.15.3:0/2298756606 learned_addr learned my addr 172.21.15.3:0/2298756606 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:57:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 0x7f3fb0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faffff700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0108880 0x7f3fb019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 -- 172.21.15.3:0/2298756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0108880 msgr2=0x7f3fb019ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0108880 0x7f3fb019ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 -- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 msgr2=0x7f3fb01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 -- 172.21.15.3:0/2298756606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8009d00 con 0x7f3fb0074950 2023-08-11T13:57:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faf7fe700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3faeffd700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 0x7f3fb0198f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9c0 tx=0x7f3fa400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.750+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004220 con 0x7f3fb0074950 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.751+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa4004380 con 0x7f3fb0074950 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.751+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb01a0b00 con 0x7f3fb0074950 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.751+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003c50 con 0x7f3fb0074950 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.751+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb01a1000 con 0x7f3fb0074950 2023-08-11T13:57:20.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.752+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb0106f30 con 0x7f3fb0074950 2023-08-11T13:57:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.754+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) v1 ==== 134614+0+0 (secure 0 0 0) 0x7f3fa400c7c0 con 0x7f3fb0074950 2023-08-11T13:57:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.755+0000 7f3facff9700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 0x7f3f980a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.755+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f3fa4014070 con 0x7f3fb0074950 2023-08-11T13:57:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.755+0000 7f3faf7fe700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 0x7f3f980a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:20.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.756+0000 7f3faf7fe700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 0x7f3f980a2380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005950 tx=0x7f3fa000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:57:20.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.757+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3fa4095310 con 0x7f3fb0074950 2023-08-11T13:57:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:20 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:57:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:20 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:57:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:20 smithi003 ceph-mon[97195]: osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] boot 2023-08-11T13:57:20.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:20 smithi003 ceph-mon[97195]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T13:57:20.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:20 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:20.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:20.976+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fb002ced0 con 0x7f3f9809fec0 2023-08-11T13:57:21.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:20 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:57:21.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:20 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:57:21.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:20 smithi187 ceph-mon[130973]: osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] boot 2023-08-11T13:57:21.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:20 smithi187 ceph-mon[130973]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T13:57:21.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:20 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:21.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:20 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:57:21.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:20 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:57:21.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:20 smithi103 ceph-mon[130434]: osd.2 [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] boot 2023-08-11T13:57:21.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:20 smithi103 ceph-mon[130434]: osdmap e18: 3 total, 3 up, 3 in 2023-08-11T13:57:21.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:20 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-08-11T13:57:22.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:22.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:22.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T13:57:22.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:22.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:22.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:22.032 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T13:57:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:22.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:22.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:22.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:22.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-08-11T13:57:22.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:22.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: osdmap e19: 3 total, 3 up, 3 in 2023-08-11T13:57:22.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T13:57:22.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:22.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:22.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:22.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:23.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:23.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:22 smithi003 ceph-mon[97195]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T13:57:23.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:22 smithi003 ceph-mon[97195]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:22 smithi187 ceph-mon[130973]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T13:57:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:22 smithi187 ceph-mon[130973]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:23.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-08-11T13:57:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:22 smithi103 ceph-mon[130434]: osdmap e20: 3 total, 3 up, 3 in 2023-08-11T13:57:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:22 smithi103 ceph-mon[130434]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:23.291 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113326]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:23.291 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:23.291 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113326]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:23.292 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113326]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:24.041 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113347]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:24.041 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:24.042 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113347]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:24.042 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 13:57:23 smithi003 sudo[113347]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:24.224 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:23 smithi103 sudo[137696]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:24.225 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:23 smithi103 sudo[137696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:24.225 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:23 smithi103 sudo[137696]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:24.225 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:23 smithi103 sudo[137696]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:24.225 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T13:57:24.225 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:24.226 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/273090046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.226 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.226 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]': finished 2023-08-11T13:57:24.226 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: osdmap e22: 4 total, 3 up, 4 in 2023-08-11T13:57:24.226 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:24.494 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T13:57:24.494 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:24.494 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/273090046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.494 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.494 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]': finished 2023-08-11T13:57:24.495 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: osdmap e22: 4 total, 3 up, 4 in 2023-08-11T13:57:24.495 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:24.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: osdmap e21: 3 total, 3 up, 3 in 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/273090046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]: dispatch 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d0f37b-9009-4887-b9fb-daa5c9d047e4"}]': finished 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: osdmap e22: 4 total, 3 up, 4 in 2023-08-11T13:57:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:24.678 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:24 smithi103 sudo[137699]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:24.678 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:24 smithi103 sudo[137699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:24.678 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:24 smithi103 sudo[137699]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:24.679 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 13:57:24 smithi103 sudo[137699]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:24.745 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138871]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:24.745 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:24.746 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138871]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:24.746 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138871]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:25.044 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138874]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:25.044 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:25.044 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138874]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:25.044 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 13:57:24 smithi187 sudo[138874]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113458]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:25.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113458]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:25.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113458]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:25.884+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f3fa4064ea0 con 0x7f3fb0074950 2023-08-11T13:57:26.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:25 smithi103 ceph-mon[130434]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:26.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:25 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:26.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113835]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113835]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113835]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 sudo[113835]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 ceph-mon[97195]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:25 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:26.368 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138877]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:26.368 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:26.368 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138877]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:26.369 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138877]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:26.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138880]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:26.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138880]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 sudo[138880]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:26.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:26.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137702]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-08-11T13:57:26.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:26.861 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137702]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:26.861 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137702]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:27.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137705]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-08-11T13:57:27.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-08-11T13:57:27.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137705]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-08-11T13:57:27.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 sudo[137705]: pam_unix(sudo:session): session closed for user root 2023-08-11T13:57:27.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2583360471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T13:57:27.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:26 smithi103 ceph-mon[130434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2583360471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:26 smithi003 ceph-mon[97195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2583360471' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-08-11T13:57:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:27.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:27.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-08-11T13:57:27.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-08-11T13:57:27.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-08-11T13:57:27.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-08-11T13:57:27.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:26 smithi187 ceph-mon[130973]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-08-11T13:57:29.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:29 smithi103 ceph-mon[130434]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:29 smithi003 ceph-mon[97195]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:29.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:29 smithi187 ceph-mon[130973]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:31.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:31 smithi103 ceph-mon[130434]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 303 KiB/s wr, 20 op/s 2023-08-11T13:57:31.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:31 smithi003 ceph-mon[97195]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 303 KiB/s wr, 20 op/s 2023-08-11T13:57:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:31 smithi187 ceph-mon[130973]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 303 KiB/s wr, 20 op/s 2023-08-11T13:57:32.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T13:57:32.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T13:57:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-08-11T13:57:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:33 smithi003 ceph-mon[97195]: Deploying daemon osd.3 on smithi003 2023-08-11T13:57:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:33 smithi003 ceph-mon[97195]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-08-11T13:57:33.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:33 smithi187 ceph-mon[130973]: Deploying daemon osd.3 on smithi003 2023-08-11T13:57:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:33 smithi187 ceph-mon[130973]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-08-11T13:57:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:33 smithi103 ceph-mon[130434]: Deploying daemon osd.3 on smithi003 2023-08-11T13:57:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:33 smithi103 ceph-mon[130434]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-08-11T13:57:35.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:35 smithi003 ceph-mon[97195]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-08-11T13:57:35.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:35 smithi187 ceph-mon[130973]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-08-11T13:57:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:35 smithi103 ceph-mon[130434]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-08-11T13:57:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:37 smithi003 ceph-mon[97195]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:37.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:37 smithi187 ceph-mon[130973]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:37 smithi103 ceph-mon[130434]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:39.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:39.287 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:38 smithi003 ceph-mon[97195]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:39.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:38 smithi187 ceph-mon[130973]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:39.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:39.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:38 smithi103 ceph-mon[130434]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:41.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:41.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:41.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.485+0000 7f3facff9700 1 -- 172.21.15.3:0/2298756606 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3fb002ced0 con 0x7f3f9809fec0 2023-08-11T13:57:41.487 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2023-08-11T13:57:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 msgr2=0x7f3f980a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:41.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 0x7f3f980a2380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005950 tx=0x7f3fa000a3b0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 msgr2=0x7f3fb0198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 0x7f3fb0198f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400e9c0 tx=0x7f3fa400ecd0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 shutdown_connections 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fb0108880 0x7f3fb019ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fb0073380 0x7f3fb01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f9809fec0 0x7f3f980a2380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 --2- 172.21.15.3:0/2298756606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb0074950 0x7f3fb0198f00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 >> 172.21.15.3:0/2298756606 conn(0x7f3fb00fbf10 msgr2=0x7f3fb00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:57:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 shutdown_connections 2023-08-11T13:57:41.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:57:41.488+0000 7f3fb5fd7700 1 -- 172.21.15.3:0/2298756606 wait complete. 2023-08-11T13:57:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-08-11T13:57:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:41.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.017 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.3.service 2023-08-11T13:57:42.022 INFO:tasks.cephadm:Deploying osd.4 on smithi103 with /dev/vg_nvme/lv_3... 2023-08-11T13:57:42.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T13:57:42.101 INFO:journalctl@ceph.osd.3.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T13:57:42.586 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:57:42.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:42.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.731 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:42 smithi003 ceph-mon[97195]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:42 smithi187 ceph-mon[130973]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:42.811 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.811 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:42.812 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:57:42.812 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.812 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-08-11T13:57:42.813 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:42 smithi103 ceph-mon[130434]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-mon[97195]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T13:57:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-mon[97195]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T13:57:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-mon[97195]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:43.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-mon[97195]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:43.543 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T13:57:43.487+0000 7f4751340700 -1 osd.3 0 waiting for initial osdmap 2023-08-11T13:57:43.543 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 13:57:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T13:57:43.493+0000 7f474992e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:57:43.641 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:43 smithi103 ceph-mon[130434]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T13:57:43.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:43 smithi103 ceph-mon[130434]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T13:57:43.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:43.642 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:43 smithi103 ceph-mon[130434]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:43.643 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:43 smithi103 ceph-mon[130434]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:43.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:43 smithi187 ceph-mon[130973]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-08-11T13:57:43.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:43 smithi187 ceph-mon[130973]: osdmap e23: 4 total, 3 up, 4 in 2023-08-11T13:57:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:43 smithi187 ceph-mon[130973]: from='osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:43 smithi187 ceph-mon[130973]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011029 s, 951 MB/s 2023-08-11T13:57:44.064 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 2023-08-11T13:57:44.065 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2023-08-11T13:57:44.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2023-08-11T13:57:44.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:44 smithi003 ceph-mon[97195]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:57:44.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:44 smithi003 ceph-mon[97195]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T13:57:44.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:44 smithi003 ceph-mon[97195]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:44 smithi187 ceph-mon[130973]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:57:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:44 smithi187 ceph-mon[130973]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T13:57:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:44 smithi187 ceph-mon[130973]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:44.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:44 smithi103 ceph-mon[130434]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:57:44.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:44 smithi103 ceph-mon[130434]: osdmap e24: 4 total, 3 up, 4 in 2023-08-11T13:57:44.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:44.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:44 smithi103 ceph-mon[130434]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-08-11T13:57:45.091 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:57:45.620 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:57:45.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:57:45.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] boot 2023-08-11T13:57:45.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: osdmap e25: 4 total, 4 up, 4 in 2023-08-11T13:57:45.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi003 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 8636M 2023-08-11T13:57:45.623 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.623 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:45.623 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:45.623 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:57:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:57:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] boot 2023-08-11T13:57:45.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: osdmap e25: 4 total, 4 up, 4 in 2023-08-11T13:57:45.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi003 2023-08-11T13:57:45.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 8636M 2023-08-11T13:57:45.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:45.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:45.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:57:45.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:57:45.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: osd.3 [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] boot 2023-08-11T13:57:45.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: osdmap e25: 4 total, 4 up, 4 in 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi003 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:57:45.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 8636M 2023-08-11T13:57:45.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:45.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:45.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:57:45.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:57:46.282 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 -- 172.21.15.103:0/4090212370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 msgr2=0x7f5a9c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:46.283 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 --2- 172.21.15.103:0/4090212370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c107d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400b0d0 tx=0x7f5a9400b3e0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.283 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 -- 172.21.15.103:0/4090212370 shutdown_connections 2023-08-11T13:57:46.283 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 --2- 172.21.15.103:0/4090212370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c107d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.283 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 --2- 172.21.15.103:0/4090212370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c0ff8e0 0x7f5a9c0ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.283 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 --2- 172.21.15.103:0/4090212370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c100ae0 0x7f5a9c100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.284 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 -- 172.21.15.103:0/4090212370 >> 172.21.15.103:0/4090212370 conn(0x7f5a9c0faea0 msgr2=0x7f5a9c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:57:46.284 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 -- 172.21.15.103:0/4090212370 shutdown_connections 2023-08-11T13:57:46.284 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.280+0000 7f5aa122c700 1 -- 172.21.15.103:0/4090212370 wait complete. 2023-08-11T13:57:46.284 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 Processor -- start 2023-08-11T13:57:46.284 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 -- start start 2023-08-11T13:57:46.286 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c0ff8e0 0x7f5a9c1956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:46.286 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c100ae0 0x7f5a9c195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:46.286 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:46.286 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19d260 con 0x7f5a9c100ae0 2023-08-11T13:57:46.286 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19d3a0 con 0x7f5a9c101480 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5aa122c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19d530 con 0x7f5a9c0ff8e0 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5a9b59e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5a9b59e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58738/0 (socket says 172.21.15.103:58738) 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5a9b59e700 1 -- 172.21.15.103:0/3559538273 learned_addr learned my addr 172.21.15.103:0/3559538273 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5a9ad9d700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c0ff8e0 0x7f5a9c1956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.281+0000 7f5a9a59c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c100ae0 0x7f5a9c195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:46.287 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 -- 172.21.15.103:0/3559538273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c0ff8e0 msgr2=0x7f5a9c1956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c0ff8e0 0x7f5a9c1956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 -- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c100ae0 msgr2=0x7f5a9c195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c100ae0 0x7f5a9c195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 -- 172.21.15.103:0/3559538273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94009d00 con 0x7f5a9c101480 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a9b59e700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94008f10 tx=0x7f5a94008f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400e040 con 0x7f5a9c101480 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c19d7b0 con 0x7f5a9c101480 2023-08-11T13:57:46.288 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c19dca0 con 0x7f5a9c101480 2023-08-11T13:57:46.289 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a940060a0 con 0x7f5a9c101480 2023-08-11T13:57:46.289 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.282+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9401b6c0 con 0x7f5a9c101480 2023-08-11T13:57:46.289 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.283+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c106e40 con 0x7f5a9c101480 2023-08-11T13:57:46.289 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.284+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5a94005200 con 0x7f5a9c101480 2023-08-11T13:57:46.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.285+0000 7f5a83fff700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 0x7f5a840a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:57:46.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.285+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f5a940cb1c0 con 0x7f5a9c101480 2023-08-11T13:57:46.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.285+0000 7f5a9ad9d700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 0x7f5a840a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:57:46.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.286+0000 7f5a9ad9d700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 0x7f5a840a2410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c100940 tx=0x7f5a90009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:57:46.290 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.289+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5a940961e0 con 0x7f5a9c101480 2023-08-11T13:57:46.485 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:57:46.483+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a9c02cd30 con 0x7f5a8409ff50 2023-08-11T13:57:46.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:46 smithi003 ceph-mon[97195]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T13:57:46.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:46 smithi003 ceph-mon[97195]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:46.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:46.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:46.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:46.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:46 smithi187 ceph-mon[130973]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T13:57:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:46 smithi187 ceph-mon[130973]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:46.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:46 smithi103 ceph-mon[130434]: osdmap e26: 4 total, 4 up, 4 in 2023-08-11T13:57:46.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:46 smithi103 ceph-mon[130434]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:46.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:57:46.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:57:46.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:47.605 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:47 smithi103 ceph-mon[130434]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:47.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:47 smithi103 ceph-mon[130434]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T13:57:47.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:47 smithi003 ceph-mon[97195]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:47.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:47 smithi003 ceph-mon[97195]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T13:57:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:47 smithi187 ceph-mon[130973]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:57:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:47 smithi187 ceph-mon[130973]: osdmap e27: 4 total, 4 up, 4 in 2023-08-11T13:57:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T13:57:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/4208538984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]': finished 2023-08-11T13:57:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: osdmap e29: 5 total, 4 up, 5 in 2023-08-11T13:57:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/4208538984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]': finished 2023-08-11T13:57:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: osdmap e29: 5 total, 4 up, 5 in 2023-08-11T13:57:49.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:57:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:57:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: osdmap e28: 4 total, 4 up, 4 in 2023-08-11T13:57:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/4208538984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]: dispatch 2023-08-11T13:57:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c77ed74-b7d5-4c96-8507-8cd0a6a5b379"}]': finished 2023-08-11T13:57:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: osdmap e29: 5 total, 4 up, 5 in 2023-08-11T13:57:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:49 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:57:50.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/3308017776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:50.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/3308017776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/3308017776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:57:51.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:51 smithi003 ceph-mon[97195]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-08-11T13:57:51.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:51 smithi187 ceph-mon[130973]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-08-11T13:57:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:51 smithi103 ceph-mon[130434]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-08-11T13:57:53.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:53 smithi103 ceph-mon[130434]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:57:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:53 smithi003 ceph-mon[97195]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:57:53.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:53 smithi187 ceph-mon[130973]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:57:55.291 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:55 smithi103 ceph-mon[130434]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-08-11T13:57:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:55 smithi003 ceph-mon[97195]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-08-11T13:57:55.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:55 smithi187 ceph-mon[130973]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-08-11T13:57:56.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T13:57:56.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:56.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:56 smithi103 ceph-mon[130434]: Deploying daemon osd.4 on smithi103 2023-08-11T13:57:56.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T13:57:56.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:56.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:56 smithi003 ceph-mon[97195]: Deploying daemon osd.4 on smithi103 2023-08-11T13:57:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-08-11T13:57:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:57:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:56 smithi187 ceph-mon[130973]: Deploying daemon osd.4 on smithi103 2023-08-11T13:57:57.408 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:57 smithi103 ceph-mon[130434]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T13:57:57.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:57 smithi003 ceph-mon[97195]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T13:57:57.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:57 smithi187 ceph-mon[130973]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T13:57:59.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:57:59 smithi103 ceph-mon[130434]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:57:59.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:57:59 smithi003 ceph-mon[97195]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:57:59.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:57:59 smithi187 ceph-mon[130973]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:58:00.668 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:00.669 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:00 smithi103 ceph-mon[130434]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:00.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:00 smithi003 ceph-mon[97195]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:00 smithi187 ceph-mon[130973]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:02.151 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.150+0000 7f5a83fff700 1 -- 172.21.15.103:0/3559538273 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a9c02cd30 con 0x7f5a8409ff50 2023-08-11T13:58:02.151 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 4 on host 'smithi103' 2023-08-11T13:58:02.153 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 msgr2=0x7f5a840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:02.153 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 0x7f5a840a2410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c100940 tx=0x7f5a90009190 comp rx=0 tx=0).stop 2023-08-11T13:58:02.153 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 msgr2=0x7f5a9c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94008f10 tx=0x7f5a94008f40 comp rx=0 tx=0).stop 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 shutdown_connections 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5a8409ff50 0x7f5a840a2410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c101480 0x7f5a9c19cc90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c0ff8e0 0x7f5a9c1956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 --2- 172.21.15.103:0/3559538273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a9c100ae0 0x7f5a9c195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 >> 172.21.15.103:0/3559538273 conn(0x7f5a9c0faea0 msgr2=0x7f5a9c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:02.154 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 shutdown_connections 2023-08-11T13:58:02.155 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:02.152+0000 7f5aa122c700 1 -- 172.21.15.103:0/3559538273 wait complete. 2023-08-11T13:58:02.585 DEBUG:teuthology.orchestra.run.smithi103:osd.4> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.4.service 2023-08-11T13:58:02.588 INFO:tasks.cephadm:Deploying osd.5 on smithi187 with /dev/vg_nvme/lv_3... 2023-08-11T13:58:02.588 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-08-11T13:58:02.620 INFO:journalctl@ceph.osd.4.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T13:58:02.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:02.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:02.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:02.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:02.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:02.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:02.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:02.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:02.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:03.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:03.152 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:58:03.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:03.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:03.929 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 13:58:03 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T13:58:03.545+0000 7f85ad3ab640 -1 osd.4 0 log_to_monitors true 2023-08-11T13:58:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:04 smithi187 ceph-mon[130973]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:04 smithi187 ceph-mon[130973]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:04 smithi103 ceph-mon[130434]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:04 smithi103 ceph-mon[130434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:04 smithi003 ceph-mon[97195]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:04 smithi003 ceph-mon[97195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-08-11T13:58:04.675 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-08-11T13:58:04.675 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-08-11T13:58:04.675 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-08-11T13:58:04.675 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-08-11T13:58:04.676 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150745 s, 696 MB/s 2023-08-11T13:58:04.676 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-08-11T13:58:05.107 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi187:vg_nvme/lv_3 2023-08-11T13:58:05.132 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T13:58:05.124+0000 7f85a2a23700 -1 osd.4 0 waiting for initial osdmap 2023-08-11T13:58:05.132 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: osdmap e30: 5 total, 4 up, 5 in 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.133 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:05.134 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T13:58:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: osdmap e30: 5 total, 4 up, 5 in 2023-08-11T13:58:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:05.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:05.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.428 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 13:58:05 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T13:58:05.131+0000 7f859c814700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:58:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-08-11T13:58:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: osdmap e30: 5 total, 4 up, 5 in 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:05.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:05.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:05.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:05.665 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:58:06.211 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi103 2023-08-11T13:58:06.211 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 8584M 2023-08-11T13:58:06.211 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:58:06.212 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T13:58:06.212 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.212 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:06 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi103 2023-08-11T13:58:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 8584M 2023-08-11T13:58:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:58:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T13:58:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:06 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi103 2023-08-11T13:58:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 8584M 2023-08-11T13:58:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:58:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: osdmap e31: 5 total, 4 up, 5 in 2023-08-11T13:58:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:06 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:06.848 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 -- 172.21.15.187:0/1324571612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 msgr2=0x7f56e40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:06.848 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1324571612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e40fffe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009990 tx=0x7f56d8009ca0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.848 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 -- 172.21.15.187:0/1324571612 shutdown_connections 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1324571612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100520 0x7f56e4107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1324571612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56e4100e40 0x7f56e4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1324571612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e40fffe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 -- 172.21.15.187:0/1324571612 >> 172.21.15.187:0/1324571612 conn(0x7f56e40fa5d0 msgr2=0x7f56e40fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 -- 172.21.15.187:0/1324571612 shutdown_connections 2023-08-11T13:58:06.849 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.847+0000 7f56ea32e700 1 -- 172.21.15.187:0/1324571612 wait complete. 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 Processor -- start 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 -- start start 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56e4100520 0x7f56e4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100e40 0x7f56e419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e419e300 con 0x7f56e4100520 2023-08-11T13:58:06.851 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e419e440 con 0x7f56e4100e40 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.848+0000 7f56ea32e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e419e5d0 con 0x7f56e40ffb80 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:55000/0 (socket says 172.21.15.187:55000) 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 -- 172.21.15.187:0/1836542281 learned_addr learned my addr 172.21.15.187:0/1836542281 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e88cb700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100e40 0x7f56e419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 -- 172.21.15.187:0/1836542281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100e40 msgr2=0x7f56e419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:06.852 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100e40 0x7f56e419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 -- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56e4100520 msgr2=0x7f56e4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56e4100520 0x7f56e4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 -- 172.21.15.187:0/1836542281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80095f0 con 0x7f56e40ffb80 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e3fff700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b530 tx=0x7f56d400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011830 con 0x7f56e40ffb80 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d4011e50 con 0x7f56e40ffb80 2023-08-11T13:58:06.853 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e419e8b0 con 0x7f56e40ffb80 2023-08-11T13:58:06.854 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400f2e0 con 0x7f56e40ffb80 2023-08-11T13:58:06.854 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.849+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e419ee30 con 0x7f56e40ffb80 2023-08-11T13:58:06.854 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.850+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e410c2d0 con 0x7f56e40ffb80 2023-08-11T13:58:06.855 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.851+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f56d4010580 con 0x7f56e40ffb80 2023-08-11T13:58:06.855 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.852+0000 7f56e17fa700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 0x7f56cc0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:06.855 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.852+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f56d40ca230 con 0x7f56e40ffb80 2023-08-11T13:58:06.855 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.852+0000 7f56e37fe700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 0x7f56cc0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:06.855 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.853+0000 7f56e37fe700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 0x7f56cc0a2490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f56d80095c0 tx=0x7f56d8009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:06.856 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:06.855+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f56d4094f90 con 0x7f56e40ffb80 2023-08-11T13:58:07.050 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:07.049+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f56e4060d60 con 0x7f56cc09ffd0 2023-08-11T13:58:07.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:58:07.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:58:07.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] boot 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: osdmap e32: 5 total, 5 up, 5 in 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:07.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:58:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] boot 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: osdmap e32: 5 total, 5 up, 5 in 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:07.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:07.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: osd.4 [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] boot 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: osdmap e32: 5 total, 5 up, 5 in 2023-08-11T13:58:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-08-11T13:58:07.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:07.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:07.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:08.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:08 smithi103 ceph-mon[130434]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:08.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:08 smithi103 ceph-mon[130434]: osdmap e33: 5 total, 5 up, 5 in 2023-08-11T13:58:08.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:08 smithi003 ceph-mon[97195]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:08.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:08 smithi003 ceph-mon[97195]: osdmap e33: 5 total, 5 up, 5 in 2023-08-11T13:58:08.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:08 smithi187 ceph-mon[130973]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:08.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:08 smithi187 ceph-mon[130973]: osdmap e33: 5 total, 5 up, 5 in 2023-08-11T13:58:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:09 smithi003 ceph-mon[97195]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:09.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:09 smithi187 ceph-mon[130973]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:09 smithi103 ceph-mon[130434]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:10.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/2921290299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]': finished 2023-08-11T13:58:10.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: osdmap e34: 6 total, 5 up, 6 in 2023-08-11T13:58:10.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:10.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/563598055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/2921290299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]': finished 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: osdmap e34: 6 total, 5 up, 6 in 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:10.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:10 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/563598055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/2921290299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]: dispatch 2023-08-11T13:58:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03c1c84c-d569-4383-b6b6-c0a38ea0ac4e"}]': finished 2023-08-11T13:58:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: osdmap e34: 6 total, 5 up, 6 in 2023-08-11T13:58:10.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:10.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:10 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/563598055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:11 smithi003 ceph-mon[97195]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:11.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:11 smithi187 ceph-mon[130973]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:11 smithi103 ceph-mon[130434]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:13 smithi003 ceph-mon[97195]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:13.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:13 smithi187 ceph-mon[130973]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:13 smithi103 ceph-mon[130434]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:15.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:15 smithi003 ceph-mon[97195]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:15.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:15 smithi187 ceph-mon[130973]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:15 smithi103 ceph-mon[130434]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:16.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T13:58:16.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:16.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T13:58:16.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:16.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-08-11T13:58:16.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:17.312 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:17 smithi187 ceph-mon[130973]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:17.312 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:17 smithi187 ceph-mon[130973]: Deploying daemon osd.5 on smithi187 2023-08-11T13:58:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:17 smithi003 ceph-mon[97195]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:17 smithi003 ceph-mon[97195]: Deploying daemon osd.5 on smithi187 2023-08-11T13:58:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:17 smithi103 ceph-mon[130434]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:17 smithi103 ceph-mon[130434]: Deploying daemon osd.5 on smithi187 2023-08-11T13:58:19.513 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:19 smithi187 ceph-mon[130973]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:19.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:19 smithi003 ceph-mon[97195]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:19 smithi103 ceph-mon[130434]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:21.475 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:21 smithi187 ceph-mon[130973]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:21.475 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.476 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.476 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:21 smithi003 ceph-mon[97195]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:21 smithi103 ceph-mon[130434]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:21.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:21.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:23.220 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.218+0000 7f56e17fa700 1 -- 172.21.15.187:0/1836542281 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f56e4060d60 con 0x7f56cc09ffd0 2023-08-11T13:58:23.220 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 5 on host 'smithi187' 2023-08-11T13:58:23.222 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.220+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 msgr2=0x7f56cc0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:23.222 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.220+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 0x7f56cc0a2490 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f56d80095c0 tx=0x7f56d8009530 comp rx=0 tx=0).stop 2023-08-11T13:58:23.222 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.220+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 msgr2=0x7f56e4196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:23.222 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.220+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b530 tx=0x7f56d400b840 comp rx=0 tx=0).stop 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 shutdown_connections 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56cc09ffd0 0x7f56cc0a2490 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56e4100e40 0x7f56e419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e40ffb80 0x7f56e4196770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 --2- 172.21.15.187:0/1836542281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56e4100520 0x7f56e4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 >> 172.21.15.187:0/1836542281 conn(0x7f56e40fa5d0 msgr2=0x7f56e40fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:23.223 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 shutdown_connections 2023-08-11T13:58:23.224 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:58:23.221+0000 7f56ea32e700 1 -- 172.21.15.187:0/1836542281 wait complete. 2023-08-11T13:58:23.382 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:23.382 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:23.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:23.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:23.749 DEBUG:teuthology.orchestra.run.smithi187:osd.5> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.5.service 2023-08-11T13:58:23.752 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2023-08-11T13:58:23.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T13:58:23.794 INFO:journalctl@ceph.osd.5.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T13:58:24.373 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:24.512 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:58:24.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:24.649 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 13:58:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T13:58:24.371+0000 7f31099c7640 -1 osd.5 0 log_to_monitors true 2023-08-11T13:58:24.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:25.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:25 smithi003 ceph-mon[97195]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:25.397 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:25 smithi003 ceph-mon[97195]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:25.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:25 smithi003 ceph-mon[97195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:25.481 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:25 smithi187 ceph-mon[130973]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:25.481 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:25 smithi187 ceph-mon[130973]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:25.481 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:25 smithi187 ceph-mon[130973]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:25 smithi103 ceph-mon[130434]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:25 smithi103 ceph-mon[130434]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:25 smithi103 ceph-mon[130434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-08-11T13:58:26.515 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T13:58:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T13:58:26.516 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2023-08-11T13:58:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2023-08-11T13:58:26.516 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.030132 s, 348 MB/s 2023-08-11T13:58:26.516 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2023-08-11T13:58:26.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T13:58:26.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: osdmap e35: 6 total, 5 up, 6 in 2023-08-11T13:58:26.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:26.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:26.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:26.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.546 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T13:58:26.233+0000 7f30ff03f700 -1 osd.5 0 waiting for initial osdmap 2023-08-11T13:58:26.546 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T13:58:26.238+0000 7f30f7e2e700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:58:26.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T13:58:26.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: osdmap e35: 6 total, 5 up, 6 in 2023-08-11T13:58:26.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:26.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:26.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:26.551 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-08-11T13:58:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: osdmap e35: 6 total, 5 up, 6 in 2023-08-11T13:58:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:26.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:26.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:26.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2023-08-11T13:58:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi187 2023-08-11T13:58:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 8584M 2023-08-11T13:58:27.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:58:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T13:58:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] boot 2023-08-11T13:58:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: osdmap e37: 6 total, 6 up, 6 in 2023-08-11T13:58:27.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:27 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi187 2023-08-11T13:58:27.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 8584M 2023-08-11T13:58:27.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] boot 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: osdmap e37: 6 total, 6 up, 6 in 2023-08-11T13:58:27.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:27 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi187 2023-08-11T13:58:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 8584M 2023-08-11T13:58:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-08-11T13:58:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: osdmap e36: 6 total, 5 up, 6 in 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: osd.5 [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] boot 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: osdmap e37: 6 total, 6 up, 6 in 2023-08-11T13:58:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:27 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-08-11T13:58:27.726 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:58:28.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:28 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:58:28.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:28 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:58:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:28 smithi003 ceph-mon[97195]: osdmap e38: 6 total, 6 up, 6 in 2023-08-11T13:58:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:28 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:58:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:28 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:58:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:28 smithi187 ceph-mon[130973]: osdmap e38: 6 total, 6 up, 6 in 2023-08-11T13:58:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:28 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:58:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:28 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:58:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:28 smithi103 ceph-mon[130434]: osdmap e38: 6 total, 6 up, 6 in 2023-08-11T13:58:29.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 -- 172.21.15.3:0/2405346846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24073380 msgr2=0x7feb24073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:29.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 --2- 172.21.15.3:0/2405346846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24073380 0x7feb24073800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb10009990 tx=0x7feb10009ca0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 -- 172.21.15.3:0/2405346846 shutdown_connections 2023-08-11T13:58:29.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 --2- 172.21.15.3:0/2405346846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2410cf90 0x7feb2410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 --2- 172.21.15.3:0/2405346846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb24074950 0x7feb24072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 --2- 172.21.15.3:0/2405346846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24073380 0x7feb24073800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 -- 172.21.15.3:0/2405346846 >> 172.21.15.3:0/2405346846 conn(0x7feb240fbf10 msgr2=0x7feb240fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.245+0000 7feb2b071700 1 -- 172.21.15.3:0/2405346846 shutdown_connections 2023-08-11T13:58:29.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.246+0000 7feb2b071700 1 -- 172.21.15.3:0/2405346846 wait complete. 2023-08-11T13:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 Processor -- start 2023-08-11T13:58:29.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 -- start start 2023-08-11T13:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb24073380 0x7feb2419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2410cf90 0x7feb241a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241a4850 con 0x7feb24074950 2023-08-11T13:58:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241a4990 con 0x7feb24073380 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb2b071700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241a4b50 con 0x7feb2410cf90 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb23fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb23fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48000/0 (socket says 172.21.15.3:48000) 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.247+0000 7feb23fff700 1 -- 172.21.15.3:0/2782754429 learned_addr learned my addr 172.21.15.3:0/2782754429 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb28e0d700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb24073380 0x7feb2419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb2960e700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2410cf90 0x7feb241a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 -- 172.21.15.3:0/2782754429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2410cf90 msgr2=0x7feb241a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2410cf90 0x7feb241a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 -- 172.21.15.3:0/2782754429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb24073380 msgr2=0x7feb2419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb24073380 0x7feb2419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 -- 172.21.15.3:0/2782754429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100095f0 con 0x7feb24074950 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb23fff700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb10005720 tx=0x7feb10009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1001d070 con 0x7feb24074950 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb241a4dd0 con 0x7feb24074950 2023-08-11T13:58:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.248+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb10021410 con 0x7feb24074950 2023-08-11T13:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.249+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10017650 con 0x7feb24074950 2023-08-11T13:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.249+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb241a5190 con 0x7feb24074950 2023-08-11T13:58:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.250+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24102ae0 con 0x7feb24074950 2023-08-11T13:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.252+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7feb10021580 con 0x7feb24074950 2023-08-11T13:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.253+0000 7feb21ffb700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 0x7feb140a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.254+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7feb100cba50 con 0x7feb24074950 2023-08-11T13:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.254+0000 7feb28e0d700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 0x7feb140a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:29.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.254+0000 7feb28e0d700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 0x7feb140a2490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb2419e1e0 tx=0x7feb18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.256+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7feb10096670 con 0x7feb24074950 2023-08-11T13:58:29.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:29.473+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7feb2402ceb0 con 0x7feb1409ffd0 2023-08-11T13:58:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:29 smithi003 ceph-mon[97195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:29 smithi003 ceph-mon[97195]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T13:58:29.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:29 smithi187 ceph-mon[130973]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:29.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:29 smithi187 ceph-mon[130973]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T13:58:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:29 smithi103 ceph-mon[130434]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:29 smithi103 ceph-mon[130434]: osdmap e39: 6 total, 6 up, 6 in 2023-08-11T13:58:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:30.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:30.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:31 smithi003 ceph-mon[97195]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:31 smithi003 ceph-mon[97195]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:31 smithi003 ceph-mon[97195]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T13:58:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:31 smithi187 ceph-mon[130973]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:31 smithi187 ceph-mon[130973]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:31 smithi187 ceph-mon[130973]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T13:58:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:31 smithi103 ceph-mon[130434]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:31 smithi103 ceph-mon[130434]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:31 smithi103 ceph-mon[130434]: osdmap e40: 6 total, 6 up, 6 in 2023-08-11T13:58:32.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:32 smithi003 ceph-mon[97195]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T13:58:32.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:32 smithi187 ceph-mon[130973]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T13:58:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:32 smithi103 ceph-mon[130434]: osdmap e41: 6 total, 6 up, 6 in 2023-08-11T13:58:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:33 smithi003 ceph-mon[97195]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:33 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]: dispatch 2023-08-11T13:58:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:33 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]': finished 2023-08-11T13:58:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:33 smithi003 ceph-mon[97195]: osdmap e42: 7 total, 6 up, 7 in 2023-08-11T13:58:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:33 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:33.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:33 smithi187 ceph-mon[130973]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:33 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]: dispatch 2023-08-11T13:58:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:33 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]': finished 2023-08-11T13:58:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:33 smithi187 ceph-mon[130973]: osdmap e42: 7 total, 6 up, 7 in 2023-08-11T13:58:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:33 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:33 smithi103 ceph-mon[130434]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:33 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]: dispatch 2023-08-11T13:58:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:33 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2623662918' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46bc04c3-30e0-461f-a65c-a9acecf575a7"}]': finished 2023-08-11T13:58:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:33 smithi103 ceph-mon[130434]: osdmap e42: 7 total, 6 up, 7 in 2023-08-11T13:58:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:33 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:34 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/400209691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:34.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:34 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/400209691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:34 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/400209691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:35.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:35 smithi003 ceph-mon[97195]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-08-11T13:58:35.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:35 smithi187 ceph-mon[130973]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-08-11T13:58:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:35 smithi103 ceph-mon[130434]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-08-11T13:58:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:37 smithi003 ceph-mon[97195]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:58:37.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:37 smithi187 ceph-mon[130973]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:58:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:37 smithi103 ceph-mon[130434]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:58:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:39 smithi003 ceph-mon[97195]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-08-11T13:58:39.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:39 smithi187 ceph-mon[130973]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-08-11T13:58:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:39 smithi103 ceph-mon[130434]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-08-11T13:58:40.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:40 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T13:58:40.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:40 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:40.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:40 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T13:58:40.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:40 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:40 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-08-11T13:58:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:40 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:41.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:41 smithi003 ceph-mon[97195]: Deploying daemon osd.6 on smithi003 2023-08-11T13:58:41.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:41 smithi003 ceph-mon[97195]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-08-11T13:58:41.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:41 smithi187 ceph-mon[130973]: Deploying daemon osd.6 on smithi003 2023-08-11T13:58:41.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:41 smithi187 ceph-mon[130973]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-08-11T13:58:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:41 smithi103 ceph-mon[130434]: Deploying daemon osd.6 on smithi003 2023-08-11T13:58:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:41 smithi103 ceph-mon[130434]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-08-11T13:58:43.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:43 smithi003 ceph-mon[97195]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:58:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:43 smithi187 ceph-mon[130973]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:58:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:43 smithi103 ceph-mon[130434]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:58:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:45 smithi187 ceph-mon[130973]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T13:58:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:45 smithi103 ceph-mon[130434]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T13:58:45.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:45 smithi003 ceph-mon[97195]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-08-11T13:58:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:47 smithi103 ceph-mon[130434]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:47 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:47 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:47 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:47.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:47 smithi003 ceph-mon[97195]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:47.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:47 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:47 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:47 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:47 smithi187 ceph-mon[130973]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:47.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:47 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:47 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:47.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:47 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:49.617 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:49 smithi003 ceph-mon[97195]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:49 smithi103 ceph-mon[130434]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:49 smithi187 ceph-mon[130973]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.037+0000 7feb21ffb700 1 -- 172.21.15.3:0/2782754429 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7feb2402ceb0 con 0x7feb1409ffd0 2023-08-11T13:58:50.039 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2023-08-11T13:58:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 msgr2=0x7feb140a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 0x7feb140a2490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb2419e1e0 tx=0x7feb18009260 comp rx=0 tx=0).stop 2023-08-11T13:58:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 msgr2=0x7feb2419d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb10005720 tx=0x7feb10009580 comp rx=0 tx=0).stop 2023-08-11T13:58:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 shutdown_connections 2023-08-11T13:58:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2410cf90 0x7feb241a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1409ffd0 0x7feb140a2490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb24073380 0x7feb2419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 --2- 172.21.15.3:0/2782754429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb24074950 0x7feb2419d240 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 >> 172.21.15.3:0/2782754429 conn(0x7feb240fbf10 msgr2=0x7feb240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 shutdown_connections 2023-08-11T13:58:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:58:50.040+0000 7feb2b071700 1 -- 172.21.15.3:0/2782754429 wait complete. 2023-08-11T13:58:50.549 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.6.service 2023-08-11T13:58:50.554 INFO:tasks.cephadm:Deploying osd.7 on smithi103 with /dev/vg_nvme/lv_2... 2023-08-11T13:58:50.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T13:58:50.642 INFO:journalctl@ceph.osd.6.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T13:58:50.791 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T13:58:50.641+0000 7f1e090d7640 -1 osd.6 0 log_to_monitors true 2023-08-11T13:58:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:50.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:50 smithi103 ceph-mon[130434]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T13:58:51.134 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:58:51.264 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.264 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:51.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:50 smithi003 ceph-mon[97195]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T13:58:51.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:51.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:58:51.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:51.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:50 smithi187 ceph-mon[130973]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-08-11T13:58:52.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:52 smithi103 ceph-mon[130434]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T13:58:52.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:52 smithi103 ceph-mon[130434]: osdmap e43: 7 total, 6 up, 7 in 2023-08-11T13:58:52.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:52.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:52 smithi103 ceph-mon[130434]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:58:52.235 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T13:58:52.004+0000 7f1dfe74f700 -1 osd.6 0 waiting for initial osdmap 2023-08-11T13:58:52.235 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T13:58:52.017+0000 7f1df8540700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:58:52.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-mon[97195]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T13:58:52.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-mon[97195]: osdmap e43: 7 total, 6 up, 7 in 2023-08-11T13:58:52.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:52.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:52 smithi003 ceph-mon[97195]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:58:52.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:52 smithi187 ceph-mon[130973]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-08-11T13:58:52.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:52 smithi187 ceph-mon[130973]: osdmap e43: 7 total, 6 up, 7 in 2023-08-11T13:58:52.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:52.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:52 smithi187 ceph-mon[130973]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T13:58:52.605 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T13:58:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T13:58:52.606 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2023-08-11T13:58:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2023-08-11T13:58:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112856 s, 929 MB/s 2023-08-11T13:58:52.606 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2023-08-11T13:58:53.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2023-08-11T13:58:53.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:53 smithi003 ceph-mon[97195]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:58:53.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:53 smithi003 ceph-mon[97195]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T13:58:53.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:53 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:53 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:53 smithi003 ceph-mon[97195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:53.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:53 smithi187 ceph-mon[130973]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:58:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:53 smithi187 ceph-mon[130973]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T13:58:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:53 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:53 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:53 smithi187 ceph-mon[130973]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:53.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:53 smithi103 ceph-mon[130434]: from='osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T13:58:53.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:53 smithi103 ceph-mon[130434]: osdmap e44: 7 total, 6 up, 7 in 2023-08-11T13:58:53.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:53 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:53 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:53.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:53 smithi103 ceph-mon[130434]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-08-11T13:58:53.627 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T13:58:54.145 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:58:54.145 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:58:54.145 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] boot 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: osdmap e45: 7 total, 7 up, 7 in 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.146 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.147 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.147 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:54.147 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:54.147 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:58:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:58:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] boot 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: osdmap e45: 7 total, 7 up, 7 in 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: osd.6 [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] boot 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: osdmap e45: 7 total, 7 up, 7 in 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-08-11T13:58:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:58:54.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:58:54.855 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.852+0000 7fe965074700 1 -- 172.21.15.103:0/4128914381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960101450 msgr2=0x7fe960107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.852+0000 7fe965074700 1 --2- 172.21.15.103:0/4128914381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960101450 0x7fe960107d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b0d0 tx=0x7fe95800b3e0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 -- 172.21.15.103:0/4128914381 shutdown_connections 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 --2- 172.21.15.103:0/4128914381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960101450 0x7fe960107d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 --2- 172.21.15.103:0/4128914381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe9600ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 --2- 172.21.15.103:0/4128914381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960100ab0 0x7fe960100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.856 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 -- 172.21.15.103:0/4128914381 >> 172.21.15.103:0/4128914381 conn(0x7fe9600fae30 msgr2=0x7fe9600fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 -- 172.21.15.103:0/4128914381 shutdown_connections 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 -- 172.21.15.103:0/4128914381 wait complete. 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.853+0000 7fe965074700 1 Processor -- start 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 -- start start 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe960195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:54.857 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960101450 0x7fe96019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96019d1d0 con 0x7fe9600ff8b0 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96019d310 con 0x7fe960100ab0 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe965074700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96019d4a0 con 0x7fe960101450 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58848/0 (socket says 172.21.15.103:58848) 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 -- 172.21.15.103:0/1615918999 learned_addr learned my addr 172.21.15.103:0/1615918999 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95f59e700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960101450 0x7fe96019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:54.858 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95ed9d700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe960195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 -- 172.21.15.103:0/1615918999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960101450 msgr2=0x7fe96019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960101450 0x7fe96019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 -- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 msgr2=0x7fe960195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe960195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 -- 172.21.15.103:0/1615918999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958009d00 con 0x7fe960100ab0 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95ed9d700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe960195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T13:58:54.859 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.854+0000 7fe95e59c700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d950 tx=0x7fe95000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950009940 con 0x7fe960100ab0 2023-08-11T13:58:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96019d780 con 0x7fe960100ab0 2023-08-11T13:58:54.860 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe950010460 con 0x7fe960100ab0 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000f600 con 0x7fe960100ab0 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96019dcd0 con 0x7fe960100ab0 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.855+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96010b0c0 con 0x7fe960100ab0 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.857+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe9500105d0 con 0x7fe960100ab0 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.857+0000 7fe947fff700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 0x7fe9480a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:58:54.861 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.857+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fe9500ca450 con 0x7fe960100ab0 2023-08-11T13:58:54.862 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.857+0000 7fe95ed9d700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 0x7fe9480a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:58:54.862 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.858+0000 7fe95ed9d700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 0x7fe9480a2490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe954005950 tx=0x7fe9540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:58:54.862 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:54.861+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe950094e50 con 0x7fe960100ab0 2023-08-11T13:58:55.061 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:58:55.060+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe960106d00 con 0x7fe94809ffd0 2023-08-11T13:58:55.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:55 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi003 2023-08-11T13:58:55.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:55 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 5757M 2023-08-11T13:58:55.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:55 smithi003 ceph-mon[97195]: osdmap e46: 7 total, 7 up, 7 in 2023-08-11T13:58:55.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:55 smithi003 ceph-mon[97195]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:55.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:55 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi003 2023-08-11T13:58:55.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:55 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 5757M 2023-08-11T13:58:55.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:55 smithi187 ceph-mon[130973]: osdmap e46: 7 total, 7 up, 7 in 2023-08-11T13:58:55.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:55 smithi187 ceph-mon[130973]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:55.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:55 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi003 2023-08-11T13:58:55.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:55 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 5757M 2023-08-11T13:58:55.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:55 smithi103 ceph-mon[130434]: osdmap e46: 7 total, 7 up, 7 in 2023-08-11T13:58:55.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:55 smithi103 ceph-mon[130434]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:56.161 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:56 smithi103 ceph-mon[130434]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:56.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:56.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:56.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:56.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:56 smithi003 ceph-mon[97195]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:56.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:56.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:56.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:56 smithi187 ceph-mon[130973]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:58:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:58:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:58:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:58:57.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:57 smithi003 ceph-mon[97195]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:57.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:57 smithi187 ceph-mon[130973]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:57.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:57 smithi103 ceph-mon[130434]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:58.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/4045184708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]': finished 2023-08-11T13:58:58.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: osdmap e47: 8 total, 7 up, 8 in 2023-08-11T13:58:58.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:58:58.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/1149575041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:58.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/4045184708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]': finished 2023-08-11T13:58:58.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: osdmap e47: 8 total, 7 up, 8 in 2023-08-11T13:58:58.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:58:58.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/1149575041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/4045184708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]: dispatch 2023-08-11T13:58:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1080578d-40d4-418d-9db4-999d831dd105"}]': finished 2023-08-11T13:58:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: osdmap e47: 8 total, 7 up, 8 in 2023-08-11T13:58:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:58:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/1149575041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:58:59.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:58:59 smithi003 ceph-mon[97195]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:59.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:58:59 smithi187 ceph-mon[130973]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:58:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:58:59 smithi103 ceph-mon[130434]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:01 smithi103 ceph-mon[130434]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:01.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:01 smithi003 ceph-mon[97195]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:01 smithi187 ceph-mon[130973]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:03.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:03 smithi103 ceph-mon[130434]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:03 smithi003 ceph-mon[97195]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:03.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:03 smithi187 ceph-mon[130973]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:04.466 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T13:59:04.466 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T13:59:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:04.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-08-11T13:59:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:05 smithi003 ceph-mon[97195]: Deploying daemon osd.7 on smithi103 2023-08-11T13:59:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:05 smithi003 ceph-mon[97195]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:05.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:05 smithi187 ceph-mon[130973]: Deploying daemon osd.7 on smithi103 2023-08-11T13:59:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:05 smithi187 ceph-mon[130973]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:05.657 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:05 smithi103 ceph-mon[130434]: Deploying daemon osd.7 on smithi103 2023-08-11T13:59:05.658 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:05 smithi103 ceph-mon[130434]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:07 smithi003 ceph-mon[97195]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:07.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:07 smithi187 ceph-mon[130973]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:07 smithi103 ceph-mon[130434]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:09 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:09 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:09 smithi103 ceph-mon[130434]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:09 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:09 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:09 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:09.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:09 smithi003 ceph-mon[97195]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:09.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:09 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:09.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:09 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:09.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:09 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:09.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:09 smithi187 ceph-mon[130973]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:09.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:09 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:10.877 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.876+0000 7fe947fff700 1 -- 172.21.15.103:0/1615918999 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe960106d00 con 0x7fe94809ffd0 2023-08-11T13:59:10.877 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 7 on host 'smithi103' 2023-08-11T13:59:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 msgr2=0x7fe9480a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 0x7fe9480a2490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe954005950 tx=0x7fe9540058e0 comp rx=0 tx=0).stop 2023-08-11T13:59:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 msgr2=0x7fe960195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe95000d950 tx=0x7fe95000dc60 comp rx=0 tx=0).stop 2023-08-11T13:59:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 shutdown_connections 2023-08-11T13:59:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe960101450 0x7fe96019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9600ff8b0 0x7fe960195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe94809ffd0 0x7fe9480a2490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 --2- 172.21.15.103:0/1615918999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe960100ab0 0x7fe960195b80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 >> 172.21.15.103:0/1615918999 conn(0x7fe9600fae30 msgr2=0x7fe960103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:59:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 shutdown_connections 2023-08-11T13:59:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T13:59:10.878+0000 7fe965074700 1 -- 172.21.15.103:0/1615918999 wait complete. 2023-08-11T13:59:11.326 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.7.service 2023-08-11T13:59:11.328 INFO:tasks.cephadm:Deploying osd.8 on smithi187 with /dev/vg_nvme/lv_2... 2023-08-11T13:59:11.329 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-08-11T13:59:11.361 INFO:journalctl@ceph.osd.7.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T13:59:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:11.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:11.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:11.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:11.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:11.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:11.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:11.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:11.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:11.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:11.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:11.898 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:59:12.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:12 smithi103 ceph-mon[130434]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:12.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:12 smithi103 ceph-mon[130434]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:12.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 13:59:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T13:59:12.106+0000 7f56259c5640 -1 osd.7 0 log_to_monitors true 2023-08-11T13:59:12.489 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:12 smithi187 ceph-mon[130973]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:12.489 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:12 smithi187 ceph-mon[130973]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:12.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:12 smithi003 ceph-mon[97195]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:12.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:12 smithi003 ceph-mon[97195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-08-11T13:59:13.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T13:59:13.200+0000 7f561b03d700 -1 osd.7 0 waiting for initial osdmap 2023-08-11T13:59:13.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T13:59:13.208+0000 7f5616e32700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: osdmap e48: 8 total, 7 up, 8 in 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:13 smithi103 ceph-mon[130434]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.460 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-08-11T13:59:13.461 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-08-11T13:59:13.461 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-08-11T13:59:13.461 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-08-11T13:59:13.461 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110615 s, 948 MB/s 2023-08-11T13:59:13.461 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-08-11T13:59:13.498 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:13.498 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T13:59:13.498 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: osdmap e48: 8 total, 7 up, 8 in 2023-08-11T13:59:13.498 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:13.499 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.499 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:13 smithi187 ceph-mon[130973]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-08-11T13:59:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: osdmap e48: 8 total, 7 up, 8 in 2023-08-11T13:59:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: from='osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:13 smithi003 ceph-mon[97195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T13:59:13.941 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi187:vg_nvme/lv_2 2023-08-11T13:59:14.493 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T13:59:14.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:59:14.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T13:59:14.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:14.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:14.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:59:14.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T13:59:14.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:14.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:14.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T13:59:14.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: osdmap e49: 8 total, 7 up, 8 in 2023-08-11T13:59:14.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:14.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:14.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:15.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:59:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:59:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi103 2023-08-11T13:59:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 5722M 2023-08-11T13:59:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] boot 2023-08-11T13:59:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: osdmap e50: 8 total, 8 up, 8 in 2023-08-11T13:59:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi103 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 5722M 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] boot 2023-08-11T13:59:15.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: osdmap e50: 8 total, 8 up, 8 in 2023-08-11T13:59:15.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:59:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:59:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi103 2023-08-11T13:59:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 5722M 2023-08-11T13:59:15.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-08-11T13:59:15.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: osd.7 [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] boot 2023-08-11T13:59:15.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: osdmap e50: 8 total, 8 up, 8 in 2023-08-11T13:59:15.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-08-11T13:59:15.692 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- 172.21.15.187:0/3973671622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 msgr2=0x7fa6b8100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:15.693 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/3973671622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8100f70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009990 tx=0x7fa6a8009ca0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.693 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- 172.21.15.187:0/3973671622 shutdown_connections 2023-08-11T13:59:15.694 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/3973671622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 0x7fa6b8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/3973671622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b80ff910 0x7fa6b80ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.696 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/3973671622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8100f70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- 172.21.15.187:0/3973671622 >> 172.21.15.187:0/3973671622 conn(0x7fa6b80faeb0 msgr2=0x7fa6b80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:59:15.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- 172.21.15.187:0/3973671622 shutdown_connections 2023-08-11T13:59:15.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- 172.21.15.187:0/3973671622 wait complete. 2023-08-11T13:59:15.697 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 Processor -- start 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.684+0000 7fa6bd32c700 1 -- start start 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b80ff910 0x7fa6b8195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 0x7fa6b819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b819d1e0 con 0x7fa6b80ff910 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b819d320 con 0x7fa6b81014b0 2023-08-11T13:59:15.698 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b819d4b0 con 0x7fa6b8100b10 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:55114/0 (socket says 172.21.15.187:55114) 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 -- 172.21.15.187:0/2086826517 learned_addr learned my addr 172.21.15.187:0/2086826517 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b77fe700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 0x7fa6b819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 -- 172.21.15.187:0/2086826517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 msgr2=0x7fa6b819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 0x7fa6b819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 -- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b80ff910 msgr2=0x7fa6b8195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:15.699 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b80ff910 0x7fa6b8195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:15.700 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 -- 172.21.15.187:0/2086826517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80095f0 con 0x7fa6b8100b10 2023-08-11T13:59:15.700 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6b67fc700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8003790 tx=0x7fa6a800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:59:15.700 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a801d070 con 0x7fa6b8100b10 2023-08-11T13:59:15.700 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a800be70 con 0x7fa6b8100b10 2023-08-11T13:59:15.700 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b819d730 con 0x7fa6b8100b10 2023-08-11T13:59:15.701 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80176e0 con 0x7fa6b8100b10 2023-08-11T13:59:15.701 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.685+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b819dbc0 con 0x7fa6b8100b10 2023-08-11T13:59:15.701 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.686+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b810b1b0 con 0x7fa6b8100b10 2023-08-11T13:59:15.701 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.687+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa6a8017840 con 0x7fa6b8100b10 2023-08-11T13:59:15.701 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.688+0000 7fa69ffff700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 0x7fa6a00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:15.702 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.688+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fa6a80cbc40 con 0x7fa6b8100b10 2023-08-11T13:59:15.702 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.688+0000 7fa6b6ffd700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 0x7fa6a00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:15.702 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.689+0000 7fa6b6ffd700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 0x7fa6a00a2410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8100970 tx=0x7fa6ac006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:59:15.703 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.692+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa6a805cc30 con 0x7fa6b8100b10 2023-08-11T13:59:15.890 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:15.889+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6b8060d60 con 0x7fa6a009ff50 2023-08-11T13:59:16.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:16 smithi003 ceph-mon[97195]: osdmap e51: 8 total, 8 up, 8 in 2023-08-11T13:59:16.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:16.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:16.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:16 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:16.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:16 smithi187 ceph-mon[130973]: osdmap e51: 8 total, 8 up, 8 in 2023-08-11T13:59:16.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:16.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:16.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:16 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:16.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:16 smithi103 ceph-mon[130434]: osdmap e51: 8 total, 8 up, 8 in 2023-08-11T13:59:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:16.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:16 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:17 smithi187 ceph-mon[130973]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:17 smithi187 ceph-mon[130973]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:17 smithi187 ceph-mon[130973]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T13:59:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:17 smithi187 ceph-mon[130973]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T13:59:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:17 smithi003 ceph-mon[97195]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:17 smithi003 ceph-mon[97195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:17 smithi003 ceph-mon[97195]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T13:59:17.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:17 smithi003 ceph-mon[97195]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T13:59:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:17 smithi103 ceph-mon[130434]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:17 smithi103 ceph-mon[130434]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:17 smithi103 ceph-mon[130434]: osdmap e52: 8 total, 8 up, 8 in 2023-08-11T13:59:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:17 smithi103 ceph-mon[130434]: osdmap e53: 8 total, 8 up, 8 in 2023-08-11T13:59:19.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:19.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T13:59:19.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/2145545498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]': finished 2023-08-11T13:59:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: osdmap e55: 9 total, 8 up, 9 in 2023-08-11T13:59:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:19 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/3918618325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/2145545498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]': finished 2023-08-11T13:59:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: osdmap e55: 9 total, 8 up, 9 in 2023-08-11T13:59:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:19 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/3918618325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: osdmap e54: 8 total, 8 up, 8 in 2023-08-11T13:59:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/2145545498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]: dispatch 2023-08-11T13:59:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "295943c9-dfde-47cd-96f0-570e5e989919"}]': finished 2023-08-11T13:59:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: osdmap e55: 9 total, 8 up, 9 in 2023-08-11T13:59:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:19 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/3918618325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:21.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:21 smithi003 ceph-mon[97195]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:21.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:21 smithi187 ceph-mon[130973]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:21 smithi103 ceph-mon[130434]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:23.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:23 smithi187 ceph-mon[130973]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:23.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:23 smithi003 ceph-mon[97195]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:23 smithi103 ceph-mon[130434]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:25.501 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:25 smithi187 ceph-mon[130973]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:25.501 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T13:59:25.501 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:25.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:25 smithi003 ceph-mon[97195]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:25.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T13:59:25.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:25 smithi103 ceph-mon[130434]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-08-11T13:59:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:26.432 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:26 smithi187 ceph-mon[130973]: Deploying daemon osd.8 on smithi187 2023-08-11T13:59:26.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:26 smithi003 ceph-mon[97195]: Deploying daemon osd.8 on smithi187 2023-08-11T13:59:26.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:26 smithi103 ceph-mon[130434]: Deploying daemon osd.8 on smithi187 2023-08-11T13:59:27.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:27 smithi003 ceph-mon[97195]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:27.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:27 smithi187 ceph-mon[130973]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:27 smithi103 ceph-mon[130434]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:29.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:29 smithi187 ceph-mon[130973]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:29.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:29 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:29 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:29 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:29 smithi003 ceph-mon[97195]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:29 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:29 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:29 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:29 smithi103 ceph-mon[130434]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:29 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:29 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:29.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:29 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:31 smithi003 ceph-mon[97195]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:31 smithi187 ceph-mon[130973]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:31.659 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 8 on host 'smithi187' 2023-08-11T13:59:31.659 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.657+0000 7fa69ffff700 1 -- 172.21.15.187:0/2086826517 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6b8060d60 con 0x7fa6a009ff50 2023-08-11T13:59:31.661 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 msgr2=0x7fa6a00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:31.661 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 0x7fa6a00a2410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8100970 tx=0x7fa6ac006d20 comp rx=0 tx=0).stop 2023-08-11T13:59:31.662 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 msgr2=0x7fa6b8195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:31.662 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8003790 tx=0x7fa6a800fa00 comp rx=0 tx=0).stop 2023-08-11T13:59:31.662 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 shutdown_connections 2023-08-11T13:59:31.662 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6a009ff50 0x7fa6a00a2410 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:31.662 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6b81014b0 0x7fa6b819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b80ff910 0x7fa6b8195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 --2- 172.21.15.187:0/2086826517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6b8100b10 0x7fa6b8195b90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:31.663 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 >> 172.21.15.187:0/2086826517 conn(0x7fa6b80faeb0 msgr2=0x7fa6b80fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:59:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 shutdown_connections 2023-08-11T13:59:31.664 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T13:59:31.660+0000 7fa6bd32c700 1 -- 172.21.15.187:0/2086826517 wait complete. 2023-08-11T13:59:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:31 smithi103 ceph-mon[130434]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:32.085 DEBUG:teuthology.orchestra.run.smithi187:osd.8> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.8.service 2023-08-11T13:59:32.088 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2023-08-11T13:59:32.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T13:59:32.118 INFO:journalctl@ceph.osd.8.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T13:59:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:32 smithi103 ceph-mon[130434]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:32.733 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.734 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.734 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:32.734 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:32.734 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:32.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-mon[130973]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:32.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:32.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:32.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:32 smithi003 ceph-mon[97195]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:59:33.044 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 13:59:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T13:59:32.979+0000 7f0649e40640 -1 osd.8 0 log_to_monitors true 2023-08-11T13:59:33.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:33 smithi003 ceph-mon[97195]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:33.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:33 smithi003 ceph-mon[97195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:33.880 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:33 smithi187 ceph-mon[130973]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:33.881 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:33 smithi187 ceph-mon[130973]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:33.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:33 smithi103 ceph-mon[130434]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:33.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:33 smithi103 ceph-mon[130434]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-08-11T13:59:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T13:59:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T13:59:34.838 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2023-08-11T13:59:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2023-08-11T13:59:34.838 INFO:teuthology.orchestra.run.smithi003.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0297648 s, 352 MB/s 2023-08-11T13:59:34.839 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 2023-08-11T13:59:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2023-08-11T13:59:34.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T13:59:34.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T13:59:34.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:34.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:34.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi187 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 5722M 2023-08-11T13:59:34.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:34.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:34.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T13:59:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T13:59:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi187 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:34.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 5722M 2023-08-11T13:59:34.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:34.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:34.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:34.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:35.044 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T13:59:34.627+0000 7f063f4b8700 -1 osd.8 0 waiting for initial osdmap 2023-08-11T13:59:35.044 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T13:59:34.637+0000 7f06392a9700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: osdmap e56: 9 total, 8 up, 9 in 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:35.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi187 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 5722M 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:35.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:35.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:35.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2023-08-11T13:59:35.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:35 smithi103 ceph-mon[130434]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:59:35.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:35 smithi103 ceph-mon[130434]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T13:59:35.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:35.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:35 smithi003 ceph-mon[97195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:59:36.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:35 smithi003 ceph-mon[97195]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T13:59:36.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:35 smithi187 ceph-mon[130973]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T13:59:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:35 smithi187 ceph-mon[130973]: osdmap e57: 9 total, 8 up, 9 in 2023-08-11T13:59:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.045 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T13:59:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T13:59:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T13:59:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] boot 2023-08-11T13:59:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: osdmap e58: 9 total, 9 up, 9 in 2023-08-11T13:59:36.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:36.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:36.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:36 smithi103 ceph-mon[130434]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T13:59:37.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T13:59:37.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T13:59:37.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] boot 2023-08-11T13:59:37.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: osdmap e58: 9 total, 9 up, 9 in 2023-08-11T13:59:37.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:37.040 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:37.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:36 smithi003 ceph-mon[97195]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: osd.8 [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] boot 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: osdmap e58: 9 total, 9 up, 9 in 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-08-11T13:59:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-08-11T13:59:37.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:36 smithi187 ceph-mon[130973]: osdmap e59: 9 total, 9 up, 9 in 2023-08-11T13:59:37.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 -- 172.21.15.3:0/1284961698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc1009b0 msgr2=0x7fddfc100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:37.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 --2- 172.21.15.3:0/1284961698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc1009b0 0x7fddfc100dd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fddec0099c0 tx=0x7fddec009cd0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 -- 172.21.15.3:0/1284961698 shutdown_connections 2023-08-11T13:59:37.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 --2- 172.21.15.3:0/1284961698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 --2- 172.21.15.3:0/1284961698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc1009b0 0x7fddfc100dd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 --2- 172.21.15.3:0/1284961698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc101bb0 0x7fddfc102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 -- 172.21.15.3:0/1284961698 >> 172.21.15.3:0/1284961698 conn(0x7fddfc0fbf10 msgr2=0x7fddfc0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:59:37.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 -- 172.21.15.3:0/1284961698 shutdown_connections 2023-08-11T13:59:37.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.709+0000 7fde0107d700 1 -- 172.21.15.3:0/1284961698 wait complete. 2023-08-11T13:59:37.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.710+0000 7fde0107d700 1 Processor -- start 2023-08-11T13:59:37.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.710+0000 7fde0107d700 1 -- start start 2023-08-11T13:59:37.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.710+0000 7fde0107d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc1009b0 0x7fddfc196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.710+0000 7fde0107d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.710+0000 7fde0107d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fde0107d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc19e1e0 con 0x7fddfc101bb0 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fde0107d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc19e320 con 0x7fddfc102550 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fde0107d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc19e4b0 con 0x7fddfc1009b0 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:37.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56138/0 (socket says 172.21.15.3:56138) 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 -- 172.21.15.3:0/2153277123 learned_addr learned my addr 172.21.15.3:0/2153277123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfad9d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc1009b0 0x7fddfc196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfb59e700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 -- 172.21.15.3:0/2153277123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc1009b0 msgr2=0x7fddfc196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc1009b0 0x7fddfc196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 -- 172.21.15.3:0/2153277123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 msgr2=0x7fddfc19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 -- 172.21.15.3:0/2153277123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0095f0 con 0x7fddfc101bb0 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfb59e700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.711+0000 7fddfa59c700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d980 tx=0x7fddf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.712+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0009b40 con 0x7fddfc101bb0 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.712+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc072bb0 con 0x7fddfc101bb0 2023-08-11T13:59:37.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.712+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddf000b6e0 con 0x7fddfc101bb0 2023-08-11T13:59:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.712+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000f620 con 0x7fddfc101bb0 2023-08-11T13:59:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.712+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc073100 con 0x7fddfc101bb0 2023-08-11T13:59:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.713+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc107ff0 con 0x7fddfc101bb0 2023-08-11T13:59:37.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.716+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fddf0010460 con 0x7fddfc101bb0 2023-08-11T13:59:37.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.718+0000 7fdde3fff700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 0x7fdde40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T13:59:37.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.718+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fddf00cad70 con 0x7fddfc101bb0 2023-08-11T13:59:37.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.718+0000 7fddfad9d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 0x7fdde40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T13:59:37.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.719+0000 7fddfad9d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 0x7fdde40a2410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddec009990 tx=0x7fddec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T13:59:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.720+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fddf00953f0 con 0x7fddfc101bb0 2023-08-11T13:59:37.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:37.936+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fddfc02cd40 con 0x7fdde409ff50 2023-08-11T13:59:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T13:59:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:38.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:38.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:38.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:38.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:38 smithi103 ceph-mon[130434]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:39.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T13:59:39.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:38 smithi003 ceph-mon[97195]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:39.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: osdmap e60: 9 total, 9 up, 9 in 2023-08-11T13:59:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T13:59:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T13:59:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T13:59:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:38 smithi187 ceph-mon[130973]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:39.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:39 smithi103 ceph-mon[130434]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T13:59:40.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:39 smithi003 ceph-mon[97195]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T13:59:40.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:39 smithi187 ceph-mon[130973]: osdmap e61: 9 total, 9 up, 9 in 2023-08-11T13:59:40.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:40 smithi103 ceph-mon[130434]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T13:59:40.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:40 smithi103 ceph-mon[130434]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:41.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:40 smithi003 ceph-mon[97195]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T13:59:41.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:40 smithi003 ceph-mon[97195]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:41.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:40 smithi187 ceph-mon[130973]: osdmap e62: 9 total, 9 up, 9 in 2023-08-11T13:59:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:40 smithi187 ceph-mon[130973]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:41.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]: dispatch 2023-08-11T13:59:41.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]': finished 2023-08-11T13:59:41.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:41 smithi103 ceph-mon[130434]: osdmap e63: 10 total, 9 up, 10 in 2023-08-11T13:59:41.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T13:59:42.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]: dispatch 2023-08-11T13:59:42.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]': finished 2023-08-11T13:59:42.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:41 smithi003 ceph-mon[97195]: osdmap e63: 10 total, 9 up, 10 in 2023-08-11T13:59:42.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T13:59:42.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]: dispatch 2023-08-11T13:59:42.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/871841037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cd878b1-5843-4397-a987-2437787cd241"}]': finished 2023-08-11T13:59:42.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:41 smithi187 ceph-mon[130973]: osdmap e63: 10 total, 9 up, 10 in 2023-08-11T13:59:42.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T13:59:42.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:42 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2391685082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:42.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:42 smithi103 ceph-mon[130434]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-08-11T13:59:43.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:42 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2391685082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:43.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:42 smithi003 ceph-mon[97195]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-08-11T13:59:43.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:42 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2391685082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T13:59:43.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:42 smithi187 ceph-mon[130973]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2023-08-11T13:59:45.516 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:45 smithi003 ceph-mon[97195]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:59:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:45 smithi187 ceph-mon[130973]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:59:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:45 smithi103 ceph-mon[130434]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T13:59:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:47 smithi003 ceph-mon[97195]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-11T13:59:47.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:47 smithi187 ceph-mon[130973]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-11T13:59:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:47 smithi103 ceph-mon[130434]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-08-11T13:59:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:49 smithi003 ceph-mon[97195]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-11T13:59:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T13:59:49.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:49.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:49 smithi187 ceph-mon[130973]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-11T13:59:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T13:59:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:49 smithi103 ceph-mon[130434]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-08-11T13:59:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:49 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-08-11T13:59:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:49 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:50.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:50 smithi003 ceph-mon[97195]: Deploying daemon osd.9 on smithi003 2023-08-11T13:59:50.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:50 smithi187 ceph-mon[130973]: Deploying daemon osd.9 on smithi003 2023-08-11T13:59:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:50 smithi103 ceph-mon[130434]: Deploying daemon osd.9 on smithi003 2023-08-11T13:59:51.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:51 smithi003 ceph-mon[97195]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:59:51.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:51 smithi187 ceph-mon[130973]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:59:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:51 smithi103 ceph-mon[130434]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-08-11T13:59:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:53 smithi003 ceph-mon[97195]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:53.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:53 smithi187 ceph-mon[130973]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:53 smithi103 ceph-mon[130434]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:55 smithi003 ceph-mon[97195]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:55.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:55 smithi187 ceph-mon[130973]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:55 smithi103 ceph-mon[130434]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:56.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:56 smithi103 ceph-mon[130434]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:56.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:56.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:56 smithi003 ceph-mon[97195]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:56 smithi187 ceph-mon[130973]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:58.954 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2023-08-11T13:59:58.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.952+0000 7fdde3fff700 1 -- 172.21.15.3:0/2153277123 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fddfc02cd40 con 0x7fdde409ff50 2023-08-11T13:59:58.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 msgr2=0x7fdde40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:58.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 0x7fdde40a2410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fddec009990 tx=0x7fddec005fb0 comp rx=0 tx=0).stop 2023-08-11T13:59:58.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 msgr2=0x7fddfc196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T13:59:58.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d980 tx=0x7fddf000dc90 comp rx=0 tx=0).stop 2023-08-11T13:59:58.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 shutdown_connections 2023-08-11T13:59:58.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdde409ff50 0x7fdde40a2410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:58.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddfc102550 0x7fddfc19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:58.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddfc1009b0 0x7fddfc196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:58.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.955+0000 7fde0107d700 1 --2- 172.21.15.3:0/2153277123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fddfc101bb0 0x7fddfc196b90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T13:59:58.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.956+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 >> 172.21.15.3:0/2153277123 conn(0x7fddfc0fbf10 msgr2=0x7fddfc105130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T13:59:58.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.956+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 shutdown_connections 2023-08-11T13:59:58.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T13:59:58.956+0000 7fde0107d700 1 -- 172.21.15.3:0/2153277123 wait complete. 2023-08-11T13:59:59.470 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:59.471 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.471 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.471 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:59.471 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:59.471 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 13:59:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:59.473 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.9.service 2023-08-11T13:59:59.477 INFO:tasks.cephadm:Deploying osd.10 on smithi103 with /dev/vg_nvme/lv_1... 2023-08-11T13:59:59.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T13:59:59.510 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:59.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:59.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:59.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 13:59:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T13:59:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T13:59:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T13:59:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T13:59:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 13:59:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T13:59:59.556 INFO:journalctl@ceph.osd.9.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T14:00:00.042 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:00:00.413 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:00 smithi103 ceph-mon[130434]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.413 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:00 smithi103 ceph-mon[130434]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.413 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:00 smithi103 ceph-mon[130434]: overall HEALTH_OK 2023-08-11T14:00:00.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:00 smithi003 ceph-mon[97195]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:00 smithi003 ceph-mon[97195]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:00 smithi003 ceph-mon[97195]: overall HEALTH_OK 2023-08-11T14:00:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:00 smithi187 ceph-mon[130973]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:00 smithi187 ceph-mon[130973]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-08-11T14:00:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:00 smithi187 ceph-mon[130973]: overall HEALTH_OK 2023-08-11T14:00:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T14:00:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T14:00:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:01 smithi103 ceph-mon[130434]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.523 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T14:00:01.523 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T14:00:01.523 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2023-08-11T14:00:01.524 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2023-08-11T14:00:01.524 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127183 s, 824 MB/s 2023-08-11T14:00:01.524 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 2023-08-11T14:00:01.524 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2023-08-11T14:00:01.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:00:01.232+0000 7fdc022fb700 -1 osd.9 0 waiting for initial osdmap 2023-08-11T14:00:01.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:00:01.247+0000 7fdbfb8eb700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:01 smithi003 ceph-mon[97195]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-08-11T14:00:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: osdmap e64: 10 total, 9 up, 10 in 2023-08-11T14:00:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: from='osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:01 smithi187 ceph-mon[130973]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2023-08-11T14:00:01.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2023-08-11T14:00:02.492 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:00:02.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:02 smithi003 ceph-mon[97195]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T14:00:02.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:02 smithi003 ceph-mon[97195]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T14:00:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:02.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:02 smithi187 ceph-mon[130973]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T14:00:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:02 smithi187 ceph-mon[130973]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T14:00:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:02 smithi103 ceph-mon[130434]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2023-08-11T14:00:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:02 smithi103 ceph-mon[130434]: osdmap e65: 10 total, 9 up, 10 in 2023-08-11T14:00:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:03.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T14:00:03.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T14:00:03.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:03.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] boot 2023-08-11T14:00:03.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: osdmap e66: 10 total, 10 up, 10 in 2023-08-11T14:00:03.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:03.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:03.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:03.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] boot 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: osdmap e66: 10 total, 10 up, 10 in 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T14:00:03.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: osd.9 [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] boot 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: osdmap e66: 10 total, 10 up, 10 in 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:03.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 -- 172.21.15.103:0/2875066319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c0ff8b0 msgr2=0x7f1d6c0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2875066319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c0ff8b0 0x7f1d6c0ffcd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0099c0 tx=0x7f1d5c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 -- 172.21.15.103:0/2875066319 shutdown_connections 2023-08-11T14:00:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2875066319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c101450 0x7f1d6c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2875066319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c0ff8b0 0x7f1d6c0ffcd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2875066319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 -- 172.21.15.103:0/2875066319 >> 172.21.15.103:0/2875066319 conn(0x7f1d6c0fae30 msgr2=0x7f1d6c0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 -- 172.21.15.103:0/2875066319 shutdown_connections 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 -- 172.21.15.103:0/2875066319 wait complete. 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.764+0000 7f1d73b62700 1 Processor -- start 2023-08-11T14:00:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 -- start start 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d6c0ff8b0 0x7f1d6c068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c101450 0x7f1d6c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1a03e0 con 0x7f1d6c0ff8b0 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1a0550 con 0x7f1d6c100ab0 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1a0710 con 0x7f1d6c101450 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:03.769 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58986/0 (socket says 172.21.15.103:58986) 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 -- 172.21.15.103:0/2743758033 learned_addr learned my addr 172.21.15.103:0/2743758033 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 -- 172.21.15.103:0/2743758033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c101450 msgr2=0x7f1d6c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c101450 0x7f1d6c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 -- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d6c0ff8b0 msgr2=0x7f1d6c068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d6c0ff8b0 0x7f1d6c068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 -- 172.21.15.103:0/2743758033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c0095f0 con 0x7f1d6c100ab0 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d710fd700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b530 tx=0x7f1d6800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68011830 con 0x7f1d6c100ab0 2023-08-11T14:00:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c1a09f0 con 0x7f1d6c100ab0 2023-08-11T14:00:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d68011e50 con 0x7f1d6c100ab0 2023-08-11T14:00:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.765+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800f2e0 con 0x7f1d6c100ab0 2023-08-11T14:00:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.766+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c1a0f40 con 0x7f1d6c100ab0 2023-08-11T14:00:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.766+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c106ea0 con 0x7f1d6c100ab0 2023-08-11T14:00:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.768+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1d68010580 con 0x7f1d6c100ab0 2023-08-11T14:00:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.768+0000 7f1d62ffd700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 0x7f1d580a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.769+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1d680cab80 con 0x7f1d6c100ab0 2023-08-11T14:00:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.769+0000 7f1d718fe700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 0x7f1d580a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.769+0000 7f1d718fe700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 0x7f1d580a2490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0036f0 tx=0x7f1d5c009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.771+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1d68095070 con 0x7f1d6c100ab0 2023-08-11T14:00:03.967 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:03.966+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d6c02ce30 con 0x7f1d5809ffd0 2023-08-11T14:00:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi003 2023-08-11T14:00:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 4318M 2023-08-11T14:00:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T14:00:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:04.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi003 2023-08-11T14:00:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 4318M 2023-08-11T14:00:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T14:00:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi003 2023-08-11T14:00:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 4318M 2023-08-11T14:00:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: osdmap e67: 10 total, 10 up, 10 in 2023-08-11T14:00:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:05.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:05 smithi103 ceph-mon[130434]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:05.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:05 smithi103 ceph-mon[130434]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:05 smithi003 ceph-mon[97195]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:05 smithi003 ceph-mon[97195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:05 smithi187 ceph-mon[130973]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:05.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:05 smithi187 ceph-mon[130973]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/1678984062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]': finished 2023-08-11T14:00:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: osdmap e68: 11 total, 10 up, 11 in 2023-08-11T14:00:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:07 smithi003 ceph-mon[97195]: from='client.? 172.21.15.103:0/4179695490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:07.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/1678984062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]': finished 2023-08-11T14:00:07.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: osdmap e68: 11 total, 10 up, 11 in 2023-08-11T14:00:07.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:07.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:07 smithi187 ceph-mon[130973]: from='client.? 172.21.15.103:0/4179695490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/1678984062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]: dispatch 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51595dfa-2865-4bde-985a-b538b1d68644"}]': finished 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: osdmap e68: 11 total, 10 up, 11 in 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:07.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:07 smithi103 ceph-mon[130434]: from='client.? 172.21.15.103:0/4179695490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:09.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:09 smithi003 ceph-mon[97195]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:09.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:09 smithi187 ceph-mon[130973]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:09 smithi103 ceph-mon[130434]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:11.501 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:11 smithi103 ceph-mon[130434]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:11 smithi003 ceph-mon[97195]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:11.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:11 smithi187 ceph-mon[130973]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:13 smithi003 ceph-mon[97195]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T14:00:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:13.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:13 smithi187 ceph-mon[130973]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:13.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T14:00:13.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:13.556 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:13 smithi103 ceph-mon[130434]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:13.557 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-08-11T14:00:13.557 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:14.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:14 smithi103 ceph-mon[130434]: Deploying daemon osd.10 on smithi103 2023-08-11T14:00:14.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:14 smithi003 ceph-mon[97195]: Deploying daemon osd.10 on smithi103 2023-08-11T14:00:14.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:14 smithi187 ceph-mon[130973]: Deploying daemon osd.10 on smithi103 2023-08-11T14:00:15.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:15 smithi003 ceph-mon[97195]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:15.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:15 smithi187 ceph-mon[130973]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:15.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:15 smithi103 ceph-mon[130434]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:17.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:17 smithi103 ceph-mon[130434]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:17.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:17 smithi187 ceph-mon[130973]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:17 smithi003 ceph-mon[97195]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:18.534 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.534 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:18.534 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:18 smithi103 ceph-mon[130434]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:18.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:18.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:18 smithi003 ceph-mon[97195]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:18.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:18.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:18.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:18 smithi187 ceph-mon[130973]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.130+0000 7f1d62ffd700 1 -- 172.21.15.103:0/2743758033 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1d6c02ce30 con 0x7f1d5809ffd0 2023-08-11T14:00:20.132 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 10 on host 'smithi103' 2023-08-11T14:00:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 msgr2=0x7f1d580a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 0x7f1d580a2490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0036f0 tx=0x7f1d5c009f10 comp rx=0 tx=0).stop 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 msgr2=0x7f1d6c19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b530 tx=0x7f1d6800b8f0 comp rx=0 tx=0).stop 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 shutdown_connections 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d6c101450 0x7f1d6c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d6c0ff8b0 0x7f1d6c068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d5809ffd0 0x7f1d580a2490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 --2- 172.21.15.103:0/2743758033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c100ab0 0x7f1d6c19d7e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 >> 172.21.15.103:0/2743758033 conn(0x7f1d6c0fae30 msgr2=0x7f1d6c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 shutdown_connections 2023-08-11T14:00:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2023-08-11T14:00:20.132+0000 7f1d73b62700 1 -- 172.21.15.103:0/2743758033 wait complete. 2023-08-11T14:00:20.581 DEBUG:teuthology.orchestra.run.smithi103:osd.10> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.10.service 2023-08-11T14:00:20.583 INFO:tasks.cephadm:Deploying osd.11 on smithi187 with /dev/vg_nvme/lv_1... 2023-08-11T14:00:20.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-08-11T14:00:20.613 INFO:journalctl@ceph.osd.10.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T14:00:21.147 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:00:21.366 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.367 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.367 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:21 smithi187 ceph-mon[130973]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:21.367 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.367 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:21 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-mon[130434]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:21.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:21 smithi003 ceph-mon[97195]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:21.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:21 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:21.726 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:00:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:00:21.434+0000 7fba203e1640 -1 osd.10 0 log_to_monitors true 2023-08-11T14:00:22.162 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:22 smithi187 ceph-mon[130973]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.164 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:22 smithi187 ceph-mon[130973]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:22 smithi103 ceph-mon[130434]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.299 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:22 smithi103 ceph-mon[130434]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:22 smithi003 ceph-mon[97195]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:22 smithi003 ceph-mon[97195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-08-11T14:00:22.638 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-08-11T14:00:22.639 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-08-11T14:00:22.639 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-08-11T14:00:22.639 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-08-11T14:00:22.639 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113956 s, 920 MB/s 2023-08-11T14:00:22.640 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-08-11T14:00:23.004 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch daemon add osd smithi187:vg_nvme/lv_1 2023-08-11T14:00:23.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T14:00:23.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T14:00:23.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:23.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:23 smithi187 ceph-mon[130973]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:23.428 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:00:23.121+0000 7fba15a59700 -1 osd.10 0 waiting for initial osdmap 2023-08-11T14:00:23.428 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:00:23.132+0000 7fba1084c700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T14:00:23.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T14:00:23.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T14:00:23.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:23.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:23 smithi103 ceph-mon[130434]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-08-11T14:00:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: osdmap e69: 11 total, 10 up, 11 in 2023-08-11T14:00:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: from='osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2023-08-11T14:00:23.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:23 smithi003 ceph-mon[97195]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-08-11T14:00:23.561 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:00:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T14:00:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T14:00:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi103 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 4292M 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:24.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:24 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T14:00:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T14:00:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi103 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 4292M 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:24.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:24 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2023-08-11T14:00:24.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: osdmap e70: 11 total, 10 up, 11 in 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi103 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 4292M 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:24 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:24.743 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- 172.21.15.187:0/501124908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc01009f0 msgr2=0x7f8dc0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:24.743 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/501124908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc01009f0 0x7f8dc0100e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009990 tx=0x7f8db0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- 172.21.15.187:0/501124908 shutdown_connections 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/501124908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc0102590 0x7f8dc0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/501124908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc01009f0 0x7f8dc0100e10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/501124908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dc0101bf0 0x7f8dc0102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- 172.21.15.187:0/501124908 >> 172.21.15.187:0/501124908 conn(0x7f8dc00fbf90 msgr2=0x7f8dc00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- 172.21.15.187:0/501124908 shutdown_connections 2023-08-11T14:00:24.744 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- 172.21.15.187:0/501124908 wait complete. 2023-08-11T14:00:24.746 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 Processor -- start 2023-08-11T14:00:24.746 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.742+0000 7f8dc5a40700 1 -- start start 2023-08-11T14:00:24.746 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc01009f0 0x7f8dc00746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:24.746 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dc0101bf0 0x7f8dc0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00739d0 con 0x7f8dc0101bf0 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0073b40 con 0x7f8dc01009f0 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01a07a0 con 0x7f8dc0102590 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:55258/0 (socket says 172.21.15.187:55258) 2023-08-11T14:00:24.747 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 -- 172.21.15.187:0/3798573206 learned_addr learned my addr 172.21.15.187:0/3798573206 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbeffd700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc01009f0 0x7f8dc00746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 -- 172.21.15.187:0/3798573206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc01009f0 msgr2=0x7f8dc00746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc01009f0 0x7f8dc00746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 -- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dc0101bf0 msgr2=0x7f8dc0072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dc0101bf0 0x7f8dc0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 -- 172.21.15.187:0/3798573206 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00095f0 con 0x7f8dc0102590 2023-08-11T14:00:24.748 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dbf7fe700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007ee0 tx=0x7f8db800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8017070 con 0x7f8dc0102590 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db800f040 con 0x7f8dc0102590 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc01a0a80 con 0x7f8dc0102590 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8005740 con 0x7f8dc0102590 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.743+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc01a0fb0 con 0x7f8dc0102590 2023-08-11T14:00:24.749 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.744+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0107fe0 con 0x7f8dc0102590 2023-08-11T14:00:24.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.745+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f8db800dc60 con 0x7f8dc0102590 2023-08-11T14:00:24.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.746+0000 7f8dc4a3e700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 0x7f8da80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:24.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.746+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f8db80caa90 con 0x7f8dc0102590 2023-08-11T14:00:24.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.746+0000 7f8dbeffd700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 0x7f8da80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:24.750 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.747+0000 7f8dbeffd700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 0x7f8da80a2410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:24.751 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.749+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8db8094e50 con 0x7f8dc0102590 2023-08-11T14:00:24.951 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:24.950+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dc002ce00 con 0x7f8da809ff50 2023-08-11T14:00:25.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T14:00:25.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] boot 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: osdmap e71: 11 total, 11 up, 11 in 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:25.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T14:00:25.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T14:00:25.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] boot 2023-08-11T14:00:25.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: osdmap e71: 11 total, 11 up, 11 in 2023-08-11T14:00:25.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:25.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:25.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:25.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:25.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:25.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T14:00:25.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T14:00:25.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: osd.10 [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] boot 2023-08-11T14:00:25.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: osdmap e71: 11 total, 11 up, 11 in 2023-08-11T14:00:25.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-08-11T14:00:25.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:25.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-08-11T14:00:25.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-08-11T14:00:25.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:26.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:26 smithi187 ceph-mon[130973]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:26.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:26 smithi187 ceph-mon[130973]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T14:00:26.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:26 smithi103 ceph-mon[130434]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:26.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:26 smithi103 ceph-mon[130434]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T14:00:26.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:26 smithi003 ceph-mon[97195]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:26.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:26 smithi003 ceph-mon[97195]: osdmap e72: 11 total, 11 up, 11 in 2023-08-11T14:00:27.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:27 smithi103 ceph-mon[130434]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T14:00:27.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:27 smithi103 ceph-mon[130434]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:27.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:27 smithi003 ceph-mon[97195]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T14:00:27.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:27 smithi003 ceph-mon[97195]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:27.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:27 smithi187 ceph-mon[130973]: osdmap e73: 11 total, 11 up, 11 in 2023-08-11T14:00:27.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:27 smithi187 ceph-mon[130973]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T14:00:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/1064765805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]': finished 2023-08-11T14:00:28.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: osdmap e75: 12 total, 11 up, 12 in 2023-08-11T14:00:28.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:28.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:28 smithi103 ceph-mon[130434]: from='client.? 172.21.15.187:0/187159469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:28.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/1064765805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]': finished 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: osdmap e75: 12 total, 11 up, 12 in 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:28 smithi003 ceph-mon[97195]: from='client.? 172.21.15.187:0/187159469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:28.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: osdmap e74: 11 total, 11 up, 11 in 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/1064765805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]: dispatch 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3738e6b-7935-4522-89c7-91cd190ab5e7"}]': finished 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: osdmap e75: 12 total, 11 up, 12 in 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:28 smithi187 ceph-mon[130973]: from='client.? 172.21.15.187:0/187159469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-08-11T14:00:29.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:29 smithi103 ceph-mon[130434]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:29 smithi003 ceph-mon[97195]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:29.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:29 smithi187 ceph-mon[130973]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:31.360 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:31 smithi187 ceph-mon[130973]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:31.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:31 smithi103 ceph-mon[130434]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:31 smithi003 ceph-mon[97195]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:33.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:33 smithi187 ceph-mon[130973]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T14:00:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:33 smithi003 ceph-mon[97195]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T14:00:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:33 smithi103 ceph-mon[130434]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-08-11T14:00:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T14:00:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:34.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T14:00:34.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-08-11T14:00:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:35.507 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:35 smithi187 ceph-mon[130973]: Deploying daemon osd.11 on smithi187 2023-08-11T14:00:35.507 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:35 smithi187 ceph-mon[130973]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T14:00:35.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:35 smithi003 ceph-mon[97195]: Deploying daemon osd.11 on smithi187 2023-08-11T14:00:35.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:35 smithi003 ceph-mon[97195]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T14:00:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:35 smithi103 ceph-mon[130434]: Deploying daemon osd.11 on smithi187 2023-08-11T14:00:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:35 smithi103 ceph-mon[130434]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-08-11T14:00:37.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:37 smithi187 ceph-mon[130973]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-08-11T14:00:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:37 smithi003 ceph-mon[97195]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-08-11T14:00:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:37 smithi103 ceph-mon[130434]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-08-11T14:00:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:38.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:38.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:38.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:38.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:38.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:39.419 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:39 smithi187 ceph-mon[130973]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-08-11T14:00:39.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:39 smithi003 ceph-mon[97195]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-08-11T14:00:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:39 smithi103 ceph-mon[130434]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-08-11T14:00:40.568 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 11 on host 'smithi187' 2023-08-11T14:00:40.568 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.566+0000 7f8dc4a3e700 1 -- 172.21.15.187:0/3798573206 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8dc002ce00 con 0x7f8da809ff50 2023-08-11T14:00:40.570 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.568+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 msgr2=0x7f8da80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:40.570 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.568+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 0x7f8da80a2410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000c00 tx=0x7f8db0005fb0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 msgr2=0x7f8dc0073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:40.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007ee0 tx=0x7f8db800d3b0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 shutdown_connections 2023-08-11T14:00:40.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dc0102590 0x7f8dc0073250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8da809ff50 0x7f8da80a2410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dc01009f0 0x7f8dc00746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 --2- 172.21.15.187:0/3798573206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dc0101bf0 0x7f8dc0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 >> 172.21.15.187:0/3798573206 conn(0x7f8dc00fbf90 msgr2=0x7f8dc0105120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 shutdown_connections 2023-08-11T14:00:40.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:40.569+0000 7f8dc5a40700 1 -- 172.21.15.187:0/3798573206 wait complete. 2023-08-11T14:00:40.955 DEBUG:teuthology.orchestra.run.smithi187:osd.11> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.11.service 2023-08-11T14:00:40.958 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-08-11T14:00:40.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd stat -f json 2023-08-11T14:00:40.988 INFO:journalctl@ceph.osd.11.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T14:00:41.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:41 smithi003 ceph-mon[97195]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:41.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.529 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-mon[130973]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:41.529 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.529 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.529 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.530 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:41 smithi103 ceph-mon[130434]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:41.699 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:00:42.243 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:00:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:00:41.969+0000 7f4740170640 -1 osd.11 0 log_to_monitors true 2023-08-11T14:00:42.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:42 smithi003 ceph-mon[97195]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:42.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:42 smithi003 ceph-mon[97195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:42.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:42 smithi187 ceph-mon[130973]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:42.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:42 smithi187 ceph-mon[130973]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:42.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:42 smithi103 ceph-mon[130434]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:42.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:42 smithi103 ceph-mon[130434]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-08-11T14:00:43.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.268+0000 7efc21273700 1 -- 172.21.15.3:0/3979941038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c074950 msgr2=0x7efc1c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.268+0000 7efc21273700 1 --2- 172.21.15.3:0/3979941038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c074950 0x7efc1c072db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0098f0 tx=0x7efc0c009c00 comp rx=0 tx=0).stop 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 -- 172.21.15.3:0/3979941038 shutdown_connections 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 --2- 172.21.15.3:0/3979941038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c108880 0x7efc1c10ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 --2- 172.21.15.3:0/3979941038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c074950 0x7efc1c072db0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 --2- 172.21.15.3:0/3979941038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 -- 172.21.15.3:0/3979941038 >> 172.21.15.3:0/3979941038 conn(0x7efc1c0fbf10 msgr2=0x7efc1c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 -- 172.21.15.3:0/3979941038 shutdown_connections 2023-08-11T14:00:43.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.269+0000 7efc21273700 1 -- 172.21.15.3:0/3979941038 wait complete. 2023-08-11T14:00:43.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.270+0000 7efc21273700 1 Processor -- start 2023-08-11T14:00:43.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.270+0000 7efc21273700 1 -- start start 2023-08-11T14:00:43.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:43.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c074950 0x7efc1c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c108880 0x7efc1c19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c1a04e0 con 0x7efc1c108880 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c1a0620 con 0x7efc1c074950 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc21273700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c1a07b0 con 0x7efc1c073380 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51706/0 (socket says 172.21.15.3:51706) 2023-08-11T14:00:43.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 -- 172.21.15.3:0/2459192 learned_addr learned my addr 172.21.15.3:0/2459192 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1b7fe700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c108880 0x7efc1c19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1a7fc700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c074950 0x7efc1c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 -- 172.21.15.3:0/2459192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c074950 msgr2=0x7efc1c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c074950 0x7efc1c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 -- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c108880 msgr2=0x7efc1c19ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c108880 0x7efc1c19ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.271+0000 7efc1affd700 1 -- 172.21.15.3:0/2459192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100095f0 con 0x7efc1c073380 2023-08-11T14:00:43.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc1affd700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009230 tx=0x7efc0c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:43.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c01c070 con 0x7efc1c073380 2023-08-11T14:00:43.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc0c00f820 con 0x7efc1c073380 2023-08-11T14:00:43.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c009520 con 0x7efc1c073380 2023-08-11T14:00:43.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c003ce0 con 0x7efc1c073380 2023-08-11T14:00:43.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.272+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c1a0cb0 con 0x7efc1c073380 2023-08-11T14:00:43.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.273+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c106f30 con 0x7efc1c073380 2023-08-11T14:00:43.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.275+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efc0c005590 con 0x7efc1c073380 2023-08-11T14:00:43.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.276+0000 7efc03fff700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 0x7efc040a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:43.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.277+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6847+0+0 (secure 0 0 0) 0x7efc0c0cbb90 con 0x7efc1c073380 2023-08-11T14:00:43.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.277+0000 7efc1a7fc700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 0x7efc040a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:43.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.277+0000 7efc1a7fc700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 0x7efc040a2410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc1c074680 tx=0x7efc10009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:43.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.280+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7efc0c095ed0 con 0x7efc1c073380 2023-08-11T14:00:43.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.505+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc1c10e400 con 0x7efc1c073380 2023-08-11T14:00:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.505+0000 7efc03fff700 1 -- 172.21.15.3:0/2459192 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7efc0c0994f0 con 0x7efc1c073380 2023-08-11T14:00:43.508 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:00:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 msgr2=0x7efc040a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 0x7efc040a2410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc1c074680 tx=0x7efc10009310 comp rx=0 tx=0).stop 2023-08-11T14:00:43.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 msgr2=0x7efc1c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009230 tx=0x7efc0c00b790 comp rx=0 tx=0).stop 2023-08-11T14:00:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 shutdown_connections 2023-08-11T14:00:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc1c108880 0x7efc1c19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c073380 0x7efc1c198950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc0409ff50 0x7efc040a2410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 --2- 172.21.15.3:0/2459192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c074950 0x7efc1c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 >> 172.21.15.3:0/2459192 conn(0x7efc1c0fbf10 msgr2=0x7efc1c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 shutdown_connections 2023-08-11T14:00:43.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:43.508+0000 7efc21273700 1 -- 172.21.15.3:0/2459192 wait complete. 2023-08-11T14:00:43.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:43.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T14:00:43.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T14:00:43.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:43.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.535 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:43.535 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:43 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:43.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:00:43.260+0000 7f47357e8700 -1 osd.11 0 waiting for initial osdmap 2023-08-11T14:00:43.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:00:43.273+0000 7f472edd8700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-11T14:00:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:43.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:43 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-08-11T14:00:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-08-11T14:00:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: osdmap e76: 12 total, 11 up, 12 in 2023-08-11T14:00:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:00:43.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:43.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:43.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:43 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:43.979 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1691762424,"num_in_osds":12,"osd_in_since":1691762427,"num_remapped_pgs":0} 2023-08-11T14:00:44.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: Detected new or changed devices on smithi187 2023-08-11T14:00:44.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 4292M 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:44.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:44.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:44.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:44 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2459192' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:44.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: Detected new or changed devices on smithi187 2023-08-11T14:00:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 4292M 2023-08-11T14:00:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T14:00:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T14:00:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:44 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2459192' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: Detected new or changed devices on smithi187 2023-08-11T14:00:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 4292M 2023-08-11T14:00:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: osdmap e77: 12 total, 11 up, 12 in 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:44.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:44 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2459192' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:44.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd stat -f json 2023-08-11T14:00:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: purged_snaps scrub starts 2023-08-11T14:00:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: purged_snaps scrub ok 2023-08-11T14:00:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:45.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] boot 2023-08-11T14:00:45.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: osdmap e78: 12 total, 12 up, 12 in 2023-08-11T14:00:45.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:45 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: purged_snaps scrub starts 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: purged_snaps scrub ok 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] boot 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: osdmap e78: 12 total, 12 up, 12 in 2023-08-11T14:00:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:45 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: purged_snaps scrub starts 2023-08-11T14:00:45.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: purged_snaps scrub ok 2023-08-11T14:00:45.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-08-11T14:00:45.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: osd.11 [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] boot 2023-08-11T14:00:45.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: osdmap e78: 12 total, 12 up, 12 in 2023-08-11T14:00:45.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:45 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-08-11T14:00:45.726 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:00:46.584 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:46 smithi003 ceph-mon[97195]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T14:00:46.584 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:46 smithi003 ceph-mon[97195]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:46 smithi187 ceph-mon[130973]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T14:00:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:46 smithi187 ceph-mon[130973]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:46.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:46 smithi103 ceph-mon[130434]: osdmap e79: 12 total, 12 up, 12 in 2023-08-11T14:00:46.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:46 smithi103 ceph-mon[130434]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:47.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.263+0000 7fe4522eb700 1 -- 172.21.15.3:0/2100921404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c101bb0 msgr2=0x7fe44c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.263+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2100921404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c101bb0 0x7fe44c102010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4400099c0 tx=0x7fe440009cd0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 -- 172.21.15.3:0/2100921404 shutdown_connections 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2100921404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c102550 0x7fe44c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2100921404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c1009b0 0x7fe44c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2100921404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c101bb0 0x7fe44c102010 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 -- 172.21.15.3:0/2100921404 >> 172.21.15.3:0/2100921404 conn(0x7fe44c0fbf10 msgr2=0x7fe44c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:47.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 -- 172.21.15.3:0/2100921404 shutdown_connections 2023-08-11T14:00:47.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.264+0000 7fe4522eb700 1 -- 172.21.15.3:0/2100921404 wait complete. 2023-08-11T14:00:47.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 Processor -- start 2023-08-11T14:00:47.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 -- start start 2023-08-11T14:00:47.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c101bb0 0x7fe44c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c102550 0x7fe44c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19e2b0 con 0x7fe44c1009b0 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19e3f0 con 0x7fe44c102550 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe4522eb700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19e580 con 0x7fe44c101bb0 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe44bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:47.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe44bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54316/0 (socket says 172.21.15.3:54316) 2023-08-11T14:00:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe44bfff700 1 -- 172.21.15.3:0/2285358609 learned_addr learned my addr 172.21.15.3:0/2285358609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.265+0000 7fe44b7fe700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c101bb0 0x7fe44c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:47.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe450888700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c102550 0x7fe44c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 -- 172.21.15.3:0/2285358609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c101bb0 msgr2=0x7fe44c196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c101bb0 0x7fe44c196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 -- 172.21.15.3:0/2285358609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c102550 msgr2=0x7fe44c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c102550 0x7fe44c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 -- 172.21.15.3:0/2285358609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400095f0 con 0x7fe44c1009b0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe44bfff700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9a0 tx=0x7fe43c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c009790 con 0x7fe44c1009b0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe43c00c810 con 0x7fe44c1009b0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c19e860 con 0x7fe44c1009b0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.266+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c003c70 con 0x7fe44c1009b0 2023-08-11T14:00:47.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.267+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c19edb0 con 0x7fe44c1009b0 2023-08-11T14:00:47.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.268+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44c108050 con 0x7fe44c1009b0 2023-08-11T14:00:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.269+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe43c00cc80 con 0x7fe44c1009b0 2023-08-11T14:00:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.271+0000 7fe4497fa700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 0x7fe4340a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.271+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe43c014070 con 0x7fe44c1009b0 2023-08-11T14:00:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.271+0000 7fe44b7fe700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 0x7fe4340a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.272+0000 7fe44b7fe700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 0x7fe4340a2490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe44000b580 tx=0x7fe440005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.275+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe43c0950b0 con 0x7fe44c1009b0 2023-08-11T14:00:47.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.485+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe44c10c200 con 0x7fe44c1009b0 2023-08-11T14:00:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.486+0000 7fe4497fa700 1 -- 172.21.15.3:0/2285358609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fe43c0986d0 con 0x7fe44c1009b0 2023-08-11T14:00:47.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 msgr2=0x7fe4340a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 0x7fe4340a2490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe44000b580 tx=0x7fe440005fb0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 msgr2=0x7fe44c196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e9a0 tx=0x7fe43c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 shutdown_connections 2023-08-11T14:00:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe43409ffd0 0x7fe4340a2490 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe44c102550 0x7fe44c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe44c1009b0 0x7fe44c196720 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 --2- 172.21.15.3:0/2285358609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe44c101bb0 0x7fe44c196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 >> 172.21.15.3:0/2285358609 conn(0x7fe44c0fbf10 msgr2=0x7fe44c105190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 shutdown_connections 2023-08-11T14:00:47.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:47.488+0000 7fe4522eb700 1 -- 172.21.15.3:0/2285358609 wait complete. 2023-08-11T14:00:48.008 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1691762444,"num_in_osds":12,"osd_in_since":1691762427,"num_remapped_pgs":0} 2023-08-11T14:00:48.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T14:00:48.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:48 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2285358609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:48.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:48 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2285358609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:48.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:48 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2285358609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-08-11T14:00:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:00:49.282 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:49 smithi003 ceph-mon[97195]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:49.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:49 smithi187 ceph-mon[130973]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:49 smithi103 ceph-mon[130434]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 -- 172.21.15.3:0/3197410221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138106940 msgr2=0x7f7138106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 --2- 172.21.15.3:0/3197410221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138106940 0x7f7138106d20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f71280098f0 tx=0x7f7128009c00 comp rx=0 tx=0).stop 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 -- 172.21.15.3:0/3197410221 shutdown_connections 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 --2- 172.21.15.3:0/3197410221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138101300 0x7f713810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 --2- 172.21.15.3:0/3197410221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138106940 0x7f7138106d20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 --2- 172.21.15.3:0/3197410221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138100940 0x7f7138100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 -- 172.21.15.3:0/3197410221 >> 172.21.15.3:0/3197410221 conn(0x7f71380fc410 msgr2=0x7f71380fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 -- 172.21.15.3:0/3197410221 shutdown_connections 2023-08-11T14:00:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.311+0000 7f713cdec700 1 -- 172.21.15.3:0/3197410221 wait complete. 2023-08-11T14:00:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.312+0000 7f713cdec700 1 Processor -- start 2023-08-11T14:00:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 -- start start 2023-08-11T14:00:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101300 0x7f713819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 0x7f7138198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138199110 con 0x7f7138100940 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138199280 con 0x7f7138106940 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713cdec700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138199440 con 0x7f7138101300 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54336/0 (socket says 172.21.15.3:54336) 2023-08-11T14:00:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f7136d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 0x7f7138198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f713659c700 1 -- 172.21.15.3:0/4164314694 learned_addr learned my addr 172.21.15.3:0/4164314694 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.313+0000 7f7135d9b700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101300 0x7f713819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 -- 172.21.15.3:0/4164314694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101300 msgr2=0x7f713819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101300 0x7f713819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 -- 172.21.15.3:0/4164314694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 msgr2=0x7f7138198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 0x7f7138198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 -- 172.21.15.3:0/4164314694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0095f0 con 0x7f7138100940 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f7136d9d700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 0x7f7138198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713659c700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7128009230 tx=0x7f712800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712801c070 con 0x7f7138100940 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128009520 con 0x7f7138100940 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71280058e0 con 0x7f7138100940 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71280038e0 con 0x7f7138100940 2023-08-11T14:00:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.314+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381a5840 con 0x7f7138100940 2023-08-11T14:00:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.315+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713804e620 con 0x7f7138100940 2023-08-11T14:00:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.317+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7128020030 con 0x7f7138100940 2023-08-11T14:00:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.319+0000 7f71277fe700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 0x7f71200a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.319+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71280cbc10 con 0x7f7138100940 2023-08-11T14:00:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.319+0000 7f7135d9b700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 0x7f71200a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:50.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.320+0000 7f7135d9b700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 0x7f71200a2410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7138105f10 tx=0x7f712c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:50.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.323+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7128095e30 con 0x7f7138100940 2023-08-11T14:00:50.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.530+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71381109b0 con 0x7f7138100940 2023-08-11T14:00:50.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.531+0000 7f71277fe700 1 -- 172.21.15.3:0/4164314694 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18376 (secure 0 0 0) 0x7f7128099450 con 0x7f7138100940 2023-08-11T14:00:50.532 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:00:50.532 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:00:45.467682+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:00:50.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.533+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 msgr2=0x7f71200a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:50.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.533+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 0x7f71200a2410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7138105f10 tx=0x7f712c009310 comp rx=0 tx=0).stop 2023-08-11T14:00:50.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 msgr2=0x7f713819ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:50.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7128009230 tx=0x7f712800fa00 comp rx=0 tx=0).stop 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 shutdown_connections 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f712009ff50 0x7f71200a2410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7138106940 0x7f7138198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138100940 0x7f713819ea20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 --2- 172.21.15.3:0/4164314694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101300 0x7f713819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 >> 172.21.15.3:0/4164314694 conn(0x7f71380fc410 msgr2=0x7f713810bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 shutdown_connections 2023-08-11T14:00:50.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:50.534+0000 7f713cdec700 1 -- 172.21.15.3:0/4164314694 wait complete. 2023-08-11T14:00:51.024 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T14:00:51.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-08-11T14:00:51.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:51 smithi003 ceph-mon[97195]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:51.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4164314694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:51.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:51 smithi187 ceph-mon[130973]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:51.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4164314694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:51 smithi103 ceph-mon[130434]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4164314694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:51.758 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:00:53.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:53 smithi003 ceph-mon[97195]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 -- 172.21.15.3:0/4219928299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667410d180 msgr2=0x7f667410f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 --2- 172.21.15.3:0/4219928299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667410d180 0x7f667410f5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b0a0 tx=0x7f666c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 -- 172.21.15.3:0/4219928299 shutdown_connections 2023-08-11T14:00:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 --2- 172.21.15.3:0/4219928299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667410d180 0x7f667410f5f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 --2- 172.21.15.3:0/4219928299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674072d50 0x7f6674073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 --2- 172.21.15.3:0/4219928299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6674073670 0x7f667410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 -- 172.21.15.3:0/4219928299 >> 172.21.15.3:0/4219928299 conn(0x7f66740fc410 msgr2=0x7f66740fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 -- 172.21.15.3:0/4219928299 shutdown_connections 2023-08-11T14:00:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.326+0000 7f6678bb6700 1 -- 172.21.15.3:0/4219928299 wait complete. 2023-08-11T14:00:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.327+0000 7f6678bb6700 1 Processor -- start 2023-08-11T14:00:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.327+0000 7f6678bb6700 1 -- start start 2023-08-11T14:00:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674073670 0x7f66741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:53.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 0x7f667419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419dd40 con 0x7f6674072d50 2023-08-11T14:00:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419deb0 con 0x7f6674073670 2023-08-11T14:00:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6678bb6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419e070 con 0x7f667410d180 2023-08-11T14:00:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f667259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:53.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f667259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47186/0 (socket says 172.21.15.3:47186) 2023-08-11T14:00:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f667259c700 1 -- 172.21.15.3:0/1164918775 learned_addr learned my addr 172.21.15.3:0/1164918775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 0x7f667419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6671d9b700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674073670 0x7f66741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:53.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 -- 172.21.15.3:0/1164918775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674073670 msgr2=0x7f66741992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674073670 0x7f66741992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 -- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 msgr2=0x7f6674198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f6672d9d700 1 -- 172.21.15.3:0/1164918775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c009d00 con 0x7f667410d180 2023-08-11T14:00:53.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.328+0000 7f667259c700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:00:53.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f6672d9d700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 0x7f667419d730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f666c009fd0 tx=0x7f666c00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:53.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c00e040 con 0x7f667410d180 2023-08-11T14:00:53.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f666c0094c0 con 0x7f667410d180 2023-08-11T14:00:53.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c005040 con 0x7f667410d180 2023-08-11T14:00:53.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667419e2f0 con 0x7f667410d180 2023-08-11T14:00:53.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.329+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66741a5bb0 con 0x7f667410d180 2023-08-11T14:00:53.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.330+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667404e620 con 0x7f667410d180 2023-08-11T14:00:53.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.332+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f666c01d470 con 0x7f667410d180 2023-08-11T14:00:53.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.334+0000 7f66637fe700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 0x7f665c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:53.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.334+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f666c0cc470 con 0x7f667410d180 2023-08-11T14:00:53.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.334+0000 7f667259c700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 0x7f665c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.334+0000 7f667259c700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 0x7f665c0a2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6664005950 tx=0x7f666400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.338+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f666c096690 con 0x7f667410d180 2023-08-11T14:00:53.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:53 smithi187 ceph-mon[130973]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.545+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6674112af0 con 0x7f667410d180 2023-08-11T14:00:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.546+0000 7f66637fe700 1 -- 172.21.15.3:0/1164918775 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f666c099cb0 con 0x7f667410d180 2023-08-11T14:00:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:00:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 msgr2=0x7f665c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 0x7f665c0a2410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6664005950 tx=0x7f666400b500 comp rx=0 tx=0).stop 2023-08-11T14:00:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 msgr2=0x7f667419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 0x7f667419d730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f666c009fd0 tx=0x7f666c00bce0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 shutdown_connections 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f667410d180 0x7f667419d730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6674072d50 0x7f6674198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f665c09ff50 0x7f665c0a2410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 --2- 172.21.15.3:0/1164918775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6674073670 0x7f66741992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 >> 172.21.15.3:0/1164918775 conn(0x7f66740fc410 msgr2=0x7f667410f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 shutdown_connections 2023-08-11T14:00:53.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:53.549+0000 7f6678bb6700 1 -- 172.21.15.3:0/1164918775 wait complete. 2023-08-11T14:00:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:53 smithi103 ceph-mon[130434]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:54.137 INFO:tasks.cephadm:Adding mds.a on smithi003 2023-08-11T14:00:54.137 INFO:tasks.cephadm:Adding mds.d on smithi003 2023-08-11T14:00:54.138 INFO:tasks.cephadm:Adding mds.g on smithi003 2023-08-11T14:00:54.138 INFO:tasks.cephadm:Adding mds.j on smithi003 2023-08-11T14:00:54.138 INFO:tasks.cephadm:Adding mds.b on smithi103 2023-08-11T14:00:54.138 INFO:tasks.cephadm:Adding mds.e on smithi103 2023-08-11T14:00:54.138 INFO:tasks.cephadm:Adding mds.h on smithi103 2023-08-11T14:00:54.139 INFO:tasks.cephadm:Adding mds.k on smithi103 2023-08-11T14:00:54.139 INFO:tasks.cephadm:Adding mds.c on smithi187 2023-08-11T14:00:54.139 INFO:tasks.cephadm:Adding mds.f on smithi187 2023-08-11T14:00:54.139 INFO:tasks.cephadm:Adding mds.i on smithi187 2023-08-11T14:00:54.139 INFO:tasks.cephadm:Adding mds.l on smithi187 2023-08-11T14:00:54.140 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l' 2023-08-11T14:00:54.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1164918775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:54.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1164918775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:54.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1164918775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:54.713 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:00:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:55 smithi003 ceph-mon[97195]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:55.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:55 smithi187 ceph-mon[130973]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:55 smithi103 ceph-mon[130434]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:55.874 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 -- 172.21.15.187:0/4171606696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11540ffb60 msgr2=0x7f11540fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:55.875 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 --2- 172.21.15.187:0/4171606696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11540ffb60 0x7f11540fffe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1144009990 tx=0x7f1144009ca0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.875 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 -- 172.21.15.187:0/4171606696 shutdown_connections 2023-08-11T14:00:55.875 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 --2- 172.21.15.187:0/4171606696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1154100520 0x7f1154107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.875 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 --2- 172.21.15.187:0/4171606696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1154100f90 0x7f11541013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.876 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 --2- 172.21.15.187:0/4171606696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11540ffb60 0x7f11540fffe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.876 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 -- 172.21.15.187:0/4171606696 >> 172.21.15.187:0/4171606696 conn(0x7f11540fa630 msgr2=0x7f11540fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:55.876 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 -- 172.21.15.187:0/4171606696 shutdown_connections 2023-08-11T14:00:55.876 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.872+0000 7f1158900700 1 -- 172.21.15.187:0/4171606696 wait complete. 2023-08-11T14:00:55.879 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 Processor -- start 2023-08-11T14:00:55.879 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 -- start start 2023-08-11T14:00:55.880 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11540ffb60 0x7f11540ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:55.880 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:55.880 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1154100f90 0x7f115419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:55.880 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115419e230 con 0x7f1154100f90 2023-08-11T14:00:55.881 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115419e3a0 con 0x7f11540ffb60 2023-08-11T14:00:55.881 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.873+0000 7f1158900700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115419e560 con 0x7f1154100520 2023-08-11T14:00:55.881 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:55.882 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:55388/0 (socket says 172.21.15.187:55388) 2023-08-11T14:00:55.883 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 -- 172.21.15.187:0/1148418770 learned_addr learned my addr 172.21.15.187:0/1148418770 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-08-11T14:00:55.883 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f115259c700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11540ffb60 0x7f11540ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:55.883 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 -- 172.21.15.187:0/1148418770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11540ffb60 msgr2=0x7f11540ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11540ffb60 0x7f11540ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 -- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1154100f90 msgr2=0x7f115419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1154100f90 0x7f115419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 -- 172.21.15.187:0/1148418770 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440095f0 con 0x7f1154100520 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1151d9b700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1144000c00 tx=0x7f114400f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:55.884 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114401d070 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f114400fce0 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440176e0 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115419e7e0 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.874+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115419ecb0 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.876+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1154106ed0 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.876+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1144022470 con 0x7f1154100520 2023-08-11T14:00:55.885 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.877+0000 7f11437fe700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 0x7f113c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:55.886 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.877+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f11440cc130 con 0x7f1154100520 2023-08-11T14:00:55.886 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.877+0000 7f115259c700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 0x7f113c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:55.887 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.878+0000 7f115259c700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 0x7f113c0a2490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:55.887 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:55.882+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f114405cc30 con 0x7f1154100520 2023-08-11T14:00:56.080 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.078+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f115402cf30 con 0x7f113c09ffd0 2023-08-11T14:00:56.088 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.087+0000 7f11437fe700 1 -- 172.21.15.187:0/1148418770 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f115402cf30 con 0x7f113c09ffd0 2023-08-11T14:00:56.088 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mds.all update... 2023-08-11T14:00:56.090 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 msgr2=0x7f113c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 0x7f113c0a2490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1148006fd0 tx=0x7f1148008040 comp rx=0 tx=0).stop 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 msgr2=0x7f115419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1144000c00 tx=0x7f114400f830 comp rx=0 tx=0).stop 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 shutdown_connections 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1154100f90 0x7f115419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f113c09ffd0 0x7f113c0a2490 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:56.091 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11540ffb60 0x7f11540ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:56.092 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 --2- 172.21.15.187:0/1148418770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1154100520 0x7f115419b440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:56.092 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 >> 172.21.15.187:0/1148418770 conn(0x7f11540fa630 msgr2=0x7f1154104010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:56.092 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 shutdown_connections 2023-08-11T14:00:56.092 INFO:teuthology.orchestra.run.smithi187.stderr:2023-08-11T14:00:56.089+0000 7f1158900700 1 -- 172.21.15.187:0/1148418770 wait complete. 2023-08-11T14:00:56.469 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.a.service 2023-08-11T14:00:56.475 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.d.service 2023-08-11T14:00:56.520 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.g.service 2023-08-11T14:00:56.557 INFO:journalctl@ceph.mds.a.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T14:00:56.560 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.j.service 2023-08-11T14:00:56.594 INFO:journalctl@ceph.mds.d.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T14:00:56.597 DEBUG:teuthology.orchestra.run.smithi103:mds.b> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.b.service 2023-08-11T14:00:56.600 DEBUG:teuthology.orchestra.run.smithi103:mds.e> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.e.service 2023-08-11T14:00:56.629 INFO:journalctl@ceph.mds.b.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T14:00:56.630 DEBUG:teuthology.orchestra.run.smithi103:mds.h> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.h.service 2023-08-11T14:00:56.634 INFO:journalctl@ceph.mds.g.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T14:00:56.656 INFO:journalctl@ceph.mds.e.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T14:00:56.657 DEBUG:teuthology.orchestra.run.smithi103:mds.k> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.k.service 2023-08-11T14:00:56.671 INFO:journalctl@ceph.mds.j.smithi003.stdout:-- Logs begin at Fri 2023-08-11 13:28:43 UTC. -- 2023-08-11T14:00:56.682 INFO:journalctl@ceph.mds.h.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T14:00:56.683 DEBUG:teuthology.orchestra.run.smithi187:mds.c> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.c.service 2023-08-11T14:00:56.685 DEBUG:teuthology.orchestra.run.smithi187:mds.f> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.f.service 2023-08-11T14:00:56.708 INFO:journalctl@ceph.mds.k.smithi103.stdout:-- Logs begin at Fri 2023-08-11 13:23:43 UTC. -- 2023-08-11T14:00:56.715 INFO:journalctl@ceph.mds.c.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T14:00:56.716 DEBUG:teuthology.orchestra.run.smithi187:mds.i> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.i.service 2023-08-11T14:00:56.742 INFO:journalctl@ceph.mds.f.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T14:00:56.743 DEBUG:teuthology.orchestra.run.smithi187:mds.l> sudo journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.l.service 2023-08-11T14:00:56.769 INFO:journalctl@ceph.mds.i.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T14:00:56.770 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-08-11T14:00:56.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:00:56.795 INFO:journalctl@ceph.mds.l.smithi187.stdout:-- Logs begin at Fri 2023-08-11 13:23:04 UTC. -- 2023-08-11T14:00:57.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/929815185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88100920 msgr2=0x7f6d88100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/929815185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88100920 0x7f6d88100d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6d780099c0 tx=0x7f6d78009cd0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/929815185 shutdown_connections 2023-08-11T14:00:57.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/929815185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 0x7f6d88108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/929815185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88100920 0x7f6d88100d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/929815185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88101b10 0x7f6d88101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/929815185 >> 172.21.15.3:0/929815185 conn(0x7f6d880fbe50 msgr2=0x7f6d880fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/929815185 shutdown_connections 2023-08-11T14:00:57.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.050+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/929815185 wait complete. 2023-08-11T14:00:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.051+0000 7f6d8d9ff700 1 Processor -- start 2023-08-11T14:00:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.051+0000 7f6d8d9ff700 1 -- start start 2023-08-11T14:00:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.051+0000 7f6d8d9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88100920 0x7f6d880746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.051+0000 7f6d8d9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d8d9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 0x7f6d88073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d8d9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880739f0 con 0x7f6d88101b10 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d8d9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88073b60 con 0x7f6d88100920 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d8d9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d881a06b0 con 0x7f6d881024b0 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47204/0 (socket says 172.21.15.3:47204) 2023-08-11T14:00:57.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 -- 172.21.15.3:0/3480835282 learned_addr learned my addr 172.21.15.3:0/3480835282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d877fe700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 0x7f6d88073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d86ffd700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88100920 0x7f6d880746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 -- 172.21.15.3:0/3480835282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 msgr2=0x7f6d88073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 0x7f6d88073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 -- 172.21.15.3:0/3480835282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88100920 msgr2=0x7f6d880746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88100920 0x7f6d880746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.052+0000 7f6d867fc700 1 -- 172.21.15.3:0/3480835282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d780095f0 con 0x7f6d88101b10 2023-08-11T14:00:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d867fc700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00f890 con 0x7f6d88101b10 2023-08-11T14:00:57.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7c010460 con 0x7f6d88101b10 2023-08-11T14:00:57.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d881a0990 con 0x7f6d88101b10 2023-08-11T14:00:57.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00e610 con 0x7f6d88101b10 2023-08-11T14:00:57.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.053+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d881a0ef0 con 0x7f6d88101b10 2023-08-11T14:00:57.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.054+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88107e90 con 0x7f6d88101b10 2023-08-11T14:00:57.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.056+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6d7c0108d0 con 0x7f6d88101b10 2023-08-11T14:00:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.057+0000 7f6d8c9fd700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 0x7f6d700a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.057+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d7c0cab60 con 0x7f6d88101b10 2023-08-11T14:00:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.057+0000 7f6d86ffd700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 0x7f6d700a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.058+0000 7f6d86ffd700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 0x7f6d700a24e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009f50 tx=0x7f6d780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.062+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6d7c094cf0 con 0x7f6d88101b10 2023-08-11T14:00:57.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.281+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d8810c200 con 0x7f6d88101b10 2023-08-11T14:00:57.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.282+0000 7f6d8c9fd700 1 -- 172.21.15.3:0/3480835282 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18376 (secure 0 0 0) 0x7f6d7c098310 con 0x7f6d88101b10 2023-08-11T14:00:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:00:57.284 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:00:45.467682+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:00:57.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 msgr2=0x7f6d700a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 0x7f6d700a24e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009f50 tx=0x7f6d780058e0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 msgr2=0x7f6d88072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).stop 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 shutdown_connections 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d881024b0 0x7f6d88073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6d700a0020 0x7f6d700a24e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d88100920 0x7f6d880746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 --2- 172.21.15.3:0/3480835282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d88101b10 0x7f6d88072d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 >> 172.21.15.3:0/3480835282 conn(0x7f6d880fbe50 msgr2=0x7f6d88104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:57.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 shutdown_connections 2023-08-11T14:00:57.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.284+0000 7f6d8d9ff700 1 -- 172.21.15.3:0/3480835282 wait complete. 2023-08-11T14:00:57.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T14:00:57.306 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:00:57.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:57.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-08-11T14:00:57.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:57.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:57.368 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:57.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:57.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:57.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:57.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:57.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:57.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:00:57.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:00:57.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:57.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:57.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:57.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 -- 172.21.15.3:0/1144506808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 msgr2=0x7feb2c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 --2- 172.21.15.3:0/1144506808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c073800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009990 tx=0x7feb1c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 -- 172.21.15.3:0/1144506808 shutdown_connections 2023-08-11T14:00:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 --2- 172.21.15.3:0/1144506808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 --2- 172.21.15.3:0/1144506808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 0x7feb2c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 --2- 172.21.15.3:0/1144506808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c073800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 -- 172.21.15.3:0/1144506808 >> 172.21.15.3:0/1144506808 conn(0x7feb2c0fbe60 msgr2=0x7feb2c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 -- 172.21.15.3:0/1144506808 shutdown_connections 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.513+0000 7feb333ff700 1 -- 172.21.15.3:0/1144506808 wait complete. 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.514+0000 7feb333ff700 1 Processor -- start 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.514+0000 7feb333ff700 1 -- start start 2023-08-11T14:00:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 0x7feb2c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1a0460 con 0x7feb2c073380 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1a05a0 con 0x7feb2c074960 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb333ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c1a0730 con 0x7feb2c1087d0 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47214/0 (socket says 172.21.15.3:47214) 2023-08-11T14:00:57.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 -- 172.21.15.3:0/4036446999 learned_addr learned my addr 172.21.15.3:0/4036446999 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3199c700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3099a700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 0x7feb2c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 -- 172.21.15.3:0/4036446999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 msgr2=0x7feb2c19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c19fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 -- 172.21.15.3:0/4036446999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 msgr2=0x7feb2c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 0x7feb2c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3119b700 1 -- 172.21.15.3:0/4036446999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c0095f0 con 0x7feb2c073380 2023-08-11T14:00:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.515+0000 7feb3199c700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb3119b700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb2800d980 tx=0x7feb2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28004220 con 0x7feb2c073380 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb28007640 con 0x7feb2c073380 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28005690 con 0x7feb2c073380 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c1a0a10 con 0x7feb2c073380 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.516+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c1a0f20 con 0x7feb2c073380 2023-08-11T14:00:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.517+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c106e80 con 0x7feb2c073380 2023-08-11T14:00:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.519+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7feb28004750 con 0x7feb2c073380 2023-08-11T14:00:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.520+0000 7feb227fc700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 0x7feb180a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.521+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb280cb070 con 0x7feb2c073380 2023-08-11T14:00:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.521+0000 7feb3099a700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 0x7feb180a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.521+0000 7feb3099a700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 0x7feb180a2490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b2e0 tx=0x7feb1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.523+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7feb28095200 con 0x7feb2c073380 2023-08-11T14:00:57.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.732+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb2c10e350 con 0x7feb2c073380 2023-08-11T14:00:57.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.732+0000 7feb227fc700 1 -- 172.21.15.3:0/4036446999 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb280160a0 con 0x7feb2c073380 2023-08-11T14:00:57.737 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:00:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.734+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 msgr2=0x7feb180a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.734+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 0x7feb180a2490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b2e0 tx=0x7feb1c005fb0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.734+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 msgr2=0x7feb2c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.734+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb2800d980 tx=0x7feb2800dc90 comp rx=0 tx=0).stop 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 shutdown_connections 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb2c1087d0 0x7feb2c19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb2c073380 0x7feb2c1988e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb1809ffd0 0x7feb180a2490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 --2- 172.21.15.3:0/4036446999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb2c074960 0x7feb2c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 >> 172.21.15.3:0/4036446999 conn(0x7feb2c0fbe60 msgr2=0x7feb2c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 shutdown_connections 2023-08-11T14:00:57.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.735+0000 7feb333ff700 1 -- 172.21.15.3:0/4036446999 wait complete. 2023-08-11T14:00:57.756 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:00:57.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 -- 172.21.15.3:0/1988807279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5258100920 msgr2=0x7f5258100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 --2- 172.21.15.3:0/1988807279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5258100920 0x7f5258100d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5248009990 tx=0x7f5248009ca0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 -- 172.21.15.3:0/1988807279 shutdown_connections 2023-08-11T14:00:57.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 --2- 172.21.15.3:0/1988807279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52581024b0 0x7f5258108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 --2- 172.21.15.3:0/1988807279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5258100920 0x7f5258100d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 --2- 172.21.15.3:0/1988807279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258101b10 0x7f5258101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 -- 172.21.15.3:0/1988807279 >> 172.21.15.3:0/1988807279 conn(0x7f52580fbe50 msgr2=0x7f52580fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:57.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 -- 172.21.15.3:0/1988807279 shutdown_connections 2023-08-11T14:00:57.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.959+0000 7f525e5ff700 1 -- 172.21.15.3:0/1988807279 wait complete. 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 Processor -- start 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 -- start start 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5258101b10 0x7f5258072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52581024b0 0x7f5258073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580739f0 con 0x7f5258100920 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5258073b60 con 0x7f5258101b10 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.960+0000 7f525e5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581a05f0 con 0x7f52581024b0 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47240/0 (socket says 172.21.15.3:47240) 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 -- 172.21.15.3:0/685695711 learned_addr learned my addr 172.21.15.3:0/685695711 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f525cb9c700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52581024b0 0x7f5258073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f52577fe700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5258101b10 0x7f5258072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 -- 172.21.15.3:0/685695711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52581024b0 msgr2=0x7f5258073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52581024b0 0x7f5258073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 -- 172.21.15.3:0/685695711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5258101b10 msgr2=0x7f5258072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5258101b10 0x7f5258072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 -- 172.21.15.3:0/685695711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c009520 con 0x7f5258100920 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.961+0000 7f5257fff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5248004710 tx=0x7f5248009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.962+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524801d070 con 0x7f5258100920 2023-08-11T14:00:57.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.962+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52480095f0 con 0x7f5258100920 2023-08-11T14:00:57.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.962+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5248021410 con 0x7f5258100920 2023-08-11T14:00:57.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.962+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248017650 con 0x7f5258100920 2023-08-11T14:00:57.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.962+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581a0b60 con 0x7f5258100920 2023-08-11T14:00:57.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.963+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525810c130 con 0x7f5258100920 2023-08-11T14:00:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.965+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5248021580 con 0x7f5258100920 2023-08-11T14:00:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.966+0000 7f52557fa700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 0x7f52400a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:00:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.966+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52480cc1c0 con 0x7f5258100920 2023-08-11T14:00:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.966+0000 7f52577fe700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 0x7f52400a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:00:57.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.967+0000 7f52577fe700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 0x7f52400a2460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f524c009df0 tx=0x7f524c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:00:57.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:57.970+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f52480963e0 con 0x7f5258100920 2023-08-11T14:00:58.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.231+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f525810c2f0 con 0x7f5258100920 2023-08-11T14:00:58.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.234+0000 7f52557fa700 1 -- 172.21.15.3:0/685695711 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f5248099a00 con 0x7f5258100920 2023-08-11T14:00:58.236 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:00:58.236 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-08-11T14:00:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 msgr2=0x7f52400a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 0x7f52400a2460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f524c009df0 tx=0x7f524c009260 comp rx=0 tx=0).stop 2023-08-11T14:00:58.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 msgr2=0x7f52580746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5248004710 tx=0x7f5248009580 comp rx=0 tx=0).stop 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 shutdown_connections 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52581024b0 0x7f5258073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5258100920 0x7f52580746d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f524009ffa0 0x7f52400a2460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 --2- 172.21.15.3:0/685695711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5258101b10 0x7f5258072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 >> 172.21.15.3:0/685695711 conn(0x7f52580fbe50 msgr2=0x7f52580fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:00:58.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 shutdown_connections 2023-08-11T14:00:58.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:00:58.237+0000 7f525e5ff700 1 -- 172.21.15.3:0/685695711 wait complete. 2023-08-11T14:00:58.241 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2023-08-11T14:00:58.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:58 smithi187 ceph-mon[130973]: Deploying daemon mds.l on smithi187 2023-08-11T14:00:58.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3480835282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:58.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4036446999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:58.442 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 systemd[1]: Starting Ceph mds.l for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:00:58.443 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 podman[155844]: 2023-08-11T14:00:58.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:58 smithi003 ceph-mon[97195]: Deploying daemon mds.l on smithi187 2023-08-11T14:00:58.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3480835282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:58.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4036446999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:58 smithi103 ceph-mon[130434]: Deploying daemon mds.l on smithi187 2023-08-11T14:00:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3480835282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:00:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4036446999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:00:59.044 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 ceph-mds[155884]: -- 172.21.15.187:0/1896963356 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9960f51e0 con 0x55a9960cac00 2023-08-11T14:00:59.044 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: starting mds.l at 2023-08-11T14:00:59.045 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 bash[155844]: 05379e5872a5019f8294cf6905eb5bb1a975a241ccedc1f9aad4dcf5a4edd43b 2023-08-11T14:00:59.045 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:00:58 smithi187 systemd[1]: Started Ceph mds.l for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:00:59.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/685695711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6832/1102333992,v1:172.21.15.187:6833/1102333992] up:boot 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: fsmap 1 up:standby 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:59.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:59.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:00:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/685695711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:00:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6832/1102333992,v1:172.21.15.187:6833/1102333992] up:boot 2023-08-11T14:00:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: fsmap 1 up:standby 2023-08-11T14:00:59.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:59.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:00:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:00:59.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:00:59.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/685695711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:00:59.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6832/1102333992,v1:172.21.15.187:6833/1102333992] up:boot 2023-08-11T14:00:59.606 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: fsmap 1 up:standby 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:00:59.607 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:00:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:00.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 -- 172.21.15.3:0/890651576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098074960 msgr2=0x7f7098072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 --2- 172.21.15.3:0/890651576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098074960 0x7f7098072db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f7088009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 -- 172.21.15.3:0/890651576 shutdown_connections 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 --2- 172.21.15.3:0/890651576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70981087f0 0x7f709810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 --2- 172.21.15.3:0/890651576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098074960 0x7f7098072db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 --2- 172.21.15.3:0/890651576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7098073380 0x7f7098073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 -- 172.21.15.3:0/890651576 >> 172.21.15.3:0/890651576 conn(0x7f70980fbe60 msgr2=0x7f70980fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 -- 172.21.15.3:0/890651576 shutdown_connections 2023-08-11T14:01:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.472+0000 7f709cdff700 1 -- 172.21.15.3:0/890651576 wait complete. 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 Processor -- start 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 -- start start 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7098074960 0x7f7098198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70981087f0 0x7f709819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981a03a0 con 0x7f7098073380 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981a04e0 con 0x7f7098074960 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709cdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70981a0670 con 0x7f70981087f0 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.473+0000 7f709659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47262/0 (socket says 172.21.15.3:47262) 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 -- 172.21.15.3:0/625752699 learned_addr learned my addr 172.21.15.3:0/625752699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f708ffff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7098074960 0x7f7098198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f7096d9d700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70981087f0 0x7f709819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 -- 172.21.15.3:0/625752699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70981087f0 msgr2=0x7f709819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70981087f0 0x7f709819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 -- 172.21.15.3:0/625752699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7098074960 msgr2=0x7f7098198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7098074960 0x7f7098198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 -- 172.21.15.3:0/625752699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880095f0 con 0x7f7098073380 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f709659c700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7088004730 tx=0x7f7088004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708801d070 con 0x7f7098073380 2023-08-11T14:01:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7088004c90 con 0x7f7098073380 2023-08-11T14:01:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.474+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088005570 con 0x7f7098073380 2023-08-11T14:01:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.475+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70981a08f0 con 0x7f7098073380 2023-08-11T14:01:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.475+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70981a0c60 con 0x7f7098073380 2023-08-11T14:01:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.476+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7098106ea0 con 0x7f7098073380 2023-08-11T14:01:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.478+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f708800fc70 con 0x7f7098073380 2023-08-11T14:01:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.479+0000 7f708f7fe700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 0x7f70840a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.479+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70880cc150 con 0x7f7098073380 2023-08-11T14:01:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.479+0000 7f708ffff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 0x7f70840a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.480+0000 7f708ffff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 0x7f70840a24e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7098074680 tx=0x7f7080009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.483+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f708805ac30 con 0x7f7098073380 2023-08-11T14:01:00.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:00 smithi003 ceph-mon[97195]: Deploying daemon mds.k on smithi103 2023-08-11T14:01:00.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:00 smithi187 ceph-mon[130973]: Deploying daemon mds.k on smithi103 2023-08-11T14:01:00.602 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:00 smithi103 ceph-mon[130434]: Deploying daemon mds.k on smithi103 2023-08-11T14:01:00.603 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 systemd[1]: Starting Ceph mds.k for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.740+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f709810e370 con 0x7f7098073380 2023-08-11T14:01:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.740+0000 7f708f7fe700 1 -- 172.21.15.3:0/625752699 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f7088096370 con 0x7f7098073380 2023-08-11T14:01:00.742 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:00.742 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-08-11T14:01:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 msgr2=0x7f70840a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 0x7f70840a24e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7098074680 tx=0x7f7080009260 comp rx=0 tx=0).stop 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 msgr2=0x7f7098198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7088004730 tx=0x7f7088004760 comp rx=0 tx=0).stop 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 shutdown_connections 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70981087f0 0x7f709819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7098073380 0x7f7098198870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f70840a0020 0x7f70840a24e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 --2- 172.21.15.3:0/625752699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7098074960 0x7f7098198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 >> 172.21.15.3:0/625752699 conn(0x7f70980fbe60 msgr2=0x7f70980fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 shutdown_connections 2023-08-11T14:01:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:00.743+0000 7f709cdff700 1 -- 172.21.15.3:0/625752699 wait complete. 2023-08-11T14:01:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2023-08-11T14:01:00.892 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 podman[154551]: 2023-08-11T14:01:00.892 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 ceph-mds[154591]: -- 172.21.15.103:0/3700981692 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561374391e0 con 0x55613740ec00 2023-08-11T14:01:00.893 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 bash[154551]: 9f2be860fd400eda56ed7052f7e7abd418f334ad51de28cf63148d953d67b9c7 2023-08-11T14:01:00.893 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: starting mds.k at 2023-08-11T14:01:01.148 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:01:00 smithi103 systemd[1]: Started Ceph mds.k for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/625752699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:01.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:01.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/625752699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:01.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:01.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:01.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/625752699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:02.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:02 smithi003 ceph-mon[97195]: Deploying daemon mds.j on smithi003 2023-08-11T14:01:02.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:02 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6832/618362106,v1:172.21.15.103:6833/618362106] up:boot 2023-08-11T14:01:02.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:02 smithi003 ceph-mon[97195]: fsmap 2 up:standby 2023-08-11T14:01:02.465 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:01:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:02 smithi187 ceph-mon[130973]: Deploying daemon mds.j on smithi003 2023-08-11T14:01:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:02 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6832/618362106,v1:172.21.15.103:6833/618362106] up:boot 2023-08-11T14:01:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:02 smithi187 ceph-mon[130973]: fsmap 2 up:standby 2023-08-11T14:01:02.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:01:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:02 smithi103 ceph-mon[130434]: Deploying daemon mds.j on smithi003 2023-08-11T14:01:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:02 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6832/618362106,v1:172.21.15.103:6833/618362106] up:boot 2023-08-11T14:01:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:02 smithi103 ceph-mon[130434]: fsmap 2 up:standby 2023-08-11T14:01:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:01:02.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:03.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.017+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/2914698382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b64076ee0 msgr2=0x7f6b64077760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.017+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/2914698382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b64076ee0 0x7f6b64077760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b0a0 tx=0x7f6b5c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/2914698382 shutdown_connections 2023-08-11T14:01:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/2914698382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b64076ee0 0x7f6b64077760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/2914698382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b640752e0 0x7f6b64075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/2914698382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076520 0x7f6b640769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/2914698382 >> 172.21.15.3:0/2914698382 conn(0x7f6b640fe0a0 msgr2=0x7f6b641004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/2914698382 shutdown_connections 2023-08-11T14:01:03.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.020+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/2914698382 wait complete. 2023-08-11T14:01:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.022+0000 7f6b6c5ff700 1 Processor -- start 2023-08-11T14:01:03.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 -- start start 2023-08-11T14:01:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b640752e0 0x7f6b6406f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64076520 0x7f6b6406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64072850 con 0x7f6b64076ee0 2023-08-11T14:01:03.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640729c0 con 0x7f6b640752e0 2023-08-11T14:01:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.023+0000 7f6b6c5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b641ad370 con 0x7f6b64076520 2023-08-11T14:01:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.024+0000 7f6b6ab9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.024+0000 7f6b6ab9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43764/0 (socket says 172.21.15.3:43764) 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.024+0000 7f6b6ab9c700 1 -- 172.21.15.3:0/1547892242 learned_addr learned my addr 172.21.15.3:0/1547892242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.024+0000 7f6b6a39b700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b640752e0 0x7f6b6406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 -- 172.21.15.3:0/1547892242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64076520 msgr2=0x7f6b6406f7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64076520 0x7f6b6406f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 -- 172.21.15.3:0/1547892242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b640752e0 msgr2=0x7f6b6406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b640752e0 0x7f6b6406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 -- 172.21.15.3:0/1547892242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009d00 con 0x7f6b64076ee0 2023-08-11T14:01:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.025+0000 7f6b6ab9c700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b780 tx=0x7f6b5c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.026+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00e040 con 0x7f6b64076ee0 2023-08-11T14:01:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.026+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b641ad5c0 con 0x7f6b64076ee0 2023-08-11T14:01:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.026+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b641adb10 con 0x7f6b64076ee0 2023-08-11T14:01:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.026+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5c009b80 con 0x7f6b64076ee0 2023-08-11T14:01:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.026+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0052a0 con 0x7f6b64076ee0 2023-08-11T14:01:03.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.032+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6b5c01d470 con 0x7f6b64076ee0 2023-08-11T14:01:03.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.034+0000 7f6b5b7fe700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 0x7f6b500a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:03.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.035+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6b5c0cc0f0 con 0x7f6b64076ee0 2023-08-11T14:01:03.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.035+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b48005320 con 0x7f6b64076ee0 2023-08-11T14:01:03.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.037+0000 7f6b6a39b700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 0x7f6b500a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:03.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.038+0000 7f6b6a39b700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 0x7f6b500a2560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009c60 tx=0x7f6b60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:03.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.044+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6b5c096310 con 0x7f6b64076ee0 2023-08-11T14:01:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.311+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b48005b50 con 0x7f6b64076ee0 2023-08-11T14:01:03.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.312+0000 7f6b5b7fe700 1 -- 172.21.15.3:0/1547892242 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f6b5c021860 con 0x7f6b64076ee0 2023-08-11T14:01:03.315 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:03.315 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T14:01:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 msgr2=0x7f6b500a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 0x7f6b500a2560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009c60 tx=0x7f6b60009260 comp rx=0 tx=0).stop 2023-08-11T14:01:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 msgr2=0x7f6b641ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:03.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b780 tx=0x7f6b5c00bd60 comp rx=0 tx=0).stop 2023-08-11T14:01:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 shutdown_connections 2023-08-11T14:01:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b64076ee0 0x7f6b641ace30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b500a00a0 0x7f6b500a2560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b640752e0 0x7f6b6406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 --2- 172.21.15.3:0/1547892242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b64076520 0x7f6b6406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 >> 172.21.15.3:0/1547892242 conn(0x7f6b640fe0a0 msgr2=0x7f6b640ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 shutdown_connections 2023-08-11T14:01:03.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:03.315+0000 7f6b6c5ff700 1 -- 172.21.15.3:0/1547892242 wait complete. 2023-08-11T14:01:03.320 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2023-08-11T14:01:03.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:03 smithi003 ceph-mon[97195]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:03.545 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:03 smithi003 systemd[1]: Starting Ceph mds.j for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:03.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:03 smithi187 ceph-mon[130973]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:03 smithi103 ceph-mon[130434]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:03.993 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:03 smithi003 podman[130570]: 2023-08-11T14:01:04.268 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:03 smithi003 bash[130570]: 4b5d51e45ded30d505e6fb444f70af699e94cde87e61893a930c133e2b97498d 2023-08-11T14:01:04.268 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:03 smithi003 ceph-mds[130610]: -- 172.21.15.3:0/1641700565 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562dc249d1e0 con 0x562dc2473000 2023-08-11T14:01:04.269 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-j[130586]: starting mds.j at 2023-08-11T14:01:04.269 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:01:04 smithi003 systemd[1]: Started Ceph mds.j for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:04.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1547892242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:04.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:04.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:04.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1547892242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:04.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:04.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:04.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1547892242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:04.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:04.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:04.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:05.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:05.338 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:05 smithi003 ceph-mon[97195]: Deploying daemon mds.d on smithi003 2023-08-11T14:01:05.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:05 smithi003 ceph-mon[97195]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:05.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:05 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6834/113676080,v1:172.21.15.3:6835/113676080] up:boot 2023-08-11T14:01:05.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:05 smithi003 ceph-mon[97195]: fsmap 3 up:standby 2023-08-11T14:01:05.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T14:01:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:05 smithi187 ceph-mon[130973]: Deploying daemon mds.d on smithi003 2023-08-11T14:01:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:05 smithi187 ceph-mon[130973]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:05.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:05 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6834/113676080,v1:172.21.15.3:6835/113676080] up:boot 2023-08-11T14:01:05.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:05 smithi187 ceph-mon[130973]: fsmap 3 up:standby 2023-08-11T14:01:05.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T14:01:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 -- 172.21.15.3:0/2611581386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d0073380 msgr2=0x7f26d0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 --2- 172.21.15.3:0/2611581386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d0073380 0x7f26d0073800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009990 tx=0x7f26c0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 -- 172.21.15.3:0/2611581386 shutdown_connections 2023-08-11T14:01:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 --2- 172.21.15.3:0/2611581386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d01087e0 0x7f26d010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 --2- 172.21.15.3:0/2611581386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0074960 0x7f26d0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 --2- 172.21.15.3:0/2611581386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d0073380 0x7f26d0073800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 -- 172.21.15.3:0/2611581386 >> 172.21.15.3:0/2611581386 conn(0x7f26d00fbe50 msgr2=0x7f26d00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 -- 172.21.15.3:0/2611581386 shutdown_connections 2023-08-11T14:01:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.565+0000 7f26d87ff700 1 -- 172.21.15.3:0/2611581386 wait complete. 2023-08-11T14:01:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.566+0000 7f26d87ff700 1 Processor -- start 2023-08-11T14:01:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.566+0000 7f26d87ff700 1 -- start start 2023-08-11T14:01:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0073380 0x7f26d0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d0074960 0x7f26d0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01a03b0 con 0x7f26d01087e0 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01a04f0 con 0x7f26d0073380 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d87ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01a0680 con 0x7f26d0074960 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43816/0 (socket says 172.21.15.3:43816) 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d5d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d0074960 0x7f26d0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 -- 172.21.15.3:0/157731616 learned_addr learned my addr 172.21.15.3:0/157731616 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d659b700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0073380 0x7f26d0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 -- 172.21.15.3:0/157731616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d0074960 msgr2=0x7f26d0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d0074960 0x7f26d0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 -- 172.21.15.3:0/157731616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0073380 msgr2=0x7f26d0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0073380 0x7f26d0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.567+0000 7f26d6d9c700 1 -- 172.21.15.3:0/157731616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00095f0 con 0x7f26d01087e0 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26d6d9c700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26c8007ee0 tx=0x7f26c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8017070 con 0x7f26d01087e0 2023-08-11T14:01:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c800f040 con 0x7f26d01087e0 2023-08-11T14:01:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d01a0960 con 0x7f26d01087e0 2023-08-11T14:01:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8005580 con 0x7f26d01087e0 2023-08-11T14:01:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.568+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d01a0e20 con 0x7f26d01087e0 2023-08-11T14:01:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.569+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0106e90 con 0x7f26d01087e0 2023-08-11T14:01:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.571+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f26c800db20 con 0x7f26d01087e0 2023-08-11T14:01:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.572+0000 7f26c77fe700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 0x7f26bc0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.573+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26c80cac30 con 0x7f26d01087e0 2023-08-11T14:01:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.573+0000 7f26d659b700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 0x7f26bc0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.573+0000 7f26d659b700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 0x7f26bc0a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009c60 tx=0x7f26cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:05.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.575+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f26c8094dc0 con 0x7f26d01087e0 2023-08-11T14:01:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:05 smithi103 ceph-mon[130434]: Deploying daemon mds.d on smithi003 2023-08-11T14:01:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:05 smithi103 ceph-mon[130434]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:05 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6834/113676080,v1:172.21.15.3:6835/113676080] up:boot 2023-08-11T14:01:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:05 smithi103 ceph-mon[130434]: fsmap 3 up:standby 2023-08-11T14:01:05.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-08-11T14:01:05.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.834+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d010e360 con 0x7f26d01087e0 2023-08-11T14:01:05.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.835+0000 7f26c77fe700 1 -- 172.21.15.3:0/157731616 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2904 (secure 0 0 0) 0x7f26c80983e0 con 0x7f26d01087e0 2023-08-11T14:01:05.836 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:05.837 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 msgr2=0x7f26bc0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 0x7f26bc0a2460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009c60 tx=0x7f26cc009260 comp rx=0 tx=0).stop 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 msgr2=0x7f26d019fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26c8007ee0 tx=0x7f26c800d3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 shutdown_connections 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d01087e0 0x7f26d019fde0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26bc09ffa0 0x7f26bc0a2460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26d0073380 0x7f26d0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 --2- 172.21.15.3:0/157731616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26d0074960 0x7f26d0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 >> 172.21.15.3:0/157731616 conn(0x7f26d00fbe50 msgr2=0x7f26d00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 shutdown_connections 2023-08-11T14:01:05.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:05.837+0000 7f26d87ff700 1 -- 172.21.15.3:0/157731616 wait complete. 2023-08-11T14:01:05.841 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2023-08-11T14:01:06.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:06 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/157731616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:06.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:06 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/157731616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:06.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:06 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/157731616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:06.791 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:06 smithi003 systemd[1]: Starting Ceph mds.d for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:07.260 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:06 smithi003 podman[130956]: 2023-08-11T14:01:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:07 smithi003 ceph-mon[97195]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:07.542 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:07 smithi003 bash[130956]: 4ce7b2a37b2b0c6424b19bdb6959449bb14a084e6821a40402065c322c4bc0e3 2023-08-11T14:01:07.542 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:07 smithi003 ceph-mds[130996]: -- 172.21.15.3:0/3274322097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5604774ef1e0 con 0x5604774c5000 2023-08-11T14:01:07.542 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:07 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-d[130972]: starting mds.d at 2023-08-11T14:01:07.542 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:01:07 smithi003 systemd[1]: Started Ceph mds.d for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:07.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:07 smithi187 ceph-mon[130973]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:07 smithi103 ceph-mon[130434]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:07.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:08.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 -- 172.21.15.3:0/3163971430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 msgr2=0x7ff8c8100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/3163971430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8100d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40099c0 tx=0x7ff8c4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 -- 172.21.15.3:0/3163971430 shutdown_connections 2023-08-11T14:01:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/3163971430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/3163971430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8100d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/3163971430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 -- 172.21.15.3:0/3163971430 >> 172.21.15.3:0/3163971430 conn(0x7ff8c80fbe50 msgr2=0x7ff8c80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 -- 172.21.15.3:0/3163971430 shutdown_connections 2023-08-11T14:01:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.068+0000 7ff8cfbff700 1 -- 172.21.15.3:0/3163971430 wait complete. 2023-08-11T14:01:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 Processor -- start 2023-08-11T14:01:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 -- start start 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c81a0390 con 0x7ff8c8100920 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c81a04d0 con 0x7ff8c8101b10 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.069+0000 7ff8cfbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c81a0660 con 0x7ff8c81024b0 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60732/0 (socket says 172.21.15.3:60732) 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 -- 172.21.15.3:0/1751331173 learned_addr learned my addr 172.21.15.3:0/1751331173 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd99b700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8ce19c700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 -- 172.21.15.3:0/1751331173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 msgr2=0x7ff8c819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 -- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 msgr2=0x7ff8c8198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 -- 172.21.15.3:0/1751331173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40095f0 con 0x7ff8c8101b10 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8ce19c700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cd19a700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9a0 tx=0x7ff8b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8009790 con 0x7ff8c8101b10 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.070+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81a0940 con 0x7ff8c8101b10 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.071+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b800cbd0 con 0x7ff8c8101b10 2023-08-11T14:01:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.071+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8018680 con 0x7ff8c8101b10 2023-08-11T14:01:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.071+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c81a0ea0 con 0x7ff8c8101b10 2023-08-11T14:01:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.072+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c8192f70 con 0x7ff8c8101b10 2023-08-11T14:01:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.074+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff8b800cd40 con 0x7ff8c8101b10 2023-08-11T14:01:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.075+0000 7ff8beffd700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 0x7ff8b40a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.075+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8b8014070 con 0x7ff8c8101b10 2023-08-11T14:01:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.075+0000 7ff8cd99b700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 0x7ff8b40a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.076+0000 7ff8cd99b700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 0x7ff8b40a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009990 tx=0x7ff8c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.078+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff8b8095530 con 0x7ff8c8101b10 2023-08-11T14:01:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.341+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c810c3d0 con 0x7ff8c8101b10 2023-08-11T14:01:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.342+0000 7ff8beffd700 1 -- 172.21.15.3:0/1751331173 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7ff8b8098b50 con 0x7ff8c8101b10 2023-08-11T14:01:08.344 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:08.344 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-11T14:01:08.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 msgr2=0x7ff8b40a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:08.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 0x7ff8b40a2460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009990 tx=0x7ff8c4005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 msgr2=0x7ff8c8198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9a0 tx=0x7ff8b800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 shutdown_connections 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81024b0 0x7ff8c819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8c8100920 0x7ff8c8198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff8b409ffa0 0x7ff8b40a2460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 --2- 172.21.15.3:0/1751331173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c8101b10 0x7ff8c8198d50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 >> 172.21.15.3:0/1751331173 conn(0x7ff8c80fbe50 msgr2=0x7ff8c8107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 shutdown_connections 2023-08-11T14:01:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:08.345+0000 7ff8cfbff700 1 -- 172.21.15.3:0/1751331173 wait complete. 2023-08-11T14:01:08.348 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 5 2023-08-11T14:01:08.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: Deploying daemon mds.e on smithi103 2023-08-11T14:01:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:08.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6836/3880902111,v1:172.21.15.3:6837/3880902111] up:boot 2023-08-11T14:01:08.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: fsmap 4 up:standby 2023-08-11T14:01:08.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T14:01:08.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:08 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1751331173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:08.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: Deploying daemon mds.e on smithi103 2023-08-11T14:01:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6836/3880902111,v1:172.21.15.3:6837/3880902111] up:boot 2023-08-11T14:01:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: fsmap 4 up:standby 2023-08-11T14:01:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T14:01:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:08 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1751331173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:08.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: Deploying daemon mds.e on smithi103 2023-08-11T14:01:08.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:08.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6836/3880902111,v1:172.21.15.3:6837/3880902111] up:boot 2023-08-11T14:01:08.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: fsmap 4 up:standby 2023-08-11T14:01:08.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-08-11T14:01:08.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:08 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1751331173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:09.178 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:08 smithi103 systemd[1]: Starting Ceph mds.e for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:09.465 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:09 smithi103 podman[154905]: 2023-08-11T14:01:09.465 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:09 smithi103 ceph-mds[154945]: -- 172.21.15.103:0/1617459227 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598bd6111e0 con 0x5598bd5e6800 2023-08-11T14:01:09.928 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: starting mds.e at 2023-08-11T14:01:09.928 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:09 smithi103 bash[154905]: cb56ab98f7e533730e83658613378dddc5cf756cc0b4dcfd54d9149aadc9186d 2023-08-11T14:01:09.929 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:01:09 smithi103 systemd[1]: Started Ceph mds.e for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:10.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:10.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 -- 172.21.15.3:0/2492464940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c101b10 msgr2=0x7f283c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 --2- 172.21.15.3:0/2492464940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c101b10 0x7f283c101f70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28300099c0 tx=0x7f2830009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 -- 172.21.15.3:0/2492464940 shutdown_connections 2023-08-11T14:01:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 --2- 172.21.15.3:0/2492464940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c1024b0 0x7f283c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 --2- 172.21.15.3:0/2492464940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c100920 0x7f283c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 --2- 172.21.15.3:0/2492464940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c101b10 0x7f283c101f70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 -- 172.21.15.3:0/2492464940 >> 172.21.15.3:0/2492464940 conn(0x7f283c0fbe50 msgr2=0x7f283c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:10.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 -- 172.21.15.3:0/2492464940 shutdown_connections 2023-08-11T14:01:10.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.581+0000 7f2841fff700 1 -- 172.21.15.3:0/2492464940 wait complete. 2023-08-11T14:01:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.582+0000 7f2841fff700 1 Processor -- start 2023-08-11T14:01:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.582+0000 7f2841fff700 1 -- start start 2023-08-11T14:01:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 0x7f283c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:10.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 0x7f283c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:10.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a0380 con 0x7f283c100920 2023-08-11T14:01:10.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a04c0 con 0x7f283c1024b0 2023-08-11T14:01:10.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f2841fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a0650 con 0x7f283c101b10 2023-08-11T14:01:10.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:10.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 0x7f283c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:10.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43872/0 (socket says 172.21.15.3:43872) 2023-08-11T14:01:10.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 -- 172.21.15.3:0/1582406863 learned_addr learned my addr 172.21.15.3:0/1582406863 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:10.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 0x7f283c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59466/0 (socket says 172.21.15.3:59466) 2023-08-11T14:01:10.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283bfff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 0x7f283c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:10.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 -- 172.21.15.3:0/1582406863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 msgr2=0x7f283c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 0x7f283c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 -- 172.21.15.3:0/1582406863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 msgr2=0x7f283c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 0x7f283c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 -- 172.21.15.3:0/1582406863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300095f0 con 0x7f283c100920 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283bfff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 0x7f283c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.583+0000 7f283b7fe700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e970 tx=0x7f282c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.584+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c009790 con 0x7f283c100920 2023-08-11T14:01:10.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.584+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f282c00eec0 con 0x7f283c100920 2023-08-11T14:01:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.584+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c1a0930 con 0x7f283c100920 2023-08-11T14:01:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.584+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c003cf0 con 0x7f283c100920 2023-08-11T14:01:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.584+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c1a0e90 con 0x7f283c100920 2023-08-11T14:01:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.585+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c10c180 con 0x7f283c100920 2023-08-11T14:01:10.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.587+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f282c00ccd0 con 0x7f283c100920 2023-08-11T14:01:10.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.589+0000 7f2838ff9700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 0x7f28240a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:10.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.589+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f282c014070 con 0x7f283c100920 2023-08-11T14:01:10.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.589+0000 7f283affd700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 0x7f28240a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:10.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.590+0000 7f283affd700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 0x7f28240a24e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f283000b580 tx=0x7f2830005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:10.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.594+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f282c0951c0 con 0x7f283c100920 2023-08-11T14:01:10.642 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:boot 2023-08-11T14:01:10.643 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: fsmap 5 up:standby 2023-08-11T14:01:10.643 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:01:10.643 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.643 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.643 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.644 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:10.644 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:10.644 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:10.644 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: Deploying daemon mds.f on smithi187 2023-08-11T14:01:10.644 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:10 smithi187 ceph-mon[130973]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:boot 2023-08-11T14:01:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: fsmap 5 up:standby 2023-08-11T14:01:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: Deploying daemon mds.f on smithi187 2023-08-11T14:01:10.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:10 smithi003 ceph-mon[97195]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.849+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f283c1a0ac0 con 0x7f283c100920 2023-08-11T14:01:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.850+0000 7f2838ff9700 1 -- 172.21.15.3:0/1582406863 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4486 (secure 0 0 0) 0x7f283c1a0ac0 con 0x7f283c100920 2023-08-11T14:01:10.852 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:10.852 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-08-11T14:01:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 msgr2=0x7f28240a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 0x7f28240a24e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f283000b580 tx=0x7f2830005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 msgr2=0x7f283c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e970 tx=0x7f282c00ec80 comp rx=0 tx=0).stop 2023-08-11T14:01:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 shutdown_connections 2023-08-11T14:01:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f28240a0020 0x7f28240a24e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f283c1024b0 0x7f283c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f283c100920 0x7f283c198800 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 --2- 172.21.15.3:0/1582406863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f283c101b10 0x7f283c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 >> 172.21.15.3:0/1582406863 conn(0x7f283c0fbe50 msgr2=0x7f283c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.852+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 shutdown_connections 2023-08-11T14:01:10.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:10.853+0000 7f2841fff700 1 -- 172.21.15.3:0/1582406863 wait complete. 2023-08-11T14:01:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-08-11T14:01:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:boot 2023-08-11T14:01:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: fsmap 5 up:standby 2023-08-11T14:01:10.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:01:10.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:10.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: Deploying daemon mds.f on smithi187 2023-08-11T14:01:10.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:10 smithi103 ceph-mon[130434]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:11.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 systemd[1]: Starting Ceph mds.f for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:11.823 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:11 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1582406863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:11.823 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 podman[156195]: 2023-08-11T14:01:11.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:11 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1582406863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:12.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:11 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1582406863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:12.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 ceph-mds[156244]: -- 172.21.15.187:0/1303556787 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563007a5d1e0 con 0x563007a32c00 2023-08-11T14:01:12.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: starting mds.f at 2023-08-11T14:01:12.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 bash[156195]: 3197c4da4a407726f66a3464e930a5362b14fbd8f27b4508864ffdd8e984e6db 2023-08-11T14:01:12.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:01:11 smithi187 systemd[1]: Started Ceph mds.f for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:12.874 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 -- 172.21.15.3:0/1157662440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460101b10 msgr2=0x7f9460101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:13.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 --2- 172.21.15.3:0/1157662440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460101b10 0x7f9460101f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9454009990 tx=0x7f9454009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 -- 172.21.15.3:0/1157662440 shutdown_connections 2023-08-11T14:01:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 --2- 172.21.15.3:0/1157662440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 --2- 172.21.15.3:0/1157662440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460100920 0x7f9460100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 --2- 172.21.15.3:0/1157662440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460101b10 0x7f9460101f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 -- 172.21.15.3:0/1157662440 >> 172.21.15.3:0/1157662440 conn(0x7f94600fbe50 msgr2=0x7f94600fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.090+0000 7f94659ff700 1 -- 172.21.15.3:0/1157662440 shutdown_connections 2023-08-11T14:01:13.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.091+0000 7f94659ff700 1 -- 172.21.15.3:0/1157662440 wait complete. 2023-08-11T14:01:13.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.092+0000 7f94659ff700 1 Processor -- start 2023-08-11T14:01:13.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.092+0000 7f94659ff700 1 -- start start 2023-08-11T14:01:13.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:13.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460101b10 0x7f9460072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600739f0 con 0x7f94601024b0 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460073b60 con 0x7f9460101b10 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f94659ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94601a0670 con 0x7f9460100920 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:13.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60162/0 (socket says 172.21.15.3:60162) 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 -- 172.21.15.3:0/3656031545 learned_addr learned my addr 172.21.15.3:0/3656031545 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945e7fc700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460101b10 0x7f9460072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 -- 172.21.15.3:0/3656031545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460101b10 msgr2=0x7f9460072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460101b10 0x7f9460072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 -- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 msgr2=0x7f9460073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:13.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945effd700 1 -- 172.21.15.3:0/3656031545 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540095f0 con 0x7f9460100920 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.093+0000 7f945f7fe700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f945effd700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f945000e970 tx=0x7f945000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450009790 con 0x7f9460100920 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94601a0950 con 0x7f9460100920 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f945000eec0 con 0x7f9460100920 2023-08-11T14:01:13.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450003cf0 con 0x7f9460100920 2023-08-11T14:01:13.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.094+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601a0e60 con 0x7f9460100920 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.095+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9460107ec0 con 0x7f9460100920 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.097+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f945000ccd0 con 0x7f9460100920 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.098+0000 7f94649fd700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 0x7f94480a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.099+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9450014070 con 0x7f9460100920 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.099+0000 7f945e7fc700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 0x7f94480a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:13.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.099+0000 7f945e7fc700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 0x7f94480a2460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f945400b2e0 tx=0x7f9454005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:13.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.100+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9450095380 con 0x7f9460100920 2023-08-11T14:01:13.256 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: Deploying daemon mds.c on smithi187 2023-08-11T14:01:13.257 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:13.258 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6834/3236312344,v1:172.21.15.187:6835/3236312344] up:boot 2023-08-11T14:01:13.258 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: fsmap 6 up:standby 2023-08-11T14:01:13.258 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:01:13.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: Deploying daemon mds.c on smithi187 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6834/3236312344,v1:172.21.15.187:6835/3236312344] up:boot 2023-08-11T14:01:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: fsmap 6 up:standby 2023-08-11T14:01:13.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:01:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.360+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94601a0ae0 con 0x7f9460100920 2023-08-11T14:01:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.361+0000 7f94649fd700 1 -- 172.21.15.3:0/3656031545 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5280 (secure 0 0 0) 0x7f94601a0ae0 con 0x7f9460100920 2023-08-11T14:01:13.363 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:13.363 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-08-11T14:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 msgr2=0x7f94480a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 0x7f94480a2460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f945400b2e0 tx=0x7f9454005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 msgr2=0x7f94600746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f945000e970 tx=0x7f945000ec80 comp rx=0 tx=0).stop 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 shutdown_connections 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94601024b0 0x7f9460073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9460100920 0x7f94600746d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f944809ffa0 0x7f94480a2460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 --2- 172.21.15.3:0/3656031545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9460101b10 0x7f9460072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:13.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 >> 172.21.15.3:0/3656031545 conn(0x7f94600fbe50 msgr2=0x7f9460105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:13.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 shutdown_connections 2023-08-11T14:01:13.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:13.364+0000 7f94659ff700 1 -- 172.21.15.3:0/3656031545 wait complete. 2023-08-11T14:01:13.367 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2023-08-11T14:01:13.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: Deploying daemon mds.c on smithi187 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6834/3236312344,v1:172.21.15.187:6835/3236312344] up:boot 2023-08-11T14:01:13.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: fsmap 6 up:standby 2023-08-11T14:01:13.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:01:13.544 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:13 smithi187 systemd[1]: Starting Ceph mds.c for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:14.032 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:13 smithi187 podman[156542]: 2023-08-11T14:01:14.032 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:14 smithi187 ceph-mds[156582]: -- 172.21.15.187:0/990125596 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c3301b1e0 con 0x555c32ff0c00 2023-08-11T14:01:14.032 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: starting mds.c at 2023-08-11T14:01:14.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:14 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3656031545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:14.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:14 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3656031545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:14.294 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:14 smithi187 bash[156542]: ddd25877ae6edac13849c7d682561e54699446a859f5f84b78faa3ee752cedf2 2023-08-11T14:01:14.294 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:01:14 smithi187 systemd[1]: Started Ceph mds.c for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:14.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:14 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3656031545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:15.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:boot 2023-08-11T14:01:15.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: fsmap 7 up:standby 2023-08-11T14:01:15.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:01:15.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:15 smithi003 ceph-mon[97195]: Deploying daemon mds.h on smithi103 2023-08-11T14:01:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:boot 2023-08-11T14:01:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: fsmap 7 up:standby 2023-08-11T14:01:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:01:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:15.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:15 smithi187 ceph-mon[130973]: Deploying daemon mds.h on smithi103 2023-08-11T14:01:15.347 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:boot 2023-08-11T14:01:15.348 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: fsmap 7 up:standby 2023-08-11T14:01:15.348 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:01:15.348 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.349 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:15.350 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:15.351 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:15.351 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:15.351 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:15.351 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:15 smithi103 ceph-mon[130434]: Deploying daemon mds.h on smithi103 2023-08-11T14:01:15.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:15.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 -- 172.21.15.3:0/1056730574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c101b10 msgr2=0x7f126c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:15.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 --2- 172.21.15.3:0/1056730574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c101b10 0x7f126c101f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12540099c0 tx=0x7f1254009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 -- 172.21.15.3:0/1056730574 shutdown_connections 2023-08-11T14:01:15.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 --2- 172.21.15.3:0/1056730574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c1024b0 0x7f126c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 --2- 172.21.15.3:0/1056730574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c100920 0x7f126c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 --2- 172.21.15.3:0/1056730574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c101b10 0x7f126c101f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 -- 172.21.15.3:0/1056730574 >> 172.21.15.3:0/1056730574 conn(0x7f126c0fbe50 msgr2=0x7f126c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:15.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 -- 172.21.15.3:0/1056730574 shutdown_connections 2023-08-11T14:01:15.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.599+0000 7f12719ff700 1 -- 172.21.15.3:0/1056730574 wait complete. 2023-08-11T14:01:15.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 Processor -- start 2023-08-11T14:01:15.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 -- start start 2023-08-11T14:01:15.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c1024b0 0x7f126c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0739f0 con 0x7f126c100920 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c073b60 con 0x7f126c101b10 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f12719ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1a06d0 con 0x7f126c1024b0 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.600+0000 7f126affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:15.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:34968/0 (socket says 172.21.15.3:34968) 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f126affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49704/0 (socket says 172.21.15.3:49704) 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 -- 172.21.15.3:0/2185586911 learned_addr learned my addr 172.21.15.3:0/2185586911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f126b7fe700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c1024b0 0x7f126c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 -- 172.21.15.3:0/2185586911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c1024b0 msgr2=0x7f126c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c1024b0 0x7f126c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 -- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 msgr2=0x7f126c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 -- 172.21.15.3:0/2185586911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540095f0 con 0x7f126c101b10 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f126affd700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f1263fff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1254009990 tx=0x7f125400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.601+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125401d070 con 0x7f126c101b10 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.602+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1254005030 con 0x7f126c101b10 2023-08-11T14:01:15.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.602+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254004b30 con 0x7f126c101b10 2023-08-11T14:01:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.602+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c1a0950 con 0x7f126c101b10 2023-08-11T14:01:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.602+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c1a0d20 con 0x7f126c101b10 2023-08-11T14:01:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.603+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c107eb0 con 0x7f126c101b10 2023-08-11T14:01:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.605+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1254004cd0 con 0x7f126c101b10 2023-08-11T14:01:15.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.606+0000 7f12709fd700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 0x7f12580a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:15.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.607+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12540cbc90 con 0x7f126c101b10 2023-08-11T14:01:15.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.607+0000 7f126affd700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 0x7f12580a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:15.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.607+0000 7f126affd700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 0x7f12580a2460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f126c101970 tx=0x7f125c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:15.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.610+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1254095eb0 con 0x7f126c101b10 2023-08-11T14:01:15.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.866+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f126c10c180 con 0x7f126c101b10 2023-08-11T14:01:15.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.867+0000 7f12709fd700 1 -- 172.21.15.3:0/2185586911 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6074 (secure 0 0 0) 0x7f12540994d0 con 0x7f126c101b10 2023-08-11T14:01:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:15.869 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-08-11T14:01:15.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.869+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 msgr2=0x7f12580a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:15.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.869+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 0x7f12580a2460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f126c101970 tx=0x7f125c009260 comp rx=0 tx=0).stop 2023-08-11T14:01:15.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.869+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 msgr2=0x7f126c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.869+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1254009990 tx=0x7f125400f690 comp rx=0 tx=0).stop 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 shutdown_connections 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f126c1024b0 0x7f126c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f126c100920 0x7f126c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f125809ffa0 0x7f12580a2460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 --2- 172.21.15.3:0/2185586911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126c101b10 0x7f126c072d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:15.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 >> 172.21.15.3:0/2185586911 conn(0x7f126c0fbe50 msgr2=0x7f126c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 shutdown_connections 2023-08-11T14:01:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:15.870+0000 7f12719ff700 1 -- 172.21.15.3:0/2185586911 wait complete. 2023-08-11T14:01:15.873 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-08-11T14:01:15.929 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:15 smithi103 systemd[1]: Starting Ceph mds.h for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:16.223 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:16 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2185586911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:16.224 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:15 smithi103 podman[155247]: 2023-08-11T14:01:16.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:16 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2185586911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:16.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:16 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2185586911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:16.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:16 smithi103 ceph-mds[155288]: -- 172.21.15.103:0/3184295226 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562c40b5d1e0 con 0x562c40b32800 2023-08-11T14:01:16.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: starting mds.h at 2023-08-11T14:01:16.679 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:16 smithi103 bash[155247]: eb8c24be5d4230ac79535ef1e634d75f0174d46253230794988a1c9eba845875 2023-08-11T14:01:16.679 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:01:16 smithi103 systemd[1]: Started Ceph mds.h for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:17.341 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:17.342 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.342 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.342 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.342 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:17.342 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:17.343 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:17 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:17.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:17.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:17.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:17 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:17.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:17.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:17 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:17.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:18.044 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:17 smithi187 systemd[1]: Starting Ceph mds.i for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 -- 172.21.15.3:0/4232456432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028101b20 msgr2=0x7f4028101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4232456432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028101b20 0x7f4028101f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f401c009920 tx=0x7f401c009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:18.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 -- 172.21.15.3:0/4232456432 shutdown_connections 2023-08-11T14:01:18.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4232456432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40281024c0 0x7f4028108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4232456432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4028100930 0x7f4028100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4232456432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028101b20 0x7f4028101f80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 -- 172.21.15.3:0/4232456432 >> 172.21.15.3:0/4232456432 conn(0x7f40280fbe60 msgr2=0x7f40280fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 -- 172.21.15.3:0/4232456432 shutdown_connections 2023-08-11T14:01:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.106+0000 7f402e7ff700 1 -- 172.21.15.3:0/4232456432 wait complete. 2023-08-11T14:01:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 Processor -- start 2023-08-11T14:01:18.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 -- start start 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028100930 0x7f40280746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 0x7f4028072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402819a530 con 0x7f4028100930 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028073790 con 0x7f4028101b20 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.107+0000 7f402e7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028073950 con 0x7f40281024c0 2023-08-11T14:01:18.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f4027fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028100930 0x7f40280746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 0x7f4028072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f402cd9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60196/0 (socket says 172.21.15.3:60196) 2023-08-11T14:01:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f402cd9c700 1 -- 172.21.15.3:0/4199145404 learned_addr learned my addr 172.21.15.3:0/4199145404 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:18.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 -- 172.21.15.3:0/4199145404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 msgr2=0x7f4028073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:18.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 -- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028100930 msgr2=0x7f40280746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:18.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028100930 0x7f40280746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 -- 172.21.15.3:0/4199145404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c009520 con 0x7f4028101b20 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f402cd9c700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.108+0000 7f40277fe700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 0x7f4028072d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.109+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c01c070 con 0x7f4028101b20 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.109+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40281a4ee0 con 0x7f4028101b20 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.109+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c00f7d0 con 0x7f4028101b20 2023-08-11T14:01:18.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.109+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0038e0 con 0x7f4028101b20 2023-08-11T14:01:18.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.109+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40281a5250 con 0x7f4028101b20 2023-08-11T14:01:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.110+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4028107f10 con 0x7f4028101b20 2023-08-11T14:01:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.112+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f401c020030 con 0x7f4028101b20 2023-08-11T14:01:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.113+0000 7f40257fa700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 0x7f40100a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:18.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.113+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f401c0cc410 con 0x7f4028101b20 2023-08-11T14:01:18.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.114+0000 7f4027fff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 0x7f40100a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:18.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.114+0000 7f4027fff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 0x7f40100a2460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4028101980 tx=0x7f4018009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:18.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.116+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f401c096630 con 0x7f4028101b20 2023-08-11T14:01:18.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:18 smithi003 ceph-mon[97195]: Deploying daemon mds.i on smithi187 2023-08-11T14:01:18.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:18 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:boot 2023-08-11T14:01:18.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:18 smithi003 ceph-mon[97195]: fsmap 8 up:standby 2023-08-11T14:01:18.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:01:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.376+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f402810c190 con 0x7f4028101b20 2023-08-11T14:01:18.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.377+0000 7f40257fa700 1 -- 172.21.15.3:0/4199145404 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6868 (secure 0 0 0) 0x7f401c099c50 con 0x7f4028101b20 2023-08-11T14:01:18.379 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:18.379 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-08-11T14:01:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 msgr2=0x7f40100a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 0x7f40100a2460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4028101980 tx=0x7f4018009310 comp rx=0 tx=0).stop 2023-08-11T14:01:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 msgr2=0x7f4028072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 0x7f4028072d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f401c000c00 tx=0x7f401c00b790 comp rx=0 tx=0).stop 2023-08-11T14:01:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 shutdown_connections 2023-08-11T14:01:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40281024c0 0x7f4028073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4028100930 0x7f40280746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f401009ffa0 0x7f40100a2460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 --2- 172.21.15.3:0/4199145404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4028101b20 0x7f4028072d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 >> 172.21.15.3:0/4199145404 conn(0x7f40280fbe60 msgr2=0x7f4028105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 shutdown_connections 2023-08-11T14:01:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:18.380+0000 7f402e7ff700 1 -- 172.21.15.3:0/4199145404 wait complete. 2023-08-11T14:01:18.384 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2023-08-11T14:01:18.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:18 smithi103 ceph-mon[130434]: Deploying daemon mds.i on smithi187 2023-08-11T14:01:18.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:18 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:boot 2023-08-11T14:01:18.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:18 smithi103 ceph-mon[130434]: fsmap 8 up:standby 2023-08-11T14:01:18.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:01:18.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-mon[130973]: Deploying daemon mds.i on smithi187 2023-08-11T14:01:18.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:boot 2023-08-11T14:01:18.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-mon[130973]: fsmap 8 up:standby 2023-08-11T14:01:18.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:01:18.465 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:18 smithi187 podman[156885]: 2023-08-11T14:01:18.794 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-mds[156925]: -- 172.21.15.187:0/1575447768 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55efcff811e0 con 0x55efcff57000 2023-08-11T14:01:18.794 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:18 smithi187 bash[156885]: debec6b5e694f828f240cd9e0b3f2cfb344a5400a9c4577828a25502c849b63a 2023-08-11T14:01:18.794 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: starting mds.i at 2023-08-11T14:01:18.794 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:01:18 smithi187 systemd[1]: Started Ceph mds.i for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:19.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:19.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4199145404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:19.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:19.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:19.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4199145404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:19.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4199145404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:20.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:20.468 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:20 smithi003 ceph-mon[97195]: Deploying daemon mds.g on smithi003 2023-08-11T14:01:20.469 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:20 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:boot 2023-08-11T14:01:20.469 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:20 smithi003 ceph-mon[97195]: fsmap 9 up:standby 2023-08-11T14:01:20.469 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:20 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:01:20.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:20 smithi187 ceph-mon[130973]: Deploying daemon mds.g on smithi003 2023-08-11T14:01:20.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:20 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:boot 2023-08-11T14:01:20.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:20 smithi187 ceph-mon[130973]: fsmap 9 up:standby 2023-08-11T14:01:20.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:20 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 -- 172.21.15.3:0/1289933864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 msgr2=0x7f7138100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 --2- 172.21.15.3:0/1289933864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138100e50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7124009920 tx=0x7f7124009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 -- 172.21.15.3:0/1289933864 shutdown_connections 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 --2- 172.21.15.3:0/1289933864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101390 0x7f7138107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 --2- 172.21.15.3:0/1289933864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71380ff800 0x7f71380ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 --2- 172.21.15.3:0/1289933864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138100e50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 -- 172.21.15.3:0/1289933864 >> 172.21.15.3:0/1289933864 conn(0x7f71380fad50 msgr2=0x7f71380fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:20.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 -- 172.21.15.3:0/1289933864 shutdown_connections 2023-08-11T14:01:20.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.631+0000 7f713ebff700 1 -- 172.21.15.3:0/1289933864 wait complete. 2023-08-11T14:01:20.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.632+0000 7f713ebff700 1 Processor -- start 2023-08-11T14:01:20.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 -- start start 2023-08-11T14:01:20.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71380ff800 0x7f7138198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 0x7f713819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71381a0410 con 0x7f7138101390 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71381a0550 con 0x7f71381009f0 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713ebff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71381a06e0 con 0x7f71380ff800 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f7137fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713d19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 0x7f713819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:20.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f713c99b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71380ff800 0x7f7138198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f7137fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:34996/0 (socket says 172.21.15.3:34996) 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.633+0000 7f7137fff700 1 -- 172.21.15.3:0/729134406 learned_addr learned my addr 172.21.15.3:0/729134406 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 -- 172.21.15.3:0/729134406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71380ff800 msgr2=0x7f7138198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71380ff800 0x7f7138198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 -- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 msgr2=0x7f713819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 0x7f713819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 -- 172.21.15.3:0/729134406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124009520 con 0x7f71381009f0 2023-08-11T14:01:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f713d19c700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 0x7f713819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7137fff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7124009230 tx=0x7f712400bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:20.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712401c070 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7124005040 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381a0960 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240039b0 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.634+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381a0d80 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.635+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713810b060 con 0x7f71381009f0 2023-08-11T14:01:20.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.637+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7124020030 con 0x7f71381009f0 2023-08-11T14:01:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.639+0000 7f7135ffb700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 0x7f71280a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.639+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71240cbe30 con 0x7f71381009f0 2023-08-11T14:01:20.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.639+0000 7f713c99b700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 0x7f71280a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:20.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.639+0000 7f713c99b700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 0x7f71280a24e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7138100850 tx=0x7f712c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:20.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.641+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f71240960e0 con 0x7f71381009f0 2023-08-11T14:01:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:20 smithi103 ceph-mon[130434]: Deploying daemon mds.g on smithi003 2023-08-11T14:01:20.766 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:20 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:boot 2023-08-11T14:01:20.766 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:20 smithi103 ceph-mon[130434]: fsmap 9 up:standby 2023-08-11T14:01:20.766 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:20 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:01:20.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.902+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f713810b340 con 0x7f71381009f0 2023-08-11T14:01:20.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.903+0000 7f7135ffb700 1 -- 172.21.15.3:0/729134406 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7664 (secure 0 0 0) 0x7f713810b340 con 0x7f71381009f0 2023-08-11T14:01:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:20.905 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-11T14:01:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 msgr2=0x7f71280a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 0x7f71280a24e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7138100850 tx=0x7f712c009310 comp rx=0 tx=0).stop 2023-08-11T14:01:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 msgr2=0x7f7138198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7124009230 tx=0x7f712400bf80 comp rx=0 tx=0).stop 2023-08-11T14:01:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 shutdown_connections 2023-08-11T14:01:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7138101390 0x7f713819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71380ff800 0x7f7138198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f71280a0020 0x7f71280a24e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 --2- 172.21.15.3:0/729134406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381009f0 0x7f7138198dd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:20.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 >> 172.21.15.3:0/729134406 conn(0x7f71380fad50 msgr2=0x7f7138067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:20.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 shutdown_connections 2023-08-11T14:01:20.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:20.906+0000 7f713ebff700 1 -- 172.21.15.3:0/729134406 wait complete. 2023-08-11T14:01:20.910 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 10 2023-08-11T14:01:21.253 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:20 smithi003 systemd[1]: Starting Ceph mds.g for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:21.541 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:21 smithi003 podman[131564]: 2023-08-11T14:01:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:21 smithi003 ceph-mon[97195]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:21 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/729134406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:21.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:21 smithi187 ceph-mon[130973]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:21.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:21 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/729134406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:21 smithi103 ceph-mon[130434]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:21 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/729134406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:22.042 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:21 smithi003 bash[131564]: 80e7fc13fac6f59b7e5a89c60969f7e045f96995b1bd766830caf7973f17300a 2023-08-11T14:01:22.042 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:21 smithi003 ceph-mds[131604]: -- 172.21.15.3:0/3707001576 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55704521f1e0 con 0x5570451f5000 2023-08-11T14:01:22.042 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:21 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: starting mds.g at 2023-08-11T14:01:22.042 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:01:21 smithi003 systemd[1]: Started Ceph mds.g for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:22.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:22.997 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:22.997 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: Deploying daemon mds.a on smithi003 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:22.998 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6838/3729253320,v1:172.21.15.3:6839/3729253320] up:boot 2023-08-11T14:01:22.999 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: fsmap 10 up:standby 2023-08-11T14:01:22.999 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:22 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:01:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: Deploying daemon mds.a on smithi003 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6838/3729253320,v1:172.21.15.3:6839/3729253320] up:boot 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: fsmap 10 up:standby 2023-08-11T14:01:23.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:22 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:01:23.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 -- 172.21.15.3:0/471636990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1364101b10 msgr2=0x7f1364101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:23.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 --2- 172.21.15.3:0/471636990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1364101b10 0x7f1364101f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f13580099c0 tx=0x7f1358009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 -- 172.21.15.3:0/471636990 shutdown_connections 2023-08-11T14:01:23.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 --2- 172.21.15.3:0/471636990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13641024b0 0x7f1364108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 --2- 172.21.15.3:0/471636990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 0x7f1364100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 --2- 172.21.15.3:0/471636990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1364101b10 0x7f1364101f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 -- 172.21.15.3:0/471636990 >> 172.21.15.3:0/471636990 conn(0x7f13640fbe50 msgr2=0x7f13640fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:23.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 -- 172.21.15.3:0/471636990 shutdown_connections 2023-08-11T14:01:23.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.148+0000 7f1368bff700 1 -- 172.21.15.3:0/471636990 wait complete. 2023-08-11T14:01:23.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 Processor -- start 2023-08-11T14:01:23.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 -- start start 2023-08-11T14:01:23.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 0x7f1364196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:23.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1364101b10 0x7f1364196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:23.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:23.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136419e1f0 con 0x7f13641024b0 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136419e330 con 0x7f1364101b10 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.149+0000 7f1368bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136419e4c0 con 0x7f1364100920 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45766/0 (socket says 172.21.15.3:45766) 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 -- 172.21.15.3:0/21515755 learned_addr learned my addr 172.21.15.3:0/21515755 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f136259c700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 0x7f1364196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:23.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1361d9b700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1364101b10 0x7f1364196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 -- 172.21.15.3:0/21515755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 msgr2=0x7f1364196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 0x7f1364196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 -- 172.21.15.3:0/21515755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1364101b10 msgr2=0x7f1364196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1364101b10 0x7f1364196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 -- 172.21.15.3:0/21515755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580095f0 con 0x7f13641024b0 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.150+0000 7f1362d9d700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f135c007ee0 tx=0x7f135c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:23.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.151+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c017070 con 0x7f13641024b0 2023-08-11T14:01:23.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.151+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136419e7a0 con 0x7f13641024b0 2023-08-11T14:01:23.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.151+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f135c00f040 con 0x7f13641024b0 2023-08-11T14:01:23.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.151+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c005580 con 0x7f13641024b0 2023-08-11T14:01:23.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.151+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136419ecb0 con 0x7f13641024b0 2023-08-11T14:01:23.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.152+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1364107eb0 con 0x7f13641024b0 2023-08-11T14:01:23.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.154+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f135c00db20 con 0x7f13641024b0 2023-08-11T14:01:23.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.155+0000 7f13537fe700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 0x7f134c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:23.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.155+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f135c0ca540 con 0x7f13641024b0 2023-08-11T14:01:23.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.155+0000 7f136259c700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 0x7f134c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:23.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.156+0000 7f136259c700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 0x7f134c0a2460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1364101970 tx=0x7f135400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:23.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.159+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f135c0946d0 con 0x7f13641024b0 2023-08-11T14:01:23.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:23.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: Deploying daemon mds.a on smithi003 2023-08-11T14:01:23.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:23.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6838/3729253320,v1:172.21.15.3:6839/3729253320] up:boot 2023-08-11T14:01:23.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: fsmap 10 up:standby 2023-08-11T14:01:23.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:22 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:01:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.425+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f136410c350 con 0x7f13641024b0 2023-08-11T14:01:23.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.426+0000 7f13537fe700 1 -- 172.21.15.3:0/21515755 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8453 (secure 0 0 0) 0x7f135c097cf0 con 0x7f13641024b0 2023-08-11T14:01:23.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:23.428 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-11T14:01:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.428+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 msgr2=0x7f134c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.428+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 0x7f134c0a2460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1364101970 tx=0x7f135400b410 comp rx=0 tx=0).stop 2023-08-11T14:01:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.428+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 msgr2=0x7f136419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.428+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f135c007ee0 tx=0x7f135c00d3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 shutdown_connections 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13641024b0 0x7f136419dc20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1364100920 0x7f1364196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f134c09ffa0 0x7f134c0a2460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 --2- 172.21.15.3:0/21515755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1364101b10 0x7f1364196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 >> 172.21.15.3:0/21515755 conn(0x7f13640fbe50 msgr2=0x7f1364104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 shutdown_connections 2023-08-11T14:01:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:23.429+0000 7f1368bff700 1 -- 172.21.15.3:0/21515755 wait complete. 2023-08-11T14:01:23.432 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 11 2023-08-11T14:01:24.065 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:23 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/21515755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:24.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:23 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/21515755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:23 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/21515755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:24.538 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 systemd[1]: Starting Ceph mds.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:24.791 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 podman[131951]: 2023-08-11T14:01:25.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:24 smithi103 ceph-mon[130434]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:25.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:24 smithi003 ceph-mon[97195]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:25.292 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 bash[131951]: 498c6e8d77493d63d6dcabfaf5df5f2858618cdef5a1c1c9473aa2dfa834b9c1 2023-08-11T14:01:25.292 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 ceph-mds[131991]: -- 172.21.15.3:0/1898169405 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c1eb62b1e0 con 0x55c1eb600800 2023-08-11T14:01:25.293 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: starting mds.a at 2023-08-11T14:01:25.293 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:01:24 smithi003 systemd[1]: Started Ceph mds.a for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:25.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:24 smithi187 ceph-mon[130973]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:25.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 -- 172.21.15.3:0/2836231214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 msgr2=0x7f2478100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 --2- 172.21.15.3:0/2836231214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478100d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2468009920 tx=0x7f2468009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 -- 172.21.15.3:0/2836231214 shutdown_connections 2023-08-11T14:01:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 --2- 172.21.15.3:0/2836231214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24781024b0 0x7f2478108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 --2- 172.21.15.3:0/2836231214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478100d40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 --2- 172.21.15.3:0/2836231214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2478101b10 0x7f2478101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 -- 172.21.15.3:0/2836231214 >> 172.21.15.3:0/2836231214 conn(0x7f24780fbe50 msgr2=0x7f24780fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 -- 172.21.15.3:0/2836231214 shutdown_connections 2023-08-11T14:01:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.660+0000 7f247dbff700 1 -- 172.21.15.3:0/2836231214 wait complete. 2023-08-11T14:01:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.661+0000 7f247dbff700 1 Processor -- start 2023-08-11T14:01:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 -- start start 2023-08-11T14:01:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2478101b10 0x7f2478198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 0x7f247819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a04a0 con 0x7f2478101b10 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a05e0 con 0x7f2478100920 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f247dbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a0770 con 0x7f24781024b0 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44896/0 (socket says 172.21.15.3:44896) 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 -- 172.21.15.3:0/492284130 learned_addr learned my addr 172.21.15.3:0/492284130 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f246ffff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2478101b10 0x7f2478198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f2477fff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 0x7f247819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 -- 172.21.15.3:0/492284130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 msgr2=0x7f247819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 0x7f247819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 -- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2478101b10 msgr2=0x7f2478198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2478101b10 0x7f2478198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.662+0000 7f24777fe700 1 -- 172.21.15.3:0/492284130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2468009520 con 0x7f2478100920 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f2477fff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 0x7f247819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:25.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f24777fe700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2468009230 tx=0x7f246800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246801c070 con 0x7f2478100920 2023-08-11T14:01:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246800f7d0 con 0x7f2478100920 2023-08-11T14:01:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24781a09f0 con 0x7f2478100920 2023-08-11T14:01:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468004990 con 0x7f2478100920 2023-08-11T14:01:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.663+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a0e10 con 0x7f2478100920 2023-08-11T14:01:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.664+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2478192f70 con 0x7f2478100920 2023-08-11T14:01:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.666+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2468020070 con 0x7f2478100920 2023-08-11T14:01:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.667+0000 7f24757fa700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 0x7f24640a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.667+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f24680cbc80 con 0x7f2478100920 2023-08-11T14:01:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.668+0000 7f246ffff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 0x7f24640a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.668+0000 7f246ffff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 0x7f24640a24e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2460009c60 tx=0x7f2460009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.669+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2468095ea0 con 0x7f2478100920 2023-08-11T14:01:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.933+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f247802cb70 con 0x7f2478100920 2023-08-11T14:01:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.934+0000 7f24757fa700 1 -- 172.21.15.3:0/492284130 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9242 (secure 0 0 0) 0x7f24680994c0 con 0x7f2478100920 2023-08-11T14:01:25.935 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:25.936 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-08-11T14:01:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 msgr2=0x7f24640a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 0x7f24640a24e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2460009c60 tx=0x7f2460009260 comp rx=0 tx=0).stop 2023-08-11T14:01:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 msgr2=0x7f2478198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:25.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2468009230 tx=0x7f246800b790 comp rx=0 tx=0).stop 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 shutdown_connections 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24781024b0 0x7f247819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24640a0020 0x7f24640a24e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2478100920 0x7f2478198920 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 --2- 172.21.15.3:0/492284130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2478101b10 0x7f2478198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 >> 172.21.15.3:0/492284130 conn(0x7f24780fbe50 msgr2=0x7f24781074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 shutdown_connections 2023-08-11T14:01:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:25.936+0000 7f247dbff700 1 -- 172.21.15.3:0/492284130 wait complete. 2023-08-11T14:01:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2023-08-11T14:01:26.270 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:26.271 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: Deploying daemon mds.b on smithi103 2023-08-11T14:01:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6840/1195165407,v1:172.21.15.3:6841/1195165407] up:boot 2023-08-11T14:01:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: fsmap 11 up:standby 2023-08-11T14:01:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:01:26.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:26 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/492284130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:26.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:26.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: Deploying daemon mds.b on smithi103 2023-08-11T14:01:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6840/1195165407,v1:172.21.15.3:6841/1195165407] up:boot 2023-08-11T14:01:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: fsmap 11 up:standby 2023-08-11T14:01:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:01:26.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:26 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/492284130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:26.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:26.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-08-11T14:01:26.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: Deploying daemon mds.b on smithi103 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6840/1195165407,v1:172.21.15.3:6841/1195165407] up:boot 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: fsmap 11 up:standby 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:01:26.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:26 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/492284130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:26.678 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:26 smithi103 systemd[1]: Starting Ceph mds.b for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:01:27.090 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:27 smithi103 ceph-mon[130434]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:27.091 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:26 smithi103 podman[155586]: 2023-08-11T14:01:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:27 smithi003 ceph-mon[97195]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:27.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:27 smithi187 ceph-mon[130973]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:27.428 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:27 smithi103 ceph-mds[155627]: -- 172.21.15.103:0/464238384 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2888251e0 con 0x55b2887fac00 2023-08-11T14:01:27.429 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: starting mds.b at 2023-08-11T14:01:27.429 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:27 smithi103 bash[155586]: ece46451161c27b4c1cef42ba999019a3aa72259ae13c42cc39b8d413b1183c7 2023-08-11T14:01:27.429 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:27 smithi103 systemd[1]: Started Ceph mds.b for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:01:27.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:28.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 -- 172.21.15.3:0/552281855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0100920 msgr2=0x7facf0100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 --2- 172.21.15.3:0/552281855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0100920 0x7facf0100d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facec009990 tx=0x7facec009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 -- 172.21.15.3:0/552281855 shutdown_connections 2023-08-11T14:01:28.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 --2- 172.21.15.3:0/552281855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf01024b0 0x7facf0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 --2- 172.21.15.3:0/552281855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0100920 0x7facf0100d40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 --2- 172.21.15.3:0/552281855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf0101b10 0x7facf0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 -- 172.21.15.3:0/552281855 >> 172.21.15.3:0/552281855 conn(0x7facf00fbe50 msgr2=0x7facf00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:28.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 -- 172.21.15.3:0/552281855 shutdown_connections 2023-08-11T14:01:28.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.182+0000 7facf7bff700 1 -- 172.21.15.3:0/552281855 wait complete. 2023-08-11T14:01:28.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.183+0000 7facf7bff700 1 Processor -- start 2023-08-11T14:01:28.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.183+0000 7facf7bff700 1 -- start start 2023-08-11T14:01:28.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf01024b0 0x7facf019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf01a0390 con 0x7facf0100920 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf01a04d0 con 0x7facf01024b0 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf7bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf01a0660 con 0x7facf0101b10 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf599b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf599b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45822/0 (socket says 172.21.15.3:45822) 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33998/0 (socket says 172.21.15.3:33998) 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf599b700 1 -- 172.21.15.3:0/2650296116 learned_addr learned my addr 172.21.15.3:0/2650296116 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:28.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf619c700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf01024b0 0x7facf019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 -- 172.21.15.3:0/2650296116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf01024b0 msgr2=0x7facf019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf01024b0 0x7facf019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 -- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 msgr2=0x7facf0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf519a700 1 -- 172.21.15.3:0/2650296116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0095f0 con 0x7facf0101b10 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.184+0000 7facf599b700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:28.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7facf519a700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7face000e9f0 tx=0x7face000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face000c9c0 con 0x7facf0101b10 2023-08-11T14:01:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face0010430 con 0x7facf0101b10 2023-08-11T14:01:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0018680 con 0x7facf0101b10 2023-08-11T14:01:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf01a0940 con 0x7facf0101b10 2023-08-11T14:01:28.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.185+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf01a0e00 con 0x7facf0101b10 2023-08-11T14:01:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.186+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf0192f70 con 0x7facf0101b10 2023-08-11T14:01:28.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.188+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7face0010860 con 0x7facf0101b10 2023-08-11T14:01:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.189+0000 7face6ffd700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 0x7facdc0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.190+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7face0014070 con 0x7facf0101b10 2023-08-11T14:01:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.190+0000 7facf599b700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 0x7facdc0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.190+0000 7facf599b700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 0x7facdc0a2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7facec00b2e0 tx=0x7facec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:28.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.193+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7face0051c30 con 0x7facf0101b10 2023-08-11T14:01:28.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:28.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:boot 2023-08-11T14:01:28.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: fsmap 12 up:standby 2023-08-11T14:01:28.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:28 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:01:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.464+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facf002cdc0 con 0x7facf0101b10 2023-08-11T14:01:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.465+0000 7face6ffd700 1 -- 172.21.15.3:0/2650296116 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10037 (secure 0 0 0) 0x7face00950e0 con 0x7facf0101b10 2023-08-11T14:01:28.467 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:28.467 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-08-11T14:01:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 msgr2=0x7facdc0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 0x7facdc0a2460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7facec00b2e0 tx=0x7facec005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 msgr2=0x7facf0198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7face000e9f0 tx=0x7face000ed00 comp rx=0 tx=0).stop 2023-08-11T14:01:28.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 shutdown_connections 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7facdc09ffa0 0x7facdc0a2460 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf01024b0 0x7facf019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7facf0100920 0x7facf0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 --2- 172.21.15.3:0/2650296116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf0101b10 0x7facf0198d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 >> 172.21.15.3:0/2650296116 conn(0x7facf00fbe50 msgr2=0x7facf0107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 shutdown_connections 2023-08-11T14:01:28.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.468+0000 7facf7bff700 1 -- 172.21.15.3:0/2650296116 wait complete. 2023-08-11T14:01:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2023-08-11T14:01:28.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:28.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:28.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:boot 2023-08-11T14:01:28.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: fsmap 12 up:standby 2023-08-11T14:01:28.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:28 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:01:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:boot 2023-08-11T14:01:28.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: fsmap 12 up:standby 2023-08-11T14:01:28.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:28 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:01:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 -- 172.21.15.3:0/2231768853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c1087e0 msgr2=0x7f996c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 --2- 172.21.15.3:0/2231768853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c1087e0 0x7f996c10ac70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f996400b0a0 tx=0x7f996400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 -- 172.21.15.3:0/2231768853 shutdown_connections 2023-08-11T14:01:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 --2- 172.21.15.3:0/2231768853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c1087e0 0x7f996c10ac70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 --2- 172.21.15.3:0/2231768853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c074960 0x7f996c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 --2- 172.21.15.3:0/2231768853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c073380 0x7f996c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 -- 172.21.15.3:0/2231768853 >> 172.21.15.3:0/2231768853 conn(0x7f996c0fbe50 msgr2=0x7f996c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 -- 172.21.15.3:0/2231768853 shutdown_connections 2023-08-11T14:01:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.696+0000 7f9972fff700 1 -- 172.21.15.3:0/2231768853 wait complete. 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.697+0000 7f9972fff700 1 Processor -- start 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 -- start start 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c074960 0x7f996c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c1087e0 0x7f996c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1a0490 con 0x7f996c1087e0 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1a05d0 con 0x7f996c074960 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9972fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1a0760 con 0x7f996c073380 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34028/0 (socket says 172.21.15.3:34028) 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 -- 172.21.15.3:0/1411820686 learned_addr learned my addr 172.21.15.3:0/1411820686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f996bfff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c074960 0x7f996c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f997159c700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c1087e0 0x7f996c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 -- 172.21.15.3:0/1411820686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c074960 msgr2=0x7f996c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c074960 0x7f996c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 -- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c1087e0 msgr2=0x7f996c19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c1087e0 0x7f996c19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.698+0000 7f9970d9b700 1 -- 172.21.15.3:0/1411820686 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009d00 con 0x7f996c073380 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9970d9b700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f996000e9a0 tx=0x7f996000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960009790 con 0x7f996c073380 2023-08-11T14:01:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c1a0a40 con 0x7f996c073380 2023-08-11T14:01:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9960010430 con 0x7f996c073380 2023-08-11T14:01:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960018680 con 0x7f996c073380 2023-08-11T14:01:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.699+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c1a0f50 con 0x7f996c073380 2023-08-11T14:01:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.700+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c106e90 con 0x7f996c073380 2023-08-11T14:01:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.702+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f99600105a0 con 0x7f996c073380 2023-08-11T14:01:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.703+0000 7f9969ffb700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 0x7f995c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.704+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9960014070 con 0x7f996c073380 2023-08-11T14:01:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.704+0000 7f996bfff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 0x7f995c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.704+0000 7f996bfff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 0x7f995c0a2460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f996c074680 tx=0x7f9958009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.707+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9960051c30 con 0x7f996c073380 2023-08-11T14:01:28.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.951+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f996c10e360 con 0x7f996c073380 2023-08-11T14:01:28.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.953+0000 7f9969ffb700 1 -- 172.21.15.3:0/1411820686 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18376 (secure 0 0 0) 0x7f9960095180 con 0x7f996c073380 2023-08-11T14:01:28.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:28.955 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:00:45.467682+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 msgr2=0x7f995c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 0x7f995c0a2460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f996c074680 tx=0x7f9958009260 comp rx=0 tx=0).stop 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 msgr2=0x7f996c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f996000e9a0 tx=0x7f996000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 shutdown_connections 2023-08-11T14:01:28.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c1087e0 0x7f996c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f996c073380 0x7f996c198910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f995c09ffa0 0x7f995c0a2460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 --2- 172.21.15.3:0/1411820686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f996c074960 0x7f996c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 >> 172.21.15.3:0/1411820686 conn(0x7f996c0fbe50 msgr2=0x7f996c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 shutdown_connections 2023-08-11T14:01:28.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:28.955+0000 7f9972fff700 1 -- 172.21.15.3:0/1411820686 wait complete. 2023-08-11T14:01:28.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-11T14:01:28.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:01:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 -- 172.21.15.3:0/3824783048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 msgr2=0x7fbb40072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 --2- 172.21.15.3:0/3824783048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb40072db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009990 tx=0x7fbb3c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 -- 172.21.15.3:0/3824783048 shutdown_connections 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 --2- 172.21.15.3:0/3824783048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb4010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 --2- 172.21.15.3:0/3824783048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb40072db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 --2- 172.21.15.3:0/3824783048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb40073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 -- 172.21.15.3:0/3824783048 >> 172.21.15.3:0/3824783048 conn(0x7fbb400fbe60 msgr2=0x7fbb400fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:29.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 -- 172.21.15.3:0/3824783048 shutdown_connections 2023-08-11T14:01:29.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.204+0000 7fbb47bff700 1 -- 172.21.15.3:0/3824783048 wait complete. 2023-08-11T14:01:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.205+0000 7fbb47bff700 1 Processor -- start 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.205+0000 7fbb47bff700 1 -- start start 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.205+0000 7fbb47bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb47bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb4019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb47bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb401a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb47bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb401a4880 con 0x7fbb40074960 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb47bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb401a49f0 con 0x7fbb40073380 2023-08-11T14:01:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb47bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb401a4bb0 con 0x7fbb4010cee0 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44942/0 (socket says 172.21.15.3:44942) 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4599b700 1 -- 172.21.15.3:0/2401577682 learned_addr learned my addr 172.21.15.3:0/2401577682 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb401a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 -- 172.21.15.3:0/2401577682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 msgr2=0x7fbb4019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4519a700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb4019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 -- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 msgr2=0x7fbb4019d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb4019d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4619c700 1 -- 172.21.15.3:0/2401577682 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0095f0 con 0x7fbb4010cee0 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4599b700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.206+0000 7fbb4519a700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb4019d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb4619c700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb401a42b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800efc0 tx=0x7fbb3800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38018070 con 0x7fbb4010cee0 2023-08-11T14:01:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb401a4e30 con 0x7fbb4010cee0 2023-08-11T14:01:29.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3800f040 con 0x7fbb4010cee0 2023-08-11T14:01:29.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800cdf0 con 0x7fbb4010cee0 2023-08-11T14:01:29.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.207+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb401a5280 con 0x7fbb4010cee0 2023-08-11T14:01:29.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.208+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb40102a30 con 0x7fbb4010cee0 2023-08-11T14:01:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.210+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbb38003a00 con 0x7fbb4010cee0 2023-08-11T14:01:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.211+0000 7fbb36ffd700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 0x7fbb2c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.211+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbb380caab0 con 0x7fbb4010cee0 2023-08-11T14:01:29.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.212+0000 7fbb4599b700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 0x7fbb2c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.212+0000 7fbb4599b700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 0x7fbb2c0a2410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b2e0 tx=0x7fbb3c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:29.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.215+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbb38094cd0 con 0x7fbb4010cee0 2023-08-11T14:01:29.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.441+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb4002cdc0 con 0x7fbb4010cee0 2023-08-11T14:01:29.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.442+0000 7fbb36ffd700 1 -- 172.21.15.3:0/2401577682 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb3801b320 con 0x7fbb4010cee0 2023-08-11T14:01:29.443 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:01:29.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 msgr2=0x7fbb2c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 0x7fbb2c0a2410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b2e0 tx=0x7fbb3c005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 msgr2=0x7fbb401a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb401a42b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800efc0 tx=0x7fbb3800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 shutdown_connections 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb4010cee0 0x7fbb401a42b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb2c09ff50 0x7fbb2c0a2410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb40073380 0x7fbb4019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 --2- 172.21.15.3:0/2401577682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40074960 0x7fbb4019d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 >> 172.21.15.3:0/2401577682 conn(0x7fbb400fbe60 msgr2=0x7fbb400fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 shutdown_connections 2023-08-11T14:01:29.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.444+0000 7fbb47bff700 1 -- 172.21.15.3:0/2401577682 wait complete. 2023-08-11T14:01:29.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-11T14:01:29.532 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:29 smithi003 ceph-mon[97195]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:29.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:29 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2650296116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:29 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1411820686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:29 smithi187 ceph-mon[130973]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:29.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:29 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2650296116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:29 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1411820686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:29 smithi103 ceph-mon[130434]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:29 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2650296116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:29 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1411820686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 -- 172.21.15.3:0/1264680641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230074960 msgr2=0x7fb230072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 --2- 172.21.15.3:0/1264680641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230074960 0x7fb230072db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009990 tx=0x7fb22c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 -- 172.21.15.3:0/1264680641 shutdown_connections 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 --2- 172.21.15.3:0/1264680641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 0x7fb23010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 --2- 172.21.15.3:0/1264680641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230074960 0x7fb230072db0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 --2- 172.21.15.3:0/1264680641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230073380 0x7fb230073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 -- 172.21.15.3:0/1264680641 >> 172.21.15.3:0/1264680641 conn(0x7fb2300fbe50 msgr2=0x7fb2300fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:29.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 -- 172.21.15.3:0/1264680641 shutdown_connections 2023-08-11T14:01:29.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.679+0000 7fb2373ff700 1 -- 172.21.15.3:0/1264680641 wait complete. 2023-08-11T14:01:29.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.681+0000 7fb2373ff700 1 Processor -- start 2023-08-11T14:01:29.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.681+0000 7fb2373ff700 1 -- start start 2023-08-11T14:01:29.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 0x7fb230198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 0x7fb23019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301a0380 con 0x7fb230074960 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301a04c0 con 0x7fb2301087e0 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb2373ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301a0650 con 0x7fb230073380 2023-08-11T14:01:29.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34050/0 (socket says 172.21.15.3:34050) 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 -- 172.21.15.3:0/990034688 learned_addr learned my addr 172.21.15.3:0/990034688 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23499a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 0x7fb230198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23599c700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 0x7fb23019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 -- 172.21.15.3:0/990034688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 msgr2=0x7fb23019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 0x7fb23019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 -- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 msgr2=0x7fb230198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 0x7fb230198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23519b700 1 -- 172.21.15.3:0/990034688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220009520 con 0x7fb230073380 2023-08-11T14:01:29.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.682+0000 7fb23499a700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 0x7fb230198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:29.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb23519b700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb22c004740 tx=0x7fb22c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:29.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c01d070 con 0x7fb230073380 2023-08-11T14:01:29.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22c004830 con 0x7fb230073380 2023-08-11T14:01:29.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0053c0 con 0x7fb230073380 2023-08-11T14:01:29.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c0095f0 con 0x7fb230073380 2023-08-11T14:01:29.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.683+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2301a0b00 con 0x7fb230073380 2023-08-11T14:01:29.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.684+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230106e90 con 0x7fb230073380 2023-08-11T14:01:29.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.686+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb22c005580 con 0x7fb230073380 2023-08-11T14:01:29.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.687+0000 7fb2267fc700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 0x7fb21c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:29.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.687+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb22c0cbfa0 con 0x7fb230073380 2023-08-11T14:01:29.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.688+0000 7fb23499a700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 0x7fb21c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:29.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.688+0000 7fb23499a700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 0x7fb21c0a24e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb220009fd0 tx=0x7fb220009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:29.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.689+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb22c0961c0 con 0x7fb230073380 2023-08-11T14:01:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.916+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb23010e360 con 0x7fb230073380 2023-08-11T14:01:29.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.916+0000 7fb2267fc700 1 -- 172.21.15.3:0/990034688 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb22c0997e0 con 0x7fb230073380 2023-08-11T14:01:29.918 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:01:29.919 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 msgr2=0x7fb21c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 0x7fb21c0a24e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb220009fd0 tx=0x7fb220009260 comp rx=0 tx=0).stop 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 msgr2=0x7fb230198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb22c004740 tx=0x7fb22c009580 comp rx=0 tx=0).stop 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 shutdown_connections 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb21c0a0020 0x7fb21c0a24e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2301087e0 0x7fb23019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb230073380 0x7fb230198800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 --2- 172.21.15.3:0/990034688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb230074960 0x7fb230198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 >> 172.21.15.3:0/990034688 conn(0x7fb2300fbe50 msgr2=0x7fb2300fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.920+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 shutdown_connections 2023-08-11T14:01:29.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:29.921+0000 7fb2373ff700 1 -- 172.21.15.3:0/990034688 wait complete. 2023-08-11T14:01:29.942 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-11T14:01:29.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-11T14:01:30.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 -- 172.21.15.3:0/4118643471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88100980 msgr2=0x7f6b88100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:30.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 --2- 172.21.15.3:0/4118643471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88100980 0x7f6b88100da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b840099c0 tx=0x7f6b84009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 -- 172.21.15.3:0/4118643471 shutdown_connections 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 --2- 172.21.15.3:0/4118643471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88102510 0x7f6b88108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 --2- 172.21.15.3:0/4118643471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88100980 0x7f6b88100da0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 --2- 172.21.15.3:0/4118643471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88101b70 0x7f6b88101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 -- 172.21.15.3:0/4118643471 >> 172.21.15.3:0/4118643471 conn(0x7f6b880fbef0 msgr2=0x7f6b880fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 -- 172.21.15.3:0/4118643471 shutdown_connections 2023-08-11T14:01:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.159+0000 7f6b905ff700 1 -- 172.21.15.3:0/4118643471 wait complete. 2023-08-11T14:01:30.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.160+0000 7f6b905ff700 1 Processor -- start 2023-08-11T14:01:30.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 -- start start 2023-08-11T14:01:30.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 0x7f6b88196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:30.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88102510 0x7f6b8819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8819e1d0 con 0x7f6b88102510 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8819e310 con 0x7f6b88101b70 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b905ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8819e4a0 con 0x7f6b88100980 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b8db9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b8e39b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 0x7f6b88196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:30.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b8eb9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88102510 0x7f6b8819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b8db9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44978/0 (socket says 172.21.15.3:44978) 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.161+0000 7f6b8db9a700 1 -- 172.21.15.3:0/3531770998 learned_addr learned my addr 172.21.15.3:0/3531770998 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 -- 172.21.15.3:0/3531770998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 msgr2=0x7f6b88196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 0x7f6b88196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 -- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88102510 msgr2=0x7f6b8819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88102510 0x7f6b8819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 -- 172.21.15.3:0/3531770998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840095f0 con 0x7f6b88101b70 2023-08-11T14:01:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8e39b700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 0x7f6b88196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b8db9a700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800e9a0 tx=0x7f6b7800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78009790 con 0x7f6b88101b70 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8819e780 con 0x7f6b88101b70 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8819ece0 con 0x7f6b88101b70 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b78010430 con 0x7f6b88101b70 2023-08-11T14:01:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.162+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78018680 con 0x7f6b88101b70 2023-08-11T14:01:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.163+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c005320 con 0x7f6b88101b70 2023-08-11T14:01:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.165+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6b7800ca10 con 0x7f6b88101b70 2023-08-11T14:01:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.166+0000 7f6b7f7fe700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 0x7f6b740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.166+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6b78014070 con 0x7f6b88101b70 2023-08-11T14:01:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.166+0000 7f6b8e39b700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 0x7f6b740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.167+0000 7f6b8e39b700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 0x7f6b740a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009990 tx=0x7f6b8401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:30.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.170+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6b78051c30 con 0x7f6b88101b70 2023-08-11T14:01:30.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:30.415+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6b6c005b50 con 0x7f6b88101b70 2023-08-11T14:01:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:30 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2401577682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:30 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/990034688' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T14:01:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:30.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:30 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2401577682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:30 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/990034688' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T14:01:30.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:30.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:30 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2401577682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:30 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/990034688' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-08-11T14:01:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.261+0000 7f6b7f7fe700 1 -- 172.21.15.3:0/3531770998 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6b78098be0 con 0x7f6b88101b70 2023-08-11T14:01:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 msgr2=0x7f6b740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:31.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 0x7f6b740a2410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009990 tx=0x7f6b8401a040 comp rx=0 tx=0).stop 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 msgr2=0x7f6b88196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800e9a0 tx=0x7f6b7800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 shutdown_connections 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b88102510 0x7f6b8819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b88100980 0x7f6b88196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6b7409ff50 0x7f6b740a2410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 --2- 172.21.15.3:0/3531770998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b88101b70 0x7f6b88196b90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 >> 172.21.15.3:0/3531770998 conn(0x7f6b880fbef0 msgr2=0x7f6b880fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:31.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 shutdown_connections 2023-08-11T14:01:31.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.266+0000 7f6b905ff700 1 -- 172.21.15.3:0/3531770998 wait complete. 2023-08-11T14:01:31.269 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-08-11T14:01:31.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-08-11T14:01:31.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 -- 172.21.15.3:0/3713756333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a8074960 msgr2=0x7fd9a8072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:31.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/3713756333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a8074960 0x7fd9a8072db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009990 tx=0x7fd9a4009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 -- 172.21.15.3:0/3713756333 shutdown_connections 2023-08-11T14:01:31.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/3713756333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a810cee0 0x7fd9a810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/3713756333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a8074960 0x7fd9a8072db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/3713756333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 0x7fd9a8073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 -- 172.21.15.3:0/3713756333 >> 172.21.15.3:0/3713756333 conn(0x7fd9a80fbe60 msgr2=0x7fd9a80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:31.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 -- 172.21.15.3:0/3713756333 shutdown_connections 2023-08-11T14:01:31.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.505+0000 7fd9b03ff700 1 -- 172.21.15.3:0/3713756333 wait complete. 2023-08-11T14:01:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.506+0000 7fd9b03ff700 1 Processor -- start 2023-08-11T14:01:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.506+0000 7fd9b03ff700 1 -- start start 2023-08-11T14:01:31.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 0x7fd9a819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 0x7fd9a819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 0x7fd9a81a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a47e0 con 0x7fd9a8073380 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a4950 con 0x7fd9a810cee0 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9b03ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81a4b10 con 0x7fd9a8074960 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ad99a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 0x7fd9a819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:31.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae19b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 0x7fd9a819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ad99a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 0x7fd9a819d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34096/0 (socket says 172.21.15.3:34096) 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ad99a700 1 -- 172.21.15.3:0/1085483263 learned_addr learned my addr 172.21.15.3:0/1085483263 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 0x7fd9a81a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 -- 172.21.15.3:0/1085483263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 msgr2=0x7fd9a819d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 0x7fd9a819d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 -- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 msgr2=0x7fd9a819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 0x7fd9a819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:31.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.507+0000 7fd9ae99c700 1 -- 172.21.15.3:0/1085483263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40095f0 con 0x7fd9a810cee0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd9ae99c700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 0x7fd9a81a4210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000efc0 tx=0x7fd9a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0018070 con 0x7fd9a810cee0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9a000f040 con 0x7fd9a810cee0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000cd40 con 0x7fd9a810cee0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a81a4d90 con 0x7fd9a810cee0 2023-08-11T14:01:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.508+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a81a51f0 con 0x7fd9a810cee0 2023-08-11T14:01:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.510+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8102a30 con 0x7fd9a810cee0 2023-08-11T14:01:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.511+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd9a0003a00 con 0x7fd9a810cee0 2023-08-11T14:01:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.513+0000 7fd99f7fe700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 0x7fd9940a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.513+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd9a00caad0 con 0x7fd9a810cee0 2023-08-11T14:01:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.513+0000 7fd9ae19b700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 0x7fd9940a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.514+0000 7fd9ae19b700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 0x7fd9940a2490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b2e0 tx=0x7fd9a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.516+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd9a0094b80 con 0x7fd9a810cee0 2023-08-11T14:01:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3531770998' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:31 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3531770998' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:31.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:31.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:31 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3531770998' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:31.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:31 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:31.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:31.735+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd9a802cdc0 con 0x7fd9a810cee0 2023-08-11T14:01:32.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.291+0000 7fd99f7fe700 1 -- 172.21.15.3:0/1085483263 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd9a001b320 con 0x7fd9a810cee0 2023-08-11T14:01:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.303+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 msgr2=0x7fd9940a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.303+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 0x7fd9940a2490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b2e0 tx=0x7fd9a4005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.303+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 msgr2=0x7fd9a81a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.303+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 0x7fd9a81a4210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000efc0 tx=0x7fd9a000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 shutdown_connections 2023-08-11T14:01:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd99409ffd0 0x7fd9940a2490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9a810cee0 0x7fd9a81a4210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd9a8073380 0x7fd9a819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 --2- 172.21.15.3:0/1085483263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9a8074960 0x7fd9a819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.305+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 >> 172.21.15.3:0/1085483263 conn(0x7fd9a80fbe60 msgr2=0x7fd9a80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:32.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.311+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 shutdown_connections 2023-08-11T14:01:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.312+0000 7fd9b03ff700 1 -- 172.21.15.3:0/1085483263 wait complete. 2023-08-11T14:01:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-08-11T14:01:32.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-11T14:01:32.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:32 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T14:01:32.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:32 smithi003 ceph-mon[97195]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T14:01:32.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:32 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:32.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:32 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1085483263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:32.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:32 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T14:01:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:32 smithi187 ceph-mon[130973]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T14:01:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:32 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:32 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1085483263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 -- 172.21.15.3:0/3376930625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68108830 msgr2=0x7faa6810acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/3376930625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68108830 0x7faa6810acc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa6000b0a0 tx=0x7faa6000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 -- 172.21.15.3:0/3376930625 shutdown_connections 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/3376930625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68108830 0x7faa6810acc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/3376930625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa680748e0 0x7faa68072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/3376930625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa68073300 0x7faa68073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 -- 172.21.15.3:0/3376930625 >> 172.21.15.3:0/3376930625 conn(0x7faa680fbed0 msgr2=0x7faa680fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 -- 172.21.15.3:0/3376930625 shutdown_connections 2023-08-11T14:01:32.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.548+0000 7faa6e3ff700 1 -- 172.21.15.3:0/3376930625 wait complete. 2023-08-11T14:01:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.549+0000 7faa6e3ff700 1 Processor -- start 2023-08-11T14:01:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.549+0000 7faa6e3ff700 1 -- start start 2023-08-11T14:01:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 0x7faa681988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa680748e0 0x7faa68198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681a0430 con 0x7faa680748e0 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681a0570 con 0x7faa68073300 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6e3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681a0700 con 0x7faa68108830 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6c99c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 0x7faa681988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:32.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6c99c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35020/0 (socket says 172.21.15.3:35020) 2023-08-11T14:01:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa6c99c700 1 -- 172.21.15.3:0/4103795393 learned_addr learned my addr 172.21.15.3:0/4103795393 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa677fe700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa680748e0 0x7faa68198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 -- 172.21.15.3:0/4103795393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 msgr2=0x7faa6819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 -- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa680748e0 msgr2=0x7faa68198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa680748e0 0x7faa68198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.550+0000 7faa67fff700 1 -- 172.21.15.3:0/4103795393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60009d00 con 0x7faa68073300 2023-08-11T14:01:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa6c99c700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:32.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa67fff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 0x7faa681988b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e9f0 tx=0x7faa5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800c9c0 con 0x7faa68073300 2023-08-11T14:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa58010430 con 0x7faa68073300 2023-08-11T14:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa681a09e0 con 0x7faa68073300 2023-08-11T14:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58018680 con 0x7faa68073300 2023-08-11T14:01:32.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.551+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa681a0ef0 con 0x7faa68073300 2023-08-11T14:01:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.552+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa68106f30 con 0x7faa68073300 2023-08-11T14:01:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.554+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7faa58010860 con 0x7faa68073300 2023-08-11T14:01:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.555+0000 7faa657fa700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 0x7faa500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:32.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.556+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7faa58014070 con 0x7faa68073300 2023-08-11T14:01:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.556+0000 7faa677fe700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 0x7faa500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:32.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.556+0000 7faa677fe700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 0x7faa500a2410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009c60 tx=0x7faa5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:32.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.559+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7faa58051c30 con 0x7faa68073300 2023-08-11T14:01:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:32 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-08-11T14:01:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:32 smithi103 ceph-mon[130434]: osdmap e80: 12 total, 12 up, 12 in 2023-08-11T14:01:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:32 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:01:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:32 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1085483263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:32.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:32.828+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faa6810e330 con 0x7faa68073300 2023-08-11T14:01:33.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.332+0000 7faa657fa700 1 -- 172.21.15.3:0/4103795393 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7faa580950e0 con 0x7faa68073300 2023-08-11T14:01:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.335+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 msgr2=0x7faa500a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 0x7faa500a2410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009c60 tx=0x7faa5c009260 comp rx=0 tx=0).stop 2023-08-11T14:01:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 msgr2=0x7faa681988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:33.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 0x7faa681988b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e9f0 tx=0x7faa5800ed00 comp rx=0 tx=0).stop 2023-08-11T14:01:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 shutdown_connections 2023-08-11T14:01:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa68108830 0x7faa6819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faa5009ff50 0x7faa500a2410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa68073300 0x7faa681988b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 --2- 172.21.15.3:0/4103795393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa680748e0 0x7faa68198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 >> 172.21.15.3:0/4103795393 conn(0x7faa680fbed0 msgr2=0x7faa6810aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.336+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 shutdown_connections 2023-08-11T14:01:33.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.337+0000 7faa6e3ff700 1 -- 172.21.15.3:0/4103795393 wait complete. 2023-08-11T14:01:33.340 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2023-08-11T14:01:33.360 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-08-11T14:01:33.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-08-11T14:01:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: pgmap v265: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T14:01:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T14:01:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4103795393' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:33 smithi003 ceph-mon[97195]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T14:01:33.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: pgmap v265: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4103795393' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:33 smithi187 ceph-mon[130973]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T14:01:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.585+0000 7f221bfff700 1 -- 172.21.15.3:0/3204303056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2214073380 msgr2=0x7f2214073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:33.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.585+0000 7f221bfff700 1 --2- 172.21.15.3:0/3204303056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2214073380 0x7f2214073800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2204009990 tx=0x7f2204009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 -- 172.21.15.3:0/3204303056 shutdown_connections 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 --2- 172.21.15.3:0/3204303056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f221410cee0 0x7f221410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 --2- 172.21.15.3:0/3204303056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214074960 0x7f2214072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 --2- 172.21.15.3:0/3204303056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2214073380 0x7f2214073800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 -- 172.21.15.3:0/3204303056 >> 172.21.15.3:0/3204303056 conn(0x7f22140fbe60 msgr2=0x7f22140fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 -- 172.21.15.3:0/3204303056 shutdown_connections 2023-08-11T14:01:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.586+0000 7f221bfff700 1 -- 172.21.15.3:0/3204303056 wait complete. 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 Processor -- start 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 -- start start 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 0x7f221419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221410cee0 0x7f22141a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a4810 con 0x7f2214074960 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a4980 con 0x7f2214073380 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.587+0000 7f221bfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a4b40 con 0x7f221410cee0 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f221959a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 0x7f221419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f221959a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51898/0 (socket says 172.21.15.3:51898) 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f221959a700 1 -- 172.21.15.3:0/1514684139 learned_addr learned my addr 172.21.15.3:0/1514684139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f221a59c700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221410cee0 0x7f22141a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:33.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 -- 172.21.15.3:0/1514684139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221410cee0 msgr2=0x7f22141a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221410cee0 0x7f22141a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 -- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 msgr2=0x7f221419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 -- 172.21.15.3:0/1514684139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040095f0 con 0x7f2214073380 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f221959a700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.588+0000 7f2219d9b700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 0x7f221419cca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.589+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221000c9c0 con 0x7f2214073380 2023-08-11T14:01:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.589+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2210010430 con 0x7f2214073380 2023-08-11T14:01:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.589+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22141a4e20 con 0x7f2214073380 2023-08-11T14:01:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.589+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210018680 con 0x7f2214073380 2023-08-11T14:01:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.589+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22141a5250 con 0x7f2214073380 2023-08-11T14:01:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.590+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2214102a30 con 0x7f2214073380 2023-08-11T14:01:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.592+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2210010860 con 0x7f2214073380 2023-08-11T14:01:33.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.594+0000 7f220affd700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 0x7f22000a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:33.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.594+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2210014070 con 0x7f2214073380 2023-08-11T14:01:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.594+0000 7f221959a700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 0x7f22000a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:33.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.595+0000 7f221959a700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 0x7f22000a2490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f221419e1a0 tx=0x7f2204005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:33.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.597+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2210051c30 con 0x7f2214073380 2023-08-11T14:01:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-08-11T14:01:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: pgmap v265: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:01:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-08-11T14:01:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: osdmap e81: 12 total, 12 up, 12 in 2023-08-11T14:01:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4103795393' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-08-11T14:01:33.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:33 smithi103 ceph-mon[130434]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-08-11T14:01:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:33.830+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2214112750 con 0x7f2214073380 2023-08-11T14:01:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.295+0000 7f220affd700 1 -- 172.21.15.3:0/1514684139 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2210095030 con 0x7f2214073380 2023-08-11T14:01:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.297+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 msgr2=0x7f22000a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.297+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 0x7f22000a2490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f221419e1a0 tx=0x7f2204005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.297+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 msgr2=0x7f221419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.297+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 0x7f221419cca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f221000e9f0 tx=0x7f221000ed00 comp rx=0 tx=0).stop 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 shutdown_connections 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f221410cee0 0x7f22141a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f220009ffd0 0x7f22000a2490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2214073380 0x7f221419cca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 --2- 172.21.15.3:0/1514684139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214074960 0x7f221419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 >> 172.21.15.3:0/1514684139 conn(0x7f22140fbe60 msgr2=0x7f22140fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 shutdown_connections 2023-08-11T14:01:34.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.298+0000 7f221bfff700 1 -- 172.21.15.3:0/1514684139 wait complete. 2023-08-11T14:01:34.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-08-11T14:01:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1514684139' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T14:01:34.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1514684139' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 -- 172.21.15.3:0/2335340900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b4101b30 msgr2=0x7fb3b4101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/2335340900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b4101b30 0x7fb3b4101f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009920 tx=0x7fb3a8009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 -- 172.21.15.3:0/2335340900 shutdown_connections 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/2335340900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b41024d0 0x7fb3b4108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/2335340900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b4100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/2335340900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b4101b30 0x7fb3b4101f90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 -- 172.21.15.3:0/2335340900 >> 172.21.15.3:0/2335340900 conn(0x7fb3b40fbe90 msgr2=0x7fb3b40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 -- 172.21.15.3:0/2335340900 shutdown_connections 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.542+0000 7fb3b93ff700 1 -- 172.21.15.3:0/2335340900 wait complete. 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.543+0000 7fb3b93ff700 1 Processor -- start 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 -- start start 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b4101b30 0x7fb3b4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 0x7fb3b4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40739f0 con 0x7fb3b4101b30 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4073b60 con 0x7fb3b41024d0 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b93ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41a06a0 con 0x7fb3b4100940 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.544+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b4101b30 0x7fb3b4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b40746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35058/0 (socket says 172.21.15.3:35058) 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 0x7fb3b4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b2ffd700 1 -- 172.21.15.3:0/1339354742 learned_addr learned my addr 172.21.15.3:0/1339354742 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 -- 172.21.15.3:0/1339354742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 msgr2=0x7fb3b40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 -- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b4101b30 msgr2=0x7fb3b4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b4101b30 0x7fb3b4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 -- 172.21.15.3:0/1339354742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8009520 con 0x7fb3b41024d0 2023-08-11T14:01:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb3b37fe700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 0x7fb3b4073250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00efc0 tx=0x7fb3ac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.545+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac018070 con 0x7fb3b41024d0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.546+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ac00f040 con 0x7fb3b41024d0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.546+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00cd40 con 0x7fb3b41024d0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.546+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b41a0920 con 0x7fb3b41024d0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.546+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41a0de0 con 0x7fb3b41024d0 2023-08-11T14:01:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.547+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b410c1a0 con 0x7fb3b41024d0 2023-08-11T14:01:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.548+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb3ac003a00 con 0x7fb3b41024d0 2023-08-11T14:01:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.550+0000 7fb39bfff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 0x7fb39c0a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.550+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fb3ac0cadd0 con 0x7fb3b41024d0 2023-08-11T14:01:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.550+0000 7fb3b2ffd700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 0x7fb39c0a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.551+0000 7fb3b2ffd700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 0x7fb39c0a2530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009cf0 tx=0x7fb3a4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.553+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb3ac094bb0 con 0x7fb3b41024d0 2023-08-11T14:01:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: osdmap e82: 12 total, 12 up, 12 in 2023-08-11T14:01:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-08-11T14:01:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-08-11T14:01:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-08-11T14:01:34.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1514684139' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-08-11T14:01:34.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:34.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:34.769+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3b41a1000 con 0x7fb3b41024d0 2023-08-11T14:01:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: pgmap v268: 97 pgs: 16 creating+peering, 27 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-08-11T14:01:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T14:01:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:active 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1339354742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:35.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:35.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: pgmap v268: 97 pgs: 16 creating+peering, 27 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-08-11T14:01:35.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T14:01:35.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:active 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1339354742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: pgmap v268: 97 pgs: 16 creating+peering, 27 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: osdmap e83: 12 total, 12 up, 12 in 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] up:active 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1339354742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-08-11T14:01:35.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:35.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:01:35.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:01:35.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:01:36.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.357+0000 7fb39bfff700 1 -- 172.21.15.3:0/1339354742 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb3ac0981d0 con 0x7fb3b41024d0 2023-08-11T14:01:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 msgr2=0x7fb39c0a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 0x7fb39c0a2530 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009cf0 tx=0x7fb3a4009310 comp rx=0 tx=0).stop 2023-08-11T14:01:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 msgr2=0x7fb3b4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 0x7fb3b4073250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00efc0 tx=0x7fb3ac00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 shutdown_connections 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb39c0a0070 0x7fb39c0a2530 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b41024d0 0x7fb3b4073250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100940 0x7fb3b40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 --2- 172.21.15.3:0/1339354742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3b4101b30 0x7fb3b4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.362+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 >> 172.21.15.3:0/1339354742 conn(0x7fb3b40fbe90 msgr2=0x7fb3b40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.363+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 shutdown_connections 2023-08-11T14:01:36.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.363+0000 7fb3b93ff700 1 -- 172.21.15.3:0/1339354742 wait complete. 2023-08-11T14:01:36.366 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2023-08-11T14:01:36.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-08-11T14:01:36.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 -- 172.21.15.3:0/99010960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 msgr2=0x7f6a5c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 --2- 172.21.15.3:0/99010960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c100d30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a500099c0 tx=0x7f6a50009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 -- 172.21.15.3:0/99010960 shutdown_connections 2023-08-11T14:01:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 --2- 172.21.15.3:0/99010960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c1024a0 0x7f6a5c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 --2- 172.21.15.3:0/99010960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c100d30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 --2- 172.21.15.3:0/99010960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c101b00 0x7f6a5c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 -- 172.21.15.3:0/99010960 >> 172.21.15.3:0/99010960 conn(0x7f6a5c0fbe60 msgr2=0x7f6a5c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 -- 172.21.15.3:0/99010960 shutdown_connections 2023-08-11T14:01:36.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.601+0000 7f6a62fff700 1 -- 172.21.15.3:0/99010960 wait complete. 2023-08-11T14:01:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.602+0000 7f6a62fff700 1 Processor -- start 2023-08-11T14:01:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.602+0000 7f6a62fff700 1 -- start start 2023-08-11T14:01:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c101b00 0x7f6a5c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c1024a0 0x7f6a5c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:36.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c1a03e0 con 0x7f6a5c100910 2023-08-11T14:01:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c1a0520 con 0x7f6a5c1024a0 2023-08-11T14:01:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a62fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c1a06b0 con 0x7f6a5c101b00 2023-08-11T14:01:36.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:36.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51928/0 (socket says 172.21.15.3:51928) 2023-08-11T14:01:36.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 -- 172.21.15.3:0/3861191963 learned_addr learned my addr 172.21.15.3:0/3861191963 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:36.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a5bfff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c101b00 0x7f6a5c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:36.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a6159c700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c1024a0 0x7f6a5c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:36.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 -- 172.21.15.3:0/3861191963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c101b00 msgr2=0x7f6a5c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:36.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c101b00 0x7f6a5c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 -- 172.21.15.3:0/3861191963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c1024a0 msgr2=0x7f6a5c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:36.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c1024a0 0x7f6a5c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:36.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.603+0000 7f6a60d9b700 1 -- 172.21.15.3:0/3861191963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500095f0 con 0x7f6a5c100910 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a60d9b700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009fd0 tx=0x7f6a5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5001d070 con 0x7f6a5c100910 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c1a0930 con 0x7f6a5c100910 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a50005050 con 0x7f6a5c100910 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50004a70 con 0x7f6a5c100910 2023-08-11T14:01:36.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.604+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c1a0d60 con 0x7f6a5c100910 2023-08-11T14:01:36.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.605+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c10c170 con 0x7f6a5c100910 2023-08-11T14:01:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.608+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6a50005450 con 0x7f6a5c100910 2023-08-11T14:01:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.610+0000 7f6a59ffb700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 0x7f6a4c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.610+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f6a500cc5b0 con 0x7f6a5c100910 2023-08-11T14:01:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.610+0000 7f6a5bfff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 0x7f6a4c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:36.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.611+0000 7f6a5bfff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 0x7f6a4c0a2410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009c60 tx=0x7f6a48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:36.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.612+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6a500961a0 con 0x7f6a5c100910 2023-08-11T14:01:36.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:36 smithi103 ceph-mon[130434]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T14:01:36.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:36 smithi003 ceph-mon[97195]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T14:01:36.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:36 smithi187 ceph-mon[130973]: osdmap e84: 12 total, 12 up, 12 in 2023-08-11T14:01:36.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:36.823+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f6a5c1a0f80 con 0x7f6a5c100910 2023-08-11T14:01:37.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.358+0000 7f6a59ffb700 1 -- 172.21.15.3:0/3861191963 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6a500997c0 con 0x7f6a5c100910 2023-08-11T14:01:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.364+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 msgr2=0x7f6a4c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:37.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.364+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 0x7f6a4c0a2410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009c60 tx=0x7f6a48009260 comp rx=0 tx=0).stop 2023-08-11T14:01:37.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.364+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 msgr2=0x7f6a5c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:37.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.364+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009fd0 tx=0x7f6a5000f690 comp rx=0 tx=0).stop 2023-08-11T14:01:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 shutdown_connections 2023-08-11T14:01:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6a4c09ff50 0x7f6a4c0a2410 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009c60 tx=0x7f6a48009260 comp rx=0 tx=0).stop 2023-08-11T14:01:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a5c1024a0 0x7f6a5c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a5c100910 0x7f6a5c198860 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 --2- 172.21.15.3:0/3861191963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a5c101b00 0x7f6a5c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.370+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 >> 172.21.15.3:0/3861191963 conn(0x7f6a5c0fbe60 msgr2=0x7f6a5c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.371+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 shutdown_connections 2023-08-11T14:01:37.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.374+0000 7f6a62fff700 1 -- 172.21.15.3:0/3861191963 wait complete. 2023-08-11T14:01:37.383 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 allow_ec_overwrites to true 2023-08-11T14:01:37.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-08-11T14:01:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.610+0000 7f0008fff700 1 -- 172.21.15.3:0/4293823547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00041046d0 msgr2=0x7f000410af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.610+0000 7f0008fff700 1 --2- 172.21.15.3:0/4293823547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00041046d0 0x7f000410af80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b0d0 tx=0x7efffc00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 -- 172.21.15.3:0/4293823547 shutdown_connections 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 --2- 172.21.15.3:0/4293823547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00041046d0 0x7f000410af80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 --2- 172.21.15.3:0/4293823547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004102b40 0x7f0004102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 --2- 172.21.15.3:0/4293823547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0004103d30 0x7f0004104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 -- 172.21.15.3:0/4293823547 >> 172.21.15.3:0/4293823547 conn(0x7f00040fe0d0 msgr2=0x7f00041004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:37.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 -- 172.21.15.3:0/4293823547 shutdown_connections 2023-08-11T14:01:37.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.611+0000 7f0008fff700 1 -- 172.21.15.3:0/4293823547 wait complete. 2023-08-11T14:01:37.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 Processor -- start 2023-08-11T14:01:37.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 -- start start 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 0x7f0004198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00041046d0 0x7f000419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00041a04b0 con 0x7f0004102b40 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00041a05f0 con 0x7f0004103d30 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0008fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00041a0780 con 0x7f00041046d0 2023-08-11T14:01:37.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0001d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 0x7f0004198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:37.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f000259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:37.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0001d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 0x7f0004198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57436/0 (socket says 172.21.15.3:57436) 2023-08-11T14:01:37.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f000259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51948/0 (socket says 172.21.15.3:51948) 2023-08-11T14:01:37.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.612+0000 7f0001d9b700 1 -- 172.21.15.3:0/838201841 learned_addr learned my addr 172.21.15.3:0/838201841 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f0002d9d700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00041046d0 0x7f000419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 -- 172.21.15.3:0/838201841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00041046d0 msgr2=0x7f000419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00041046d0 0x7f000419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 -- 172.21.15.3:0/838201841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 msgr2=0x7f0004198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 0x7f0004198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:37.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 -- 172.21.15.3:0/838201841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc009d00 con 0x7f0004102b40 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f000259c700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9c0 tx=0x7efff400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4004220 con 0x7f0004102b40 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efff4004380 con 0x7f0004102b40 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00041a0a60 con 0x7f0004102b40 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.613+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4003be0 con 0x7f0004102b40 2023-08-11T14:01:37.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.614+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00041a0f70 con 0x7f0004102b40 2023-08-11T14:01:37.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.615+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000410a080 con 0x7f0004102b40 2023-08-11T14:01:37.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.618+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efff4003d40 con 0x7f0004102b40 2023-08-11T14:01:37.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.620+0000 7efff37fe700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 0x7effec0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:37.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.620+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7efff4014070 con 0x7f0004102b40 2023-08-11T14:01:37.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.620+0000 7f0001d9b700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 0x7effec0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:37.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.620+0000 7f0001d9b700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 0x7effec0a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efff8009d20 tx=0x7efff8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:37.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.622+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7efff4095420 con 0x7f0004102b40 2023-08-11T14:01:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:37 smithi103 ceph-mon[130434]: pgmap v271: 97 pgs: 16 creating+peering, 62 active+clean, 19 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-08-11T14:01:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:37 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T14:01:37.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:37 smithi103 ceph-mon[130434]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T14:01:37.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:37 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T14:01:37.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:37 smithi003 ceph-mon[97195]: pgmap v271: 97 pgs: 16 creating+peering, 62 active+clean, 19 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-08-11T14:01:37.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:37 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T14:01:37.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:37 smithi003 ceph-mon[97195]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T14:01:37.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:37 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T14:01:37.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:37 smithi187 ceph-mon[130973]: pgmap v271: 97 pgs: 16 creating+peering, 62 active+clean, 19 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2023-08-11T14:01:37.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:37 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-08-11T14:01:37.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:37 smithi187 ceph-mon[130973]: osdmap e85: 12 total, 12 up, 12 in 2023-08-11T14:01:37.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:37 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-08-11T14:01:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:37.882+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f000402cdc0 con 0x7f0004102b40 2023-08-11T14:01:38.360 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:01:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: 2023-08-11T14:01:38.019+0000 7f80c8aa2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-11T14:01:38.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.364+0000 7efff37fe700 1 -- 172.21.15.3:0/838201841 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7efff4098a40 con 0x7f0004102b40 2023-08-11T14:01:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 msgr2=0x7effec0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 0x7effec0a2410 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efff8009d20 tx=0x7efff8009260 comp rx=0 tx=0).stop 2023-08-11T14:01:38.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 msgr2=0x7f0004198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efff400e9c0 tx=0x7efff400ecd0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 shutdown_connections 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00041046d0 0x7f000419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004102b40 0x7f0004198930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7effec09ff50 0x7effec0a2410 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 --2- 172.21.15.3:0/838201841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0004103d30 0x7f0004198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 >> 172.21.15.3:0/838201841 conn(0x7f00040fe0d0 msgr2=0x7f00041071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:38.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 shutdown_connections 2023-08-11T14:01:38.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.367+0000 7f0008fff700 1 -- 172.21.15.3:0/838201841 wait complete. 2023-08-11T14:01:38.370 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 4 to fsmap 2023-08-11T14:01:38.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 -- 172.21.15.3:0/1458204056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3868100920 msgr2=0x7f3868100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 --2- 172.21.15.3:0/1458204056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3868100920 0x7f3868100d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38580099c0 tx=0x7f3858009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 -- 172.21.15.3:0/1458204056 shutdown_connections 2023-08-11T14:01:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 --2- 172.21.15.3:0/1458204056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38681024b0 0x7f3868108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 --2- 172.21.15.3:0/1458204056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3868100920 0x7f3868100d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 --2- 172.21.15.3:0/1458204056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 0x7f3868101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 -- 172.21.15.3:0/1458204056 >> 172.21.15.3:0/1458204056 conn(0x7f38680fbe50 msgr2=0x7f38680fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 -- 172.21.15.3:0/1458204056 shutdown_connections 2023-08-11T14:01:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.596+0000 7f386cbff700 1 -- 172.21.15.3:0/1458204056 wait complete. 2023-08-11T14:01:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 Processor -- start 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 -- start start 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 0x7f3868198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38681024b0 0x7f386819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a0340 con 0x7f38681024b0 2023-08-11T14:01:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a0480 con 0x7f3868100920 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.598+0000 7f386cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38681a0610 con 0x7f3868101b10 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57440/0 (socket says 172.21.15.3:57440) 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 -- 172.21.15.3:0/2632609752 learned_addr learned my addr 172.21.15.3:0/2632609752 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f3866d9d700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38681024b0 0x7f386819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f385ffff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 0x7f3868198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 -- 172.21.15.3:0/2632609752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 msgr2=0x7f3868198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 0x7f3868198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 -- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38681024b0 msgr2=0x7f386819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38681024b0 0x7f386819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 -- 172.21.15.3:0/2632609752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580095f0 con 0x7f3868100920 2023-08-11T14:01:38.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.599+0000 7f386659c700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3858009fd0 tx=0x7f3858009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.600+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385801d070 con 0x7f3868100920 2023-08-11T14:01:38.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.600+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38681a0890 con 0x7f3868100920 2023-08-11T14:01:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.600+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3858005030 con 0x7f3868100920 2023-08-11T14:01:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.600+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858004a90 con 0x7f3868100920 2023-08-11T14:01:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.600+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38681a0cc0 con 0x7f3868100920 2023-08-11T14:01:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.601+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3868192f20 con 0x7f3868100920 2023-08-11T14:01:38.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.603+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f38580051a0 con 0x7f3868100920 2023-08-11T14:01:38.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.604+0000 7f385f7fe700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 0x7f38540a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:38.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.605+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f38580cc5d0 con 0x7f3868100920 2023-08-11T14:01:38.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.605+0000 7f385ffff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 0x7f38540a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:38.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.605+0000 7f385ffff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 0x7f38540a2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3850009c60 tx=0x7f3850009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:38.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.607+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f38580961a0 con 0x7f3868100920 2023-08-11T14:01:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:38 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T14:01:38.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:38 smithi103 ceph-mon[130434]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T14:01:38.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:38 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T14:01:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:38 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T14:01:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:38 smithi003 ceph-mon[97195]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T14:01:38.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:38 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T14:01:38.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:38 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3861191963' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-08-11T14:01:38.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:38 smithi187 ceph-mon[130973]: osdmap e86: 12 total, 12 up, 12 in 2023-08-11T14:01:38.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:38 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-08-11T14:01:38.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.866+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f386810c130 con 0x7f3868100920 2023-08-11T14:01:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.867+0000 7f385f7fe700 1 -- 172.21.15.3:0/2632609752 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11255 (secure 0 0 0) 0x7f38580997c0 con 0x7f3868100920 2023-08-11T14:01:38.868 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:38.868 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:01:38.353686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 msgr2=0x7f38540a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 0x7f38540a2460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3850009c60 tx=0x7f3850009260 comp rx=0 tx=0).stop 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 msgr2=0x7f38681987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3858009fd0 tx=0x7f3858009580 comp rx=0 tx=0).stop 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 shutdown_connections 2023-08-11T14:01:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38681024b0 0x7f386819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f385409ffa0 0x7f38540a2460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3868100920 0x7f38681987c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 --2- 172.21.15.3:0/2632609752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3868101b10 0x7f3868198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 >> 172.21.15.3:0/2632609752 conn(0x7f38680fbe50 msgr2=0x7f3868107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 shutdown_connections 2023-08-11T14:01:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:38.869+0000 7f386cbff700 1 -- 172.21.15.3:0/2632609752 wait complete. 2023-08-11T14:01:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2023-08-11T14:01:38.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/2277330720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8101b10 msgr2=0x7ff6a8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/2277330720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8101b10 0x7ff6a8101f70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6900099c0 tx=0x7ff690009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/2277330720 shutdown_connections 2023-08-11T14:01:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/2277330720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a81024b0 0x7ff6a8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/2277330720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a8100920 0x7ff6a8100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/2277330720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8101b10 0x7ff6a8101f70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/2277330720 >> 172.21.15.3:0/2277330720 conn(0x7ff6a80fbe50 msgr2=0x7ff6a80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/2277330720 shutdown_connections 2023-08-11T14:01:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.093+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/2277330720 wait complete. 2023-08-11T14:01:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.094+0000 7ff6ad1ff700 1 Processor -- start 2023-08-11T14:01:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 -- start start 2023-08-11T14:01:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a81024b0 0x7ff6a819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81a0370 con 0x7ff6a8101b10 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81a04b0 con 0x7ff6a8100920 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6ad1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81a0640 con 0x7ff6a81024b0 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51968/0 (socket says 172.21.15.3:51968) 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57458/0 (socket says 172.21.15.3:57458) 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a659c700 1 -- 172.21.15.3:0/1973615547 learned_addr learned my addr 172.21.15.3:0/1973615547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a759e700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a81024b0 0x7ff6a819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 -- 172.21.15.3:0/1973615547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a81024b0 msgr2=0x7ff6a819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a81024b0 0x7ff6a819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 -- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 msgr2=0x7ff6a8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.095+0000 7ff6a6d9d700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff6a6d9d700 1 -- 172.21.15.3:0/1973615547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900095f0 con 0x7ff6a8100920 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff6a659c700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff6a6d9d700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9c0 tx=0x7ff69800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698004220 con 0x7ff6a8100920 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a81a0920 con 0x7ff6a8100920 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff698004380 con 0x7ff6a8100920 2023-08-11T14:01:39.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698003be0 con 0x7ff6a8100920 2023-08-11T14:01:39.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.096+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a81a0e30 con 0x7ff6a8100920 2023-08-11T14:01:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.097+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a810c180 con 0x7ff6a8100920 2023-08-11T14:01:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.099+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff698003d40 con 0x7ff6a8100920 2023-08-11T14:01:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.101+0000 7ff69ffff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 0x7ff6940a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.101+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff698014070 con 0x7ff6a8100920 2023-08-11T14:01:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.101+0000 7ff6a659c700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 0x7ff6940a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.102+0000 7ff6a659c700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 0x7ff6940a2460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff690005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.103+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff698095420 con 0x7ff6a8100920 2023-08-11T14:01:39.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.315+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a81a1050 con 0x7ff6a8100920 2023-08-11T14:01:39.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.317+0000 7ff69ffff700 1 -- 172.21.15.3:0/1973615547 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7ff698098a40 con 0x7ff6a8100920 2023-08-11T14:01:39.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:39.320 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:39.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 msgr2=0x7ff6940a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 0x7ff6940a2460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff690000c00 tx=0x7ff690005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 msgr2=0x7ff6a81987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9c0 tx=0x7ff69800ecd0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 shutdown_connections 2023-08-11T14:01:39.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6a81024b0 0x7ff6a819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff69409ffa0 0x7ff6940a2460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6a8100920 0x7ff6a81987f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 --2- 172.21.15.3:0/1973615547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6a8101b10 0x7ff6a8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 >> 172.21.15.3:0/1973615547 conn(0x7ff6a80fbe50 msgr2=0x7ff6a80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.319+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 shutdown_connections 2023-08-11T14:01:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.320+0000 7ff6ad1ff700 1 -- 172.21.15.3:0/1973615547 wait complete. 2023-08-11T14:01:39.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:39.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 -- 172.21.15.3:0/3525421381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 msgr2=0x7f16ec108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 --2- 172.21.15.3:0/3525421381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec108d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16e400b0d0 tx=0x7f16e400b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 -- 172.21.15.3:0/3525421381 shutdown_connections 2023-08-11T14:01:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 --2- 172.21.15.3:0/3525421381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec108d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 --2- 172.21.15.3:0/3525421381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 0x7f16ec100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 --2- 172.21.15.3:0/3525421381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 -- 172.21.15.3:0/3525421381 >> 172.21.15.3:0/3525421381 conn(0x7f16ec0fbe60 msgr2=0x7f16ec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 -- 172.21.15.3:0/3525421381 shutdown_connections 2023-08-11T14:01:39.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.544+0000 7f16f43ff700 1 -- 172.21.15.3:0/3525421381 wait complete. 2023-08-11T14:01:39.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.545+0000 7f16f43ff700 1 Processor -- start 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.545+0000 7f16f43ff700 1 -- start start 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 0x7f16ec1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a0460 con 0x7f16ec1024c0 2023-08-11T14:01:39.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a05a0 con 0x7f16ec101b20 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f43ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1a0730 con 0x7f16ec100930 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51982/0 (socket says 172.21.15.3:51982) 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 -- 172.21.15.3:0/326643967 learned_addr learned my addr 172.21.15.3:0/326643967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f219b700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 0x7f16ec1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f199a700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 -- 172.21.15.3:0/326643967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 msgr2=0x7f16ec1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 0x7f16ec1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 -- 172.21.15.3:0/326643967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 msgr2=0x7f16ec198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f299c700 1 -- 172.21.15.3:0/326643967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009520 con 0x7f16ec1024c0 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.546+0000 7f16f199a700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16f299c700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16e4008eb0 tx=0x7f16e4008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400e040 con 0x7f16ec1024c0 2023-08-11T14:01:39.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16e400be30 con 0x7f16ec1024c0 2023-08-11T14:01:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e401d470 con 0x7f16ec1024c0 2023-08-11T14:01:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e4009d00 con 0x7f16ec1024c0 2023-08-11T14:01:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.547+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1a0c50 con 0x7f16ec1024c0 2023-08-11T14:01:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.548+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec10c1a0 con 0x7f16ec1024c0 2023-08-11T14:01:39.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.550+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f16e4005190 con 0x7f16ec1024c0 2023-08-11T14:01:39.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.552+0000 7f16e37fe700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 0x7f16d80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.552+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f16e40cc480 con 0x7f16ec1024c0 2023-08-11T14:01:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.552+0000 7f16f219b700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 0x7f16d80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:39.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.553+0000 7f16f219b700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 0x7f16d80a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16ec101980 tx=0x7f16e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:39.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.556+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f16e4096050 con 0x7f16ec1024c0 2023-08-11T14:01:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: pgmap v274: 129 pgs: 16 creating+peering, 95 active+clean, 18 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2023-08-11T14:01:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T14:01:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T14:01:39.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:39.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2632609752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:39.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:39 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1973615547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:39.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.766+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16ec10c3e0 con 0x7f16ec1024c0 2023-08-11T14:01:39.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.768+0000 7f16e37fe700 1 -- 172.21.15.3:0/326643967 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7f16e4099670 con 0x7f16ec1024c0 2023-08-11T14:01:39.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:39.771 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:39.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.770+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 msgr2=0x7f16d80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.770+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 0x7f16d80a2460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16ec101980 tx=0x7f16e8009260 comp rx=0 tx=0).stop 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.770+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 msgr2=0x7f16ec19fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.770+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16e4008eb0 tx=0x7f16e4008ee0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 shutdown_connections 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec1024c0 0x7f16ec19fe90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16ec100930 0x7f16ec1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f16d809ffa0 0x7f16d80a2460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 --2- 172.21.15.3:0/326643967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16ec101b20 0x7f16ec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 >> 172.21.15.3:0/326643967 conn(0x7f16ec0fbe60 msgr2=0x7f16ec107300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 shutdown_connections 2023-08-11T14:01:39.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:39.771+0000 7f16f43ff700 1 -- 172.21.15.3:0/326643967 wait complete. 2023-08-11T14:01:39.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: pgmap v274: 129 pgs: 16 creating+peering, 95 active+clean, 18 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2023-08-11T14:01:39.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T14:01:39.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T14:01:39.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:39.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2632609752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:39.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1973615547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:39.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:01:39.816 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: pgmap v274: 129 pgs: 16 creating+peering, 95 active+clean, 18 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2023-08-11T14:01:39.816 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: osdmap e87: 12 total, 12 up, 12 in 2023-08-11T14:01:39.816 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/838201841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-08-11T14:01:39.817 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:39.817 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2632609752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:39.817 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:39 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1973615547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.017+0000 7efd603ff700 1 -- 172.21.15.3:0/2909767513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58100920 msgr2=0x7efd58100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.017+0000 7efd603ff700 1 --2- 172.21.15.3:0/2909767513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58100920 0x7efd58100d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd540099c0 tx=0x7efd54009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 -- 172.21.15.3:0/2909767513 shutdown_connections 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 --2- 172.21.15.3:0/2909767513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd58108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 --2- 172.21.15.3:0/2909767513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58100920 0x7efd58100d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 --2- 172.21.15.3:0/2909767513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58101b10 0x7efd58101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 -- 172.21.15.3:0/2909767513 >> 172.21.15.3:0/2909767513 conn(0x7efd580fbe50 msgr2=0x7efd580fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 -- 172.21.15.3:0/2909767513 shutdown_connections 2023-08-11T14:01:40.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.018+0000 7efd603ff700 1 -- 172.21.15.3:0/2909767513 wait complete. 2023-08-11T14:01:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 Processor -- start 2023-08-11T14:01:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 -- start start 2023-08-11T14:01:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58100920 0x7efd58198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd5819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a03a0 con 0x7efd58101b10 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a04e0 con 0x7efd581024b0 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd603ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a0670 con 0x7efd58100920 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd5d99a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd5d99a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51992/0 (socket says 172.21.15.3:51992) 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd5d99a700 1 -- 172.21.15.3:0/227078512 learned_addr learned my addr 172.21.15.3:0/227078512 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd5e99c700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd5819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.019+0000 7efd5e19b700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58100920 0x7efd58198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 -- 172.21.15.3:0/227078512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58100920 msgr2=0x7efd58198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58100920 0x7efd58198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 -- 172.21.15.3:0/227078512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 msgr2=0x7efd5819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd5819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 -- 172.21.15.3:0/227078512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540095f0 con 0x7efd58101b10 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5e99c700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd5819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:40.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd5d99a700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9a0 tx=0x7efd4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48009790 con 0x7efd58101b10 2023-08-11T14:01:40.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd48010430 con 0x7efd58101b10 2023-08-11T14:01:40.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48018680 con 0x7efd58101b10 2023-08-11T14:01:40.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.020+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd581a0950 con 0x7efd58101b10 2023-08-11T14:01:40.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.021+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd581a0e10 con 0x7efd58101b10 2023-08-11T14:01:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.022+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd58192f70 con 0x7efd58101b10 2023-08-11T14:01:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.024+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efd480105a0 con 0x7efd58101b10 2023-08-11T14:01:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.025+0000 7efd4f7fe700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 0x7efd440a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.025+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efd48014070 con 0x7efd58101b10 2023-08-11T14:01:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.025+0000 7efd5e19b700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 0x7efd440a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.026+0000 7efd5e19b700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 0x7efd440a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd54009990 tx=0x7efd54005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.029+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7efd48051c30 con 0x7efd58101b10 2023-08-11T14:01:40.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.285+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd5810c180 con 0x7efd58101b10 2023-08-11T14:01:40.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.286+0000 7efd4f7fe700 1 -- 172.21.15.3:0/227078512 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11255 (secure 0 0 0) 0x7efd48095180 con 0x7efd58101b10 2023-08-11T14:01:40.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:40.288 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14502,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24500,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24527,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:01:38.353686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-08-11T14:01:40.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 msgr2=0x7efd440a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 0x7efd440a2460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efd54009990 tx=0x7efd54005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 msgr2=0x7efd58198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9a0 tx=0x7efd4800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 shutdown_connections 2023-08-11T14:01:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd4409ffa0 0x7efd440a2460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd581024b0 0x7efd5819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd58100920 0x7efd58198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 --2- 172.21.15.3:0/227078512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd58101b10 0x7efd58198d60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 >> 172.21.15.3:0/227078512 conn(0x7efd580fbe50 msgr2=0x7efd581073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 shutdown_connections 2023-08-11T14:01:40.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.288+0000 7efd603ff700 1 -- 172.21.15.3:0/227078512 wait complete. 2023-08-11T14:01:40.292 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2023-08-11T14:01:40.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:40.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 -- 172.21.15.3:0/870061707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b4101b20 msgr2=0x7ff1b4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/870061707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b4101b20 0x7ff1b4101f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0099c0 tx=0x7ff19c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 -- 172.21.15.3:0/870061707 shutdown_connections 2023-08-11T14:01:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/870061707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b41024c0 0x7ff1b4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/870061707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 0x7ff1b4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/870061707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b4101b20 0x7ff1b4101f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 -- 172.21.15.3:0/870061707 >> 172.21.15.3:0/870061707 conn(0x7ff1b40fbe60 msgr2=0x7ff1b40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 -- 172.21.15.3:0/870061707 shutdown_connections 2023-08-11T14:01:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.513+0000 7ff1b89ff700 1 -- 172.21.15.3:0/870061707 wait complete. 2023-08-11T14:01:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 Processor -- start 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 -- start start 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 0x7ff1b40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 0x7ff1b4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40739f0 con 0x7ff1b41024c0 2023-08-11T14:01:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4073b60 con 0x7ff1b4101b20 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.514+0000 7ff1b89ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b41a06d0 con 0x7ff1b4100930 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1abfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 0x7ff1b4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1abfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57512/0 (socket says 172.21.15.3:57512) 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1abfff700 1 -- 172.21.15.3:0/4061412080 learned_addr learned my addr 172.21.15.3:0/4061412080 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:40.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b259c700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 0x7ff1b40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 -- 172.21.15.3:0/4061412080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 msgr2=0x7ff1b40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 0x7ff1b40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 -- 172.21.15.3:0/4061412080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 msgr2=0x7ff1b4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 -- 172.21.15.3:0/4061412080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0095f0 con 0x7ff1b41024c0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1abfff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.515+0000 7ff1b2d9d700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 0x7ff1b4073250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00efc0 tx=0x7ff1ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.516+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac018070 con 0x7ff1b41024c0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.516+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1ac00f040 con 0x7ff1b41024c0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.516+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00cd40 con 0x7ff1b41024c0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.516+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b41a0950 con 0x7ff1b41024c0 2023-08-11T14:01:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.516+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b41a0e10 con 0x7ff1b41024c0 2023-08-11T14:01:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.517+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b4107ed0 con 0x7ff1b41024c0 2023-08-11T14:01:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.519+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff1ac003a00 con 0x7ff1b41024c0 2023-08-11T14:01:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.520+0000 7ff1ab7fe700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 0x7ff1a00a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.520+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff1ac0caf90 con 0x7ff1b41024c0 2023-08-11T14:01:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.520+0000 7ff1b259c700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 0x7ff1a00a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.521+0000 7ff1b259c700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 0x7ff1a00a24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4101980 tx=0x7ff1a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.524+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff1ac094b60 con 0x7ff1b41024c0 2023-08-11T14:01:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:40 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/326643967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:40 smithi103 ceph-mon[130434]: pgmap v276: 129 pgs: 14 creating+peering, 105 active+clean, 10 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 849 B/s wr, 6 op/s 2023-08-11T14:01:40.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:40 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/227078512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.735+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b410c190 con 0x7ff1b41024c0 2023-08-11T14:01:40.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.736+0000 7ff1ab7fe700 1 -- 172.21.15.3:0/4061412080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7ff1ac098180 con 0x7ff1b41024c0 2023-08-11T14:01:40.739 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:40.739 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:40.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.738+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 msgr2=0x7ff1a00a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.738+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 0x7ff1a00a24e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4101980 tx=0x7ff1a4009260 comp rx=0 tx=0).stop 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 msgr2=0x7ff1b4073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 0x7ff1b4073250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00efc0 tx=0x7ff1ac00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 shutdown_connections 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1b41024c0 0x7ff1b4073250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1b4100930 0x7ff1b40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1a00a0020 0x7ff1a00a24e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 --2- 172.21.15.3:0/4061412080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1b4101b20 0x7ff1b4072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 >> 172.21.15.3:0/4061412080 conn(0x7ff1b40fbe60 msgr2=0x7ff1b4105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 shutdown_connections 2023-08-11T14:01:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.739+0000 7ff1b89ff700 1 -- 172.21.15.3:0/4061412080 wait complete. 2023-08-11T14:01:40.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:40 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/326643967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:40 smithi003 ceph-mon[97195]: pgmap v276: 129 pgs: 14 creating+peering, 105 active+clean, 10 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 849 B/s wr, 6 op/s 2023-08-11T14:01:40.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:40 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/227078512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-11T14:01:32.831322+0000', 'modified': '2023-08-11T14:01:38.353686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.103:6839/2494449018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2494449018}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2494449018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-08-11T14:01:40.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-11T14:01:40.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T14:01:40.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:40 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/326643967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:40 smithi187 ceph-mon[130973]: pgmap v276: 129 pgs: 14 creating+peering, 105 active+clean, 10 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 849 B/s wr, 6 op/s 2023-08-11T14:01:40.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:40 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/227078512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:01:40.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.965+0000 7fc7747ff700 1 -- 172.21.15.3:0/1059599641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 msgr2=0x7fc76c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.965+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1059599641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c108d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc764009fc0 tx=0x7fc76400b330 comp rx=0 tx=0).stop 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 -- 172.21.15.3:0/1059599641 shutdown_connections 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1059599641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c108d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1059599641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c100910 0x7fc76c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1059599641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c101b00 0x7fc76c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 -- 172.21.15.3:0/1059599641 >> 172.21.15.3:0/1059599641 conn(0x7fc76c0fbe60 msgr2=0x7fc76c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 -- 172.21.15.3:0/1059599641 shutdown_connections 2023-08-11T14:01:40.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.966+0000 7fc7747ff700 1 -- 172.21.15.3:0/1059599641 wait complete. 2023-08-11T14:01:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 Processor -- start 2023-08-11T14:01:40.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 -- start start 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c100910 0x7fc76c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a0480 con 0x7fc76c101b00 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a05c0 con 0x7fc76c1024a0 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc7747ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a0750 con 0x7fc76c100910 2023-08-11T14:01:40.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52020/0 (socket says 172.21.15.3:52020) 2023-08-11T14:01:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc771d9a700 1 -- 172.21.15.3:0/1340751131 learned_addr learned my addr 172.21.15.3:0/1340751131 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc772d9c700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.967+0000 7fc77259b700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c100910 0x7fc76c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 -- 172.21.15.3:0/1340751131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c100910 msgr2=0x7fc76c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c100910 0x7fc76c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 -- 172.21.15.3:0/1340751131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 msgr2=0x7fc76c19feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c19feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 -- 172.21.15.3:0/1340751131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764009c40 con 0x7fc76c101b00 2023-08-11T14:01:40.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc7637fe700 1 -- 172.21.15.3:0/1340751131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c009790 con 0x7fc76c101b00 2023-08-11T14:01:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c1a0a30 con 0x7fc76c101b00 2023-08-11T14:01:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc7637fe700 1 -- 172.21.15.3:0/1340751131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc75c00ce80 con 0x7fc76c101b00 2023-08-11T14:01:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc7637fe700 1 -- 172.21.15.3:0/1340751131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c018680 con 0x7fc76c101b00 2023-08-11T14:01:40.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.968+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c1a0f90 con 0x7fc76c101b00 2023-08-11T14:01:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.972+0000 7fc7637fe700 1 -- 172.21.15.3:0/1340751131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc75c010430 con 0x7fc76c101b00 2023-08-11T14:01:40.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.973+0000 7fc788200b80 1 Processor -- start 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.973+0000 7fc788200b80 1 -- start start 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.973+0000 7fc7637fe700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 0x7fc7580a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.973+0000 7fc7637fe700 1 -- 172.21.15.3:0/1340751131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc75c014070 con 0x7fc76c101b00 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae0f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae0495e0 con 0x5557ae0f13e0 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae046f00 con 0x5557ae0f0a80 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc788200b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae0427f0 con 0x5557ae0f1da0 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc772d9c700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 0x7fc7580a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57544/0 (socket says 172.21.15.3:57544) 2023-08-11T14:01:40.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 -- 172.21.15.3:0/2364479935 learned_addr learned my addr 172.21.15.3:0/2364479935 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc772d9c700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae0f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 -- 172.21.15.3:0/2364479935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 msgr2=0x5557ae0f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae0f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 -- 172.21.15.3:0/2364479935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 msgr2=0x5557ae0f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc77259b700 1 -- 172.21.15.3:0/2364479935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557adec25e0 con 0x5557ae0f0a80 2023-08-11T14:01:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc772d9c700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae0f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.974+0000 7fc771d9a700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.975+0000 7fc772d9c700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 0x7fc7580a2410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc764009730 tx=0x7fc764007460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.975+0000 7fc77259b700 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc768009d30 tx=0x7fc768011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be9d1479fae529fa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.975+0000 7fc760ff9700 1 -- 172.21.15.3:0/2364479935 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768004220 con 0x5557ae0f0a80 2023-08-11T14:01:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.975+0000 7fc760ff9700 1 -- 172.21.15.3:0/2364479935 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc768004cf0 con 0x5557ae0f0a80 2023-08-11T14:01:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.975+0000 7fc760ff9700 1 -- 172.21.15.3:0/2364479935 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768005540 con 0x5557ae0f0a80 2023-08-11T14:01:40.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- 172.21.15.3:0/2364479935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 msgr2=0x5557ae0f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc768009d30 tx=0x7fc768011720 comp rx=0 tx=0).stop 2023-08-11T14:01:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- 172.21.15.3:0/2364479935 shutdown_connections 2023-08-11T14:01:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae0f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0f0ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 --2- 172.21.15.3:0/2364479935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- 172.21.15.3:0/2364479935 >> 172.21.15.3:0/2364479935 conn(0x5557ae01d900 msgr2=0x5557ae0eb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- 172.21.15.3:0/2364479935 shutdown_connections 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- 172.21.15.3:0/2364479935 wait complete. 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.976+0000 7fc788200b80 1 -- start start 2023-08-11T14:01:40.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 Processor -- start 2023-08-11T14:01:40.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae101680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557adfe0a70 con 0x5557ae0f13e0 2023-08-11T14:01:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557adfe0770 con 0x5557ae0f0a80 2023-08-11T14:01:40.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557adec48d0 con 0x5557ae0f1da0 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc788200b80 10 client.0 _ll_register_callbacks cb 0x5557adfe3a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc77259b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52042/0 (socket says 172.21.15.3:52042) 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc771d9a700 1 -- 172.21.15.3:0/1284529335 learned_addr learned my addr 172.21.15.3:0/1284529335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.977+0000 7fc772d9c700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 -- 172.21.15.3:0/1284529335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 msgr2=0x5557ae101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae101680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 -- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 msgr2=0x5557ae0fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 -- 172.21.15.3:0/1284529335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c1a0f90 con 0x5557ae0f13e0 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc771d9a700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc75c095370 tx=0x7fc75c099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37d72d6540a5eee server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc755ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c09ccf0 con 0x5557ae0f13e0 2023-08-11T14:01:40.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557ae0e3a40 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc75c0a0cd0 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc75c0a0840 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.978+0000 7fc788200b80 10 client.14625 fetch_fsmap learned FSMap version 17 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7fc768009960 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc75c064080 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc75c0b3030 con 0x5557ae0f13e0 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 10 client.14625 fetch_fsmap finished waiting for FSMap version 17 2023-08-11T14:01:40.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 10 client.14625 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:01:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 20 client.14625 populate_metadata read hostname 'smithi003' 2023-08-11T14:01:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae105cc0 0x5557ae1047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 4 client.14625 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:01:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.979+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557ae0bf490 con 0x5557ae105cc0 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.980+0000 7fc77259b700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae105cc0 0x5557ae1047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.980+0000 7fc755ffb700 10 client.14625 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.980+0000 7fc77259b700 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae105cc0 0x5557ae1047e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9adbe011473a23f3 server_cookie=8caca8b5043fbf29 in_seq=0 out_seq=0 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc755ffb700 1 -- 172.21.15.3:0/1284529335 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5557ae0bf490 con 0x5557ae105cc0 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc755ffb700 10 client.14625 handle_command_reply: tid=0 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 client.14625 shutdown 2023-08-11T14:01:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 msgr2=0x5557ae0fa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc75c095370 tx=0x7fc75c099400 comp rx=0 tx=0).stop 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 shutdown_connections 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f1da0 0x5557ae101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae105cc0 0x5557ae1047e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f0a80 0x5557ae0fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 --2- 172.21.15.3:0/1284529335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f13e0 0x5557ae0fa640 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 >> 172.21.15.3:0/1284529335 conn(0x5557ae01d900 msgr2=0x5557ae0e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:40.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.981+0000 7fc755ffb700 0 client.14625 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:01:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.982+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 shutdown_connections 2023-08-11T14:01:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.982+0000 7fc788200b80 1 -- 172.21.15.3:0/1284529335 wait complete. 2023-08-11T14:01:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:40.982+0000 7fc788200b80 20 client.14625 trim_cache size 0 max 16384 2023-08-11T14:01:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 Processor -- start 2023-08-11T14:01:41.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 -- start start 2023-08-11T14:01:41.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 0x5557ae104ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f13e0 0x5557ae0eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557adfdcd80 con 0x5557ae0f0a80 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae00bf20 con 0x5557ae01d900 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.000+0000 7fc788200b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae00cc30 con 0x5557ae0f13e0 2023-08-11T14:01:41.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52052/0 (socket says 172.21.15.3:52052) 2023-08-11T14:01:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 -- 172.21.15.3:0/469821295 learned_addr learned my addr 172.21.15.3:0/469821295 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc77259b700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 0x5557ae104ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc772d9c700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f13e0 0x5557ae0eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 -- 172.21.15.3:0/469821295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f13e0 msgr2=0x5557ae0eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f13e0 0x5557ae0eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 -- 172.21.15.3:0/469821295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 msgr2=0x5557ae104ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 0x5557ae104ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 -- 172.21.15.3:0/469821295 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557ade6ed90 con 0x5557ae0f0a80 2023-08-11T14:01:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc77259b700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 0x5557ae104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.001+0000 7fc771d9a700 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc75c098bc0 tx=0x7fc75c097470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f446af6ac84c921 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc7617fa700 1 -- 172.21.15.3:0/469821295 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0a0cd0 con 0x5557ae0f0a80 2023-08-11T14:01:41.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc7617fa700 1 -- 172.21.15.3:0/469821295 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557ae0e3a40 con 0x5557ae0f0a80 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc7617fa700 1 -- 172.21.15.3:0/469821295 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0b8310 con 0x5557ae0f0a80 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 -- 172.21.15.3:0/469821295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 msgr2=0x5557ae101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc75c098bc0 tx=0x7fc75c097470 comp rx=0 tx=0).stop 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 -- 172.21.15.3:0/469821295 shutdown_connections 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f13e0 0x5557ae0eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae01d900 0x5557ae104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 --2- 172.21.15.3:0/469821295 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae0f0a80 0x5557ae101610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 -- 172.21.15.3:0/469821295 >> 172.21.15.3:0/469821295 conn(0x5557ae0f9240 msgr2=0x5557ae1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 -- 172.21.15.3:0/469821295 shutdown_connections 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.002+0000 7fc788200b80 1 -- 172.21.15.3:0/469821295 wait complete. 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 -- start start 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 Processor -- start 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f0a80 0x5557ae0fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f13e0 0x5557ae159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae159a70 con 0x5557ae01d900 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae159bb0 con 0x5557ae0f13e0 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557ae159d20 con 0x5557ae0f0a80 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.003+0000 7fc788200b80 10 client.0 _ll_register_callbacks cb 0x7fc75c098860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52060/0 (socket says 172.21.15.3:52060) 2023-08-11T14:01:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 -- 172.21.15.3:0/2490422981 learned_addr learned my addr 172.21.15.3:0/2490422981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc771d9a700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f0a80 0x5557ae0fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc772d9c700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f13e0 0x5557ae159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 -- 172.21.15.3:0/2490422981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f0a80 msgr2=0x5557ae0fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f0a80 0x5557ae0fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 -- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f13e0 msgr2=0x5557ae159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f13e0 0x5557ae159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 -- 172.21.15.3:0/2490422981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557adec25e0 con 0x5557ae01d900 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc77259b700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc768009840 tx=0x7fc768005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d071a686696087 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.004+0000 7fc755ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768004710 con 0x5557ae01d900 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc768004cf0 con 0x5557ae01d900 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc76801f040 con 0x5557ae01d900 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc768004870 con 0x5557ae01d900 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 10 client.14631 fetch_fsmap learned FSMap version 17 2023-08-11T14:01:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7fc768009960 con 0x5557ae01d900 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc768041080 con 0x5557ae01d900 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fc76802c6e0 con 0x5557ae01d900 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 10 client.14631 fetch_fsmap finished waiting for FSMap version 17 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 10 client.14631 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 20 client.14631 populate_metadata read hostname 'smithi003' 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae0f1da0 0x5557ae15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:41.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 4 client.14631 mds_command: new command op to 24569 tid=0[{"prefix": "status"}] 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.005+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5557ae03b7d0 con 0x5557ae0f1da0 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.006+0000 7fc771d9a700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae0f1da0 0x5557ae15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.006+0000 7fc771d9a700 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae0f1da0 0x5557ae15b3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be89517358aea1 server_cookie=f5e5d5b7b0d0bd8e in_seq=0 out_seq=0 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.006+0000 7fc755ffb700 10 client.14631 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc755ffb700 1 -- 172.21.15.3:0/2490422981 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5557ae03b7d0 con 0x5557ae0f1da0 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc755ffb700 10 client.14631 handle_command_reply: tid=0 2023-08-11T14:01:41.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 client.14631 shutdown 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 msgr2=0x5557ae0fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc768009840 tx=0x7fc768005da0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 shutdown_connections 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557ae0f1da0 0x5557ae15b3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae0f13e0 0x5557ae159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5557ae01d900 0x5557ae0fc610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 --2- 172.21.15.3:0/2490422981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557ae0f0a80 0x5557ae0fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc755ffb700 0 client.14631 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:01:41.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 >> 172.21.15.3:0/2490422981 conn(0x5557ae0f9240 msgr2=0x5557ae0eb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:41.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 shutdown_connections 2023-08-11T14:01:41.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 1 -- 172.21.15.3:0/2490422981 wait complete. 2023-08-11T14:01:41.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.007+0000 7fc788200b80 20 client.14631 trim_cache size 0 max 16384 2023-08-11T14:01:41.018 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:01:41.018 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:01:41.018 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-08-11T14:01:41.018 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24569, 2023-08-11T14:01:41.018 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.6779711080000004, 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 17, 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 87, 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:01:41.019 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 13.915546374 2023-08-11T14:01:41.020 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:01:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 msgr2=0x7fc7580a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:41.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 0x7fc7580a2410 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc764009730 tx=0x7fc764007460 comp rx=0 tx=0).stop 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 msgr2=0x7fc76c198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9a0 tx=0x7fc75c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 shutdown_connections 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc75809ff50 0x7fc7580a2410 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc76c1024a0 0x7fc76c19feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c100910 0x7fc76c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 --2- 172.21.15.3:0/1340751131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc76c101b00 0x7fc76c198e40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 >> 172.21.15.3:0/1340751131 conn(0x7fc76c0fbe60 msgr2=0x7fc76c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 shutdown_connections 2023-08-11T14:01:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:41.008+0000 7fc7747ff700 1 -- 172.21.15.3:0/1340751131 wait complete. 2023-08-11T14:01:41.027 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T14:01:41.028 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:41.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph fs ls 2023-08-11T14:01:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4061412080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:41.753 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:01:41.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4061412080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:41.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4061412080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:43.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 -- 172.21.15.3:0/4226196619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c073670 msgr2=0x7f746c10cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 --2- 172.21.15.3:0/4226196619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c073670 0x7f746c10cc40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f745c009990 tx=0x7f745c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 -- 172.21.15.3:0/4226196619 shutdown_connections 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 --2- 172.21.15.3:0/4226196619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c10d180 0x7f746c10f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 --2- 172.21.15.3:0/4226196619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 --2- 172.21.15.3:0/4226196619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c073670 0x7f746c10cc40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 -- 172.21.15.3:0/4226196619 >> 172.21.15.3:0/4226196619 conn(0x7f746c0fc410 msgr2=0x7f746c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:43.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 -- 172.21.15.3:0/4226196619 shutdown_connections 2023-08-11T14:01:43.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.296+0000 7f747384d700 1 -- 172.21.15.3:0/4226196619 wait complete. 2023-08-11T14:01:43.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.297+0000 7f747384d700 1 Processor -- start 2023-08-11T14:01:43.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.297+0000 7f747384d700 1 -- start start 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.297+0000 7f747384d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.297+0000 7f747384d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c073670 0x7f746c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.297+0000 7f747384d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 0x7f746c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f747384d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c199150 con 0x7f746c10d180 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f747384d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c1992c0 con 0x7f746c073670 2023-08-11T14:01:43.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f747384d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c199480 con 0x7f746c072d50 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f7471dea700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 0x7f746c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60710/0 (socket says 172.21.15.3:60710) 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 -- 172.21.15.3:0/4158889412 learned_addr learned my addr 172.21.15.3:0/4158889412 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f7470de8700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c073670 0x7f746c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 -- 172.21.15.3:0/4158889412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c073670 msgr2=0x7f746c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:43.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c073670 0x7f746c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 -- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 msgr2=0x7f746c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 0x7f746c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f74715e9700 1 -- 172.21.15.3:0/4158889412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c0095f0 con 0x7f746c072d50 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.298+0000 7f7471dea700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 0x7f746c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f74715e9700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f746800e9a0 tx=0x7f746800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468009790 con 0x7f746c072d50 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7468010430 con 0x7f746c072d50 2023-08-11T14:01:43.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468018680 con 0x7f746c072d50 2023-08-11T14:01:43.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c199760 con 0x7f746c072d50 2023-08-11T14:01:43.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.299+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c1a5a50 con 0x7f746c072d50 2023-08-11T14:01:43.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.300+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c04e620 con 0x7f746c072d50 2023-08-11T14:01:43.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.302+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f74680105a0 con 0x7f746c072d50 2023-08-11T14:01:43.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.303+0000 7f74627fc700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 0x7f74580a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:43.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.303+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7468014070 con 0x7f746c072d50 2023-08-11T14:01:43.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.303+0000 7f7470de8700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 0x7f74580a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:43.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.304+0000 7f7470de8700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 0x7f74580a2410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f746c10e820 tx=0x7f745c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:43.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.307+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7468051c30 con 0x7f746c072d50 2023-08-11T14:01:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:43 smithi003 ceph-mon[97195]: pgmap v277: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2023-08-11T14:01:43.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:43 smithi187 ceph-mon[130973]: pgmap v277: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2023-08-11T14:01:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.560+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f746c199950 con 0x7f746c072d50 2023-08-11T14:01:43.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.561+0000 7f74627fc700 1 -- 172.21.15.3:0/4158889412 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f746c199950 con 0x7f746c072d50 2023-08-11T14:01:43.563 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T14:01:43.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 msgr2=0x7f74580a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:43.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 0x7f74580a2410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f746c10e820 tx=0x7f745c005fd0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 msgr2=0x7f746c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:43.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f746800e9a0 tx=0x7f746800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 shutdown_connections 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f746c10d180 0x7f746c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c072d50 0x7f746c19ea60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f745809ff50 0x7f74580a2410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 --2- 172.21.15.3:0/4158889412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f746c073670 0x7f746c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.563+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 >> 172.21.15.3:0/4158889412 conn(0x7f746c0fc410 msgr2=0x7f746c10f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.564+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 shutdown_connections 2023-08-11T14:01:43.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:43.564+0000 7f747384d700 1 -- 172.21.15.3:0/4158889412 wait complete. 2023-08-11T14:01:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:43 smithi103 ceph-mon[130434]: pgmap v277: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2023-08-11T14:01:44.039 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-11T14:01:44.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T14:01:44.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-08-11T14:01:44.040 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-08-11T14:01:44.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:44.041 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-11T14:01:44.041 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T14:01:44.041 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T14:01:44.041 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T14:01:44.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:44.042 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-08-11T14:01:44.087 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T14:01:44.087 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T14:01:44.087 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T14:01:44.088 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:01:44.088 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-08-11T14:01:44.088 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-08-11T14:01:44.089 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41235sec preferred_lft 41235sec 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:01:44.090 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:01:44.091 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-08-11T14:01:44.091 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-08-11T14:01:44.092 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T14:01:44.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:44.093 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:01:44.093 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:01:44.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-08-11T14:01:44.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-08-11T14:01:44.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-08-11T14:01:44.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T14:01:44.094 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:01:44.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:44 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4158889412' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:01:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:44 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4158889412' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:01:44.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:44.620 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T14:01:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:44 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4158889412' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:01:44.695 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-08-11T14:01:44.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:44.698 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-08-11T14:01:44.765 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-08-11T14:01:44.766 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T14:01:44.766 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T14:01:44.766 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T14:01:44.766 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T14:01:44.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:44.768 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:01:44.769 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:01:44.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T14:01:44.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T14:01:44.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T14:01:44.770 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:01:45.490 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:45 smithi003 ceph-mon[97195]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 524 B/s wr, 4 op/s 2023-08-11T14:01:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:45.494 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-08-11T14:01:45.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:45.540 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-08-11T14:01:45.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:45 smithi187 ceph-mon[130973]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 524 B/s wr, 4 op/s 2023-08-11T14:01:45.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:45.626 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:01:45.626 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:01:45.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T14:01:45.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-08-11T14:01:45.627 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:01:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:45 smithi103 ceph-mon[130434]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 524 B/s wr, 4 op/s 2023-08-11T14:01:45.811 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-08-11T14:01:45.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:45.812 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:01:45.812 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:01:45.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-08-11T14:01:45.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T14:01:45.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-08-11T14:01:45.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-08-11T14:01:45.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-08-11T14:01:45.814 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:01:46.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:46.108 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:01:46.108 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:01:46.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-08-11T14:01:46.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T14:01:46.109 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:01:46.286 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-08-11T14:01:46.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T14:01:46.287 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:46.333 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-08-11T14:01:46.334 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T14:01:46.334 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:46.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-08-11T14:01:46.527 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T14:01:46.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T14:01:46.597 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T14:01:46.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T14:01:46.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:46.601 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-08-11T14:01:46.644 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-11T14:01:46.644 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-08-11T14:01:46.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-08-11T14:01:46.761 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T14:01:46.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T14:01:46.832 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T14:01:46.835 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T14:01:46.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:46.836 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-08-11T14:01:46.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2023-08-11T14:01:46.888+0000 7f15307ce5c0 -1 init, newargv = 0x563f060d1140 newargc=15 2023-08-11T14:01:46.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[134455]: starting ceph client 2023-08-11T14:01:46.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[134455]: starting fuse 2023-08-11T14:01:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:47 smithi003 ceph-mon[97195]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2023-08-11T14:01:47.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:47 smithi187 ceph-mon[130973]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2023-08-11T14:01:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:47 smithi103 ceph-mon[130434]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2023-08-11T14:01:47.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-08-11T14:01:48.063 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-11T14:01:48.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-11T14:01:48.135 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-11T14:01:48.138 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-11T14:01:48.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:48.140 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-08-11T14:01:48.184 INFO:teuthology.orchestra.run.smithi003.stdout:121 2023-08-11T14:01:48.184 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-08-11T14:01:48.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2023-08-11T14:01:48.185 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-08-11T14:01:48.185 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-08-11T14:01:48.186 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-08-11T14:01:48.186 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-08-11T14:01:48.186 DEBUG:teuthology.orchestra.run.smithi003:> 2023-08-11T14:01:48.186 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-08-11T14:01:48.187 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-08-11T14:01:48.187 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-08-11T14:01:48.187 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-08-11T14:01:48.188 DEBUG:teuthology.orchestra.run.smithi003:> 2023-08-11T14:01:48.188 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-08-11T14:01:48.188 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-08-11T14:01:48.188 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-08-11T14:01:48.189 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-08-11T14:01:48.189 DEBUG:teuthology.orchestra.run.smithi003:> 2023-08-11T14:01:48.189 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-08-11T14:01:48.189 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-11T14:01:48.190 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-11T14:01:48.190 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-11T14:01:48.190 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-08-11T14:01:48.190 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-08-11T14:01:48.191 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-08-11T14:01:48.191 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-11T14:01:48.191 DEBUG:teuthology.orchestra.run.smithi003:> 2023-08-11T14:01:48.192 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2023-08-11T14:01:48.192 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-08-11T14:01:48.372 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.134455.asok 2023-08-11T14:01:48.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-11T14:01:48.386 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.134455.asok 2023-08-11T14:01:48.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:48.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.134455.asok status 2023-08-11T14:01:48.617 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:01:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-08-11T14:01:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "92684807f016ce045ac66e37fdde6f8c47857bcc", 2023-08-11T14:01:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.0.0-5437-g92684807 (92684807f016ce045ac66e37fdde6f8c47857bcc) reef (dev)", 2023-08-11T14:01:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2023-08-11T14:01:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-08-11T14:01:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-08-11T14:01:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "134455", 2023-08-11T14:01:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-08-11T14:01:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:01:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-08-11T14:01:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-08-11T14:01:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14643, 2023-08-11T14:01:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-08-11T14:01:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-08-11T14:01:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-08-11T14:01:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 14643 2023-08-11T14:01:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:01:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-08-11T14:01:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-08-11T14:01:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:01:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1138218622 2023-08-11T14:01:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:01:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:01:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-08-11T14:01:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-08-11T14:01:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:01:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1138218622 2023-08-11T14:01:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:01:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.14643 172.21.15.3:0/1138218622", 2023-08-11T14:01:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "172.21.15.3:0/1138218622", 2023-08-11T14:01:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-08-11T14:01:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 17, 2023-08-11T14:01:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 87, 2023-08-11T14:01:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-08-11T14:01:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-08-11T14:01:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-08-11T14:01:48.626 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:01:48.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:48.633 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-08-11T14:01:48.634 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-08-11T14:01:48.679 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:48.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:48.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:48.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:48.791 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-08-11T14:01:48.912 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2023-08-11T14:01:48.912 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-08-11T14:01:48.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:48.913 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:01:48.914 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:01:48.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 -- 172.21.15.3:0/2316020783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 msgr2=0x7fd924072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/2316020783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924072db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd918009990 tx=0x7fd918009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 -- 172.21.15.3:0/2316020783 shutdown_connections 2023-08-11T14:01:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/2316020783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9241087e0 0x7fd92410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/2316020783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924072db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/2316020783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd924073380 0x7fd924073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 -- 172.21.15.3:0/2316020783 >> 172.21.15.3:0/2316020783 conn(0x7fd9240fbe50 msgr2=0x7fd9240fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 -- 172.21.15.3:0/2316020783 shutdown_connections 2023-08-11T14:01:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.198+0000 7fd92b1ff700 1 -- 172.21.15.3:0/2316020783 wait complete. 2023-08-11T14:01:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.200+0000 7fd92b1ff700 1 Processor -- start 2023-08-11T14:01:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.200+0000 7fd92b1ff700 1 -- start start 2023-08-11T14:01:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd924073380 0x7fd9241987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9241087e0 0x7fd92419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a0360 con 0x7fd924074960 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a04a0 con 0x7fd924073380 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92b1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a0630 con 0x7fd9241087e0 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd928f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd924073380 0x7fd9241987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35222/0 (socket says 172.21.15.3:35222) 2023-08-11T14:01:49.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 -- 172.21.15.3:0/3516396228 learned_addr learned my addr 172.21.15.3:0/3516396228 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd92979c700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9241087e0 0x7fd92419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 -- 172.21.15.3:0/3516396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9241087e0 msgr2=0x7fd92419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9241087e0 0x7fd92419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 -- 172.21.15.3:0/3516396228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd924073380 msgr2=0x7fd9241987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd924073380 0x7fd9241987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.201+0000 7fd923fff700 1 -- 172.21.15.3:0/3516396228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180095f0 con 0x7fd924074960 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd923fff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e970 tx=0x7fd91000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:49.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910009790 con 0x7fd924074960 2023-08-11T14:01:49.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9241a0910 con 0x7fd924074960 2023-08-11T14:01:49.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91000c7e0 con 0x7fd924074960 2023-08-11T14:01:49.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910003c70 con 0x7fd924074960 2023-08-11T14:01:49.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.202+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a0e20 con 0x7fd924074960 2023-08-11T14:01:49.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.203+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd924106e90 con 0x7fd924074960 2023-08-11T14:01:49.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.205+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd91000cc50 con 0x7fd924074960 2023-08-11T14:01:49.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.206+0000 7fd921ffb700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 0x7fd9140a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.207+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd910014070 con 0x7fd924074960 2023-08-11T14:01:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.207+0000 7fd928f9b700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 0x7fd9140a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.207+0000 7fd928f9b700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 0x7fd9140a2460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd91800b2e0 tx=0x7fd918005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:49.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.211+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd910095170 con 0x7fd924074960 2023-08-11T14:01:49.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.426+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd92410e310 con 0x7fd924074960 2023-08-11T14:01:49.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.428+0000 7fd921ffb700 1 -- 172.21.15.3:0/3516396228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7fd910003dd0 con 0x7fd924074960 2023-08-11T14:01:49.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:49.430 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.430+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 msgr2=0x7fd9140a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.430+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 0x7fd9140a2460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd91800b2e0 tx=0x7fd918005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.430+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 msgr2=0x7fd924198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.430+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e970 tx=0x7fd91000ec80 comp rx=0 tx=0).stop 2023-08-11T14:01:49.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 shutdown_connections 2023-08-11T14:01:49.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9241087e0 0x7fd92419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd91409ffa0 0x7fd9140a2460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd924073380 0x7fd9241987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 --2- 172.21.15.3:0/3516396228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd924074960 0x7fd924198d20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 >> 172.21.15.3:0/3516396228 conn(0x7fd9240fbe50 msgr2=0x7fd9240fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 shutdown_connections 2023-08-11T14:01:49.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.431+0000 7fd92b1ff700 1 -- 172.21.15.3:0/3516396228 wait complete. 2023-08-11T14:01:49.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:49 smithi003 ceph-mon[97195]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-08-11T14:01:49.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:01:49.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:01:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 -- 172.21.15.3:0/2757193451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 msgr2=0x7fde4c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 --2- 172.21.15.3:0/2757193451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c072db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009990 tx=0x7fde3c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 -- 172.21.15.3:0/2757193451 shutdown_connections 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 --2- 172.21.15.3:0/2757193451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 0x7fde4c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 --2- 172.21.15.3:0/2757193451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c072db0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 --2- 172.21.15.3:0/2757193451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 0x7fde4c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.656+0000 7fde523ff700 1 -- 172.21.15.3:0/2757193451 >> 172.21.15.3:0/2757193451 conn(0x7fde4c0fbea0 msgr2=0x7fde4c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.657+0000 7fde523ff700 1 -- 172.21.15.3:0/2757193451 shutdown_connections 2023-08-11T14:01:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.657+0000 7fde523ff700 1 -- 172.21.15.3:0/2757193451 wait complete. 2023-08-11T14:01:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.657+0000 7fde523ff700 1 Processor -- start 2023-08-11T14:01:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 -- start start 2023-08-11T14:01:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 0x7fde4c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 0x7fde4c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1a0320 con 0x7fde4c074960 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1a0460 con 0x7fde4c1087f0 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde523ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1a05f0 con 0x7fde4c073380 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35240/0 (socket says 172.21.15.3:35240) 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde43fff700 1 -- 172.21.15.3:0/2879426257 learned_addr learned my addr 172.21.15.3:0/2879426257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde5099c700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 0x7fde4c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.658+0000 7fde4bfff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 0x7fde4c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 -- 172.21.15.3:0/2879426257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 msgr2=0x7fde4c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 0x7fde4c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 -- 172.21.15.3:0/2879426257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 msgr2=0x7fde4c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 0x7fde4c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 -- 172.21.15.3:0/2879426257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0095f0 con 0x7fde4c074960 2023-08-11T14:01:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde43fff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d980 tx=0x7fde3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400f890 con 0x7fde4c074960 2023-08-11T14:01:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde34010460 con 0x7fde4c074960 2023-08-11T14:01:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340195c0 con 0x7fde4c074960 2023-08-11T14:01:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c1a08d0 con 0x7fde4c074960 2023-08-11T14:01:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.659+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c1a0d90 con 0x7fde4c074960 2023-08-11T14:01:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.660+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c106ea0 con 0x7fde4c074960 2023-08-11T14:01:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.662+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fde340105d0 con 0x7fde4c074960 2023-08-11T14:01:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.664+0000 7fde49ffb700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 0x7fde380a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.664+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fde340cb160 con 0x7fde4c074960 2023-08-11T14:01:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.664+0000 7fde4bfff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 0x7fde380a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.664+0000 7fde4bfff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 0x7fde380a2490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b5c0 tx=0x7fde3c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.668+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fde34094ca0 con 0x7fde4c074960 2023-08-11T14:01:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:49 smithi103 ceph-mon[130434]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-08-11T14:01:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:49 smithi187 ceph-mon[130973]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2023-08-11T14:01:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.877+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde4c02cdc0 con 0x7fde4c074960 2023-08-11T14:01:49.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.877+0000 7fde49ffb700 1 -- 172.21.15.3:0/2879426257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde340982c0 con 0x7fde4c074960 2023-08-11T14:01:49.879 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:01:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 msgr2=0x7fde380a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 0x7fde380a2490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b5c0 tx=0x7fde3c005960 comp rx=0 tx=0).stop 2023-08-11T14:01:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 msgr2=0x7fde4c198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d980 tx=0x7fde3400dc90 comp rx=0 tx=0).stop 2023-08-11T14:01:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 shutdown_connections 2023-08-11T14:01:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde3809ffd0 0x7fde380a2490 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde4c1087f0 0x7fde4c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde4c073380 0x7fde4c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 --2- 172.21.15.3:0/2879426257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde4c074960 0x7fde4c198ce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 >> 172.21.15.3:0/2879426257 conn(0x7fde4c0fbea0 msgr2=0x7fde4c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 shutdown_connections 2023-08-11T14:01:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:49.880+0000 7fde523ff700 1 -- 172.21.15.3:0/2879426257 wait complete. 2023-08-11T14:01:49.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:01:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.104+0000 7f02abdff700 1 -- 172.21.15.3:0/2959622466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 msgr2=0x7f02a4108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 --2- 172.21.15.3:0/2959622466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a4108d50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f029c00b0d0 tx=0x7f029c00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 -- 172.21.15.3:0/2959622466 shutdown_connections 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 --2- 172.21.15.3:0/2959622466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a4108d50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 --2- 172.21.15.3:0/2959622466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4100910 0x7f02a4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 --2- 172.21.15.3:0/2959622466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4101b00 0x7f02a4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 -- 172.21.15.3:0/2959622466 >> 172.21.15.3:0/2959622466 conn(0x7f02a40fbe60 msgr2=0x7f02a40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 -- 172.21.15.3:0/2959622466 shutdown_connections 2023-08-11T14:01:50.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.105+0000 7f02abdff700 1 -- 172.21.15.3:0/2959622466 wait complete. 2023-08-11T14:01:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 Processor -- start 2023-08-11T14:01:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 -- start start 2023-08-11T14:01:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 0x7f02a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4101b00 0x7f02a4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a04d0 con 0x7f02a41024a0 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a0610 con 0x7f02a4100910 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.106+0000 7f02abdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a07a0 con 0x7f02a4101b00 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35256/0 (socket says 172.21.15.3:35256) 2023-08-11T14:01:50.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 -- 172.21.15.3:0/3620507600 learned_addr learned my addr 172.21.15.3:0/3620507600 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02a9b9b700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 0x7f02a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02a939a700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4101b00 0x7f02a4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 -- 172.21.15.3:0/3620507600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4101b00 msgr2=0x7f02a4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4101b00 0x7f02a4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 -- 172.21.15.3:0/3620507600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 msgr2=0x7f02a4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 0x7f02a4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 -- 172.21.15.3:0/3620507600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009520 con 0x7f02a41024a0 2023-08-11T14:01:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02a9b9b700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 0x7f02a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.107+0000 7f02aa39c700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f029c008eb0 tx=0x7f029c008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.108+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e040 con 0x7f02a41024a0 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.108+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029c00bdb0 con 0x7f02a41024a0 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.108+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c009d00 con 0x7f02a41024a0 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.108+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c01d470 con 0x7f02a41024a0 2023-08-11T14:01:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.108+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41a0d60 con 0x7f02a41024a0 2023-08-11T14:01:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.109+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a4192f70 con 0x7f02a41024a0 2023-08-11T14:01:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.112+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f029c005000 con 0x7f02a41024a0 2023-08-11T14:01:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.113+0000 7f029affd700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 0x7f02900a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.114+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f029c0cc9c0 con 0x7f02a41024a0 2023-08-11T14:01:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.114+0000 7f02a9b9b700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 0x7f02900a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.114+0000 7f02a9b9b700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 0x7f02900a2490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02a4101960 tx=0x7f02a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:50.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.115+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f029c096590 con 0x7f02a41024a0 2023-08-11T14:01:50.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.328+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02a410c390 con 0x7f02a41024a0 2023-08-11T14:01:50.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.328+0000 7f029affd700 1 -- 172.21.15.3:0/3620507600 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f029c094e60 con 0x7f02a41024a0 2023-08-11T14:01:50.330 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:01:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.330+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 msgr2=0x7f02900a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.330+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 0x7f02900a2490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02a4101960 tx=0x7f02a0009260 comp rx=0 tx=0).stop 2023-08-11T14:01:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.330+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 msgr2=0x7f02a419ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.330+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f029c008eb0 tx=0x7f029c008ee0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 shutdown_connections 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02a41024a0 0x7f02a419ff00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029009ffd0 0x7f02900a2490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02a4100910 0x7f02a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 --2- 172.21.15.3:0/3620507600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02a4101b00 0x7f02a4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 >> 172.21.15.3:0/3620507600 conn(0x7f02a40fbe60 msgr2=0x7f02a41073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 shutdown_connections 2023-08-11T14:01:50.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.331+0000 7f02abdff700 1 -- 172.21.15.3:0/3620507600 wait complete. 2023-08-11T14:01:50.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:01:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 -- 172.21.15.3:0/1897526944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 msgr2=0x7f4770100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 --2- 172.21.15.3:0/1897526944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f4770100d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f476c009920 tx=0x7f476c009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 -- 172.21.15.3:0/1897526944 shutdown_connections 2023-08-11T14:01:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 --2- 172.21.15.3:0/1897526944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47701024a0 0x7f4770108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 --2- 172.21.15.3:0/1897526944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f4770100d30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 --2- 172.21.15.3:0/1897526944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4770101b00 0x7f4770101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 -- 172.21.15.3:0/1897526944 >> 172.21.15.3:0/1897526944 conn(0x7f47700fbe60 msgr2=0x7f47700fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:50.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 -- 172.21.15.3:0/1897526944 shutdown_connections 2023-08-11T14:01:50.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.556+0000 7f47783ff700 1 -- 172.21.15.3:0/1897526944 wait complete. 2023-08-11T14:01:50.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.557+0000 7f47783ff700 1 Processor -- start 2023-08-11T14:01:50.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.557+0000 7f47783ff700 1 -- start start 2023-08-11T14:01:50.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.557+0000 7f47783ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f47701988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f47783ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 0x7f4770198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f47783ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47701024a0 0x7f477019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f47783ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0430 con 0x7f4770101b00 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f47783ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0570 con 0x7f4770100910 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f47783ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701a0700 con 0x7f47701024a0 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477599a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 0x7f4770198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f47701988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477599a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 0x7f4770198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35264/0 (socket says 172.21.15.3:35264) 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477599a700 1 -- 172.21.15.3:0/679468691 learned_addr learned my addr 172.21.15.3:0/679468691 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477699c700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47701024a0 0x7f477019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 -- 172.21.15.3:0/679468691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47701024a0 msgr2=0x7f477019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47701024a0 0x7f477019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 -- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 msgr2=0x7f4770198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 0x7f4770198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.558+0000 7f477619b700 1 -- 172.21.15.3:0/679468691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009520 con 0x7f4770100910 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f477619b700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f47701988b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f476c000c00 tx=0x7f476c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c01c070 con 0x7f4770100910 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47701a0980 con 0x7f4770100910 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f476c00f7d0 con 0x7f4770100910 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0038e0 con 0x7f4770100910 2023-08-11T14:01:50.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.559+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47701a0db0 con 0x7f4770100910 2023-08-11T14:01:50.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.560+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770192f10 con 0x7f4770100910 2023-08-11T14:01:50.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.562+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f476c020030 con 0x7f4770100910 2023-08-11T14:01:50.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.563+0000 7f47677fe700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 0x7f475c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:50.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.563+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f476c0cc640 con 0x7f4770100910 2023-08-11T14:01:50.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.564+0000 7f477599a700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 0x7f475c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:50.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.564+0000 7f477599a700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 0x7f475c0a2410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4760009d20 tx=0x7f4760009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:50.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.565+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f476c096210 con 0x7f4770100910 2023-08-11T14:01:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3516396228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2879426257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.777+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f477010c2b0 con 0x7f4770100910 2023-08-11T14:01:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.778+0000 7f47677fe700 1 -- 172.21.15.3:0/679468691 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f476c099830 con 0x7f4770100910 2023-08-11T14:01:50.780 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:01:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 msgr2=0x7f475c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 0x7f475c0a2410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4760009d20 tx=0x7f4760009260 comp rx=0 tx=0).stop 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 msgr2=0x7f47701988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f47701988b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f476c000c00 tx=0x7f476c00b790 comp rx=0 tx=0).stop 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 shutdown_connections 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47701024a0 0x7f477019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f475c09ff50 0x7f475c0a2410 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770100910 0x7f47701988b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 --2- 172.21.15.3:0/679468691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4770101b00 0x7f4770198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 >> 172.21.15.3:0/679468691 conn(0x7f47700fbe60 msgr2=0x7f4770107520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 shutdown_connections 2023-08-11T14:01:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:50.781+0000 7f47783ff700 1 -- 172.21.15.3:0/679468691 wait complete. 2023-08-11T14:01:50.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3516396228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:50.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2879426257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:50.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3516396228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:50.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2879426257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:50.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:01:51.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 -- 172.21.15.3:0/2761180223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34281087d0 msgr2=0x7f342810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 --2- 172.21.15.3:0/2761180223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34281087d0 0x7f342810ac60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f342000b0a0 tx=0x7f342000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 -- 172.21.15.3:0/2761180223 shutdown_connections 2023-08-11T14:01:51.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 --2- 172.21.15.3:0/2761180223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34281087d0 0x7f342810ac60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 --2- 172.21.15.3:0/2761180223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 0x7f3428072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 --2- 172.21.15.3:0/2761180223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3428073380 0x7f3428073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 -- 172.21.15.3:0/2761180223 >> 172.21.15.3:0/2761180223 conn(0x7f34280fbe60 msgr2=0x7f34280fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 -- 172.21.15.3:0/2761180223 shutdown_connections 2023-08-11T14:01:51.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.005+0000 7f342d3ff700 1 -- 172.21.15.3:0/2761180223 wait complete. 2023-08-11T14:01:51.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.006+0000 7f342d3ff700 1 Processor -- start 2023-08-11T14:01:51.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.006+0000 7f342d3ff700 1 -- start start 2023-08-11T14:01:51.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 0x7f3428198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34281087d0 0x7f342819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34281a04e0 con 0x7f3428073380 2023-08-11T14:01:51.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34281a0620 con 0x7f34281087d0 2023-08-11T14:01:51.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f342d3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34281a07b0 con 0x7f3428074960 2023-08-11T14:01:51.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35280/0 (socket says 172.21.15.3:35280) 2023-08-11T14:01:51.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 -- 172.21.15.3:0/3756445917 learned_addr learned my addr 172.21.15.3:0/3756445917 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:51.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f34267fc700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 0x7f3428198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f34277fe700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34281087d0 0x7f342819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 -- 172.21.15.3:0/3756445917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 msgr2=0x7f3428198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 0x7f3428198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 -- 172.21.15.3:0/3756445917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34281087d0 msgr2=0x7f342819ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34281087d0 0x7f342819ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.007+0000 7f3426ffd700 1 -- 172.21.15.3:0/3756445917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3420009d00 con 0x7f3428073380 2023-08-11T14:01:51.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f3426ffd700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9a0 tx=0x7f341800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:51.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418009790 con 0x7f3428073380 2023-08-11T14:01:51.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3418010430 con 0x7f3428073380 2023-08-11T14:01:51.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34281a0a90 con 0x7f3428073380 2023-08-11T14:01:51.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418018680 con 0x7f3428073380 2023-08-11T14:01:51.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.008+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34281a0fa0 con 0x7f3428073380 2023-08-11T14:01:51.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.009+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3428106e80 con 0x7f3428073380 2023-08-11T14:01:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.011+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f34180105a0 con 0x7f3428073380 2023-08-11T14:01:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.012+0000 7f340ffff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 0x7f34100a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.013+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3418014070 con 0x7f3428073380 2023-08-11T14:01:51.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.013+0000 7f34267fc700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 0x7f34100a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.013+0000 7f34267fc700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 0x7f34100a2410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3428074680 tx=0x7f341c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.017+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3418051c30 con 0x7f3428073380 2023-08-11T14:01:51.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.227+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f342802cb70 con 0x7f3428073380 2023-08-11T14:01:51.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.228+0000 7f340ffff700 1 -- 172.21.15.3:0/3756445917 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3418095180 con 0x7f3428073380 2023-08-11T14:01:51.229 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:01:51.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 msgr2=0x7f34100a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 0x7f34100a2410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3428074680 tx=0x7f341c009260 comp rx=0 tx=0).stop 2023-08-11T14:01:51.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 msgr2=0x7f3428198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9a0 tx=0x7f341800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 shutdown_connections 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f341009ff50 0x7f34100a2410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34281087d0 0x7f342819ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3428073380 0x7f3428198960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 --2- 172.21.15.3:0/3756445917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3428074960 0x7f3428198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.230+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 >> 172.21.15.3:0/3756445917 conn(0x7f34280fbe60 msgr2=0x7f34280fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:51.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.231+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 shutdown_connections 2023-08-11T14:01:51.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.231+0000 7f342d3ff700 1 -- 172.21.15.3:0/3756445917 wait complete. 2023-08-11T14:01:51.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T14:01:51.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 -- 172.21.15.3:0/782980931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 msgr2=0x7f35c0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 --2- 172.21.15.3:0/782980931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0101f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f35a80099c0 tx=0x7f35a8009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 -- 172.21.15.3:0/782980931 shutdown_connections 2023-08-11T14:01:51.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 --2- 172.21.15.3:0/782980931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c01024c0 0x7f35c0108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 --2- 172.21.15.3:0/782980931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c0100930 0x7f35c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 --2- 172.21.15.3:0/782980931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0101f80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 -- 172.21.15.3:0/782980931 >> 172.21.15.3:0/782980931 conn(0x7f35c00fbea0 msgr2=0x7f35c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 -- 172.21.15.3:0/782980931 shutdown_connections 2023-08-11T14:01:51.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.456+0000 7f35c4dff700 1 -- 172.21.15.3:0/782980931 wait complete. 2023-08-11T14:01:51.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.457+0000 7f35c4dff700 1 Processor -- start 2023-08-11T14:01:51.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.457+0000 7f35c4dff700 1 -- start start 2023-08-11T14:01:51.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c0100930 0x7f35c01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c01024c0 0x7f35c019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01a0350 con 0x7f35c0101b20 2023-08-11T14:01:51.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01a0490 con 0x7f35c0100930 2023-08-11T14:01:51.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35c4dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01a0620 con 0x7f35c01024c0 2023-08-11T14:01:51.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35300/0 (socket says 172.21.15.3:35300) 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35be59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c0100930 0x7f35c01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 -- 172.21.15.3:0/230189093 learned_addr learned my addr 172.21.15.3:0/230189093 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35bed9d700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c01024c0 0x7f35c019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 -- 172.21.15.3:0/230189093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c01024c0 msgr2=0x7f35c019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c01024c0 0x7f35c019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 -- 172.21.15.3:0/230189093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c0100930 msgr2=0x7f35c01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c0100930 0x7f35c01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.458+0000 7f35b7fff700 1 -- 172.21.15.3:0/230189093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80095f0 con 0x7f35c0101b20 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35b7fff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a801d070 con 0x7f35c0101b20 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c01a08a0 con 0x7f35c0101b20 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a80054f0 con 0x7f35c0101b20 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8004bd0 con 0x7f35c0101b20 2023-08-11T14:01:51.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.459+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c01a0d20 con 0x7f35c0101b20 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.460+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c0192f30 con 0x7f35c0101b20 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.462+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f35a800fb80 con 0x7f35c0101b20 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.463+0000 7f35b77fe700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 0x7f35ac0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.464+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f35a80cc260 con 0x7f35c0101b20 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.464+0000 7f35be59c700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 0x7f35ac0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:51.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.464+0000 7f35be59c700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 0x7f35ac0a2490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35c0101980 tx=0x7f35b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:51.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.467+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f35a8095e30 con 0x7f35c0101b20 2023-08-11T14:01:51.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:51 smithi003 ceph-mon[97195]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 346 B/s wr, 0 op/s 2023-08-11T14:01:51.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3620507600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/679468691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3756445917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:51 smithi103 ceph-mon[130434]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 346 B/s wr, 0 op/s 2023-08-11T14:01:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3620507600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/679468691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3756445917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.696+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35c010c140 con 0x7f35c0101b20 2023-08-11T14:01:51.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.697+0000 7f35b77fe700 1 -- 172.21.15.3:0/230189093 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+580 (secure 0 0 0) 0x7f35a8099450 con 0x7f35c0101b20 2023-08-11T14:01:51.699 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2366653083 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:51.699 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1070970995 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1776709742 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:51.700 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3848927136 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:51.701 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3010670797 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:51.701 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1680525538 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 msgr2=0x7f35ac0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 0x7f35ac0a2490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35c0101980 tx=0x7f35b0009260 comp rx=0 tx=0).stop 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 msgr2=0x7f35c0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a800f690 comp rx=0 tx=0).stop 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 shutdown_connections 2023-08-11T14:01:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c01024c0 0x7f35c019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f35ac09ffd0 0x7f35ac0a2490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35c0100930 0x7f35c01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 --2- 172.21.15.3:0/230189093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f35c0101b20 0x7f35c0198d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 >> 172.21.15.3:0/230189093 conn(0x7f35c00fbea0 msgr2=0x7f35c0107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 shutdown_connections 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:51.699+0000 7f35c4dff700 1 -- 172.21.15.3:0/230189093 wait complete. 2023-08-11T14:01:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2023-08-11T14:01:51.719 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:01:51.720 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:01:51.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:51 smithi187 ceph-mon[130973]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 346 B/s wr, 0 op/s 2023-08-11T14:01:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3620507600' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/679468691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:51.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3756445917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:52.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 -- 172.21.15.3:0/2538533298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae8103d20 msgr2=0x7f0ae8104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/2538533298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae8103d20 0x7f0ae8104180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0099c0 tx=0x7f0adc009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 -- 172.21.15.3:0/2538533298 shutdown_connections 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/2538533298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae81046c0 0x7f0ae810af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/2538533298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae8102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/2538533298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae8103d20 0x7f0ae8104180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 -- 172.21.15.3:0/2538533298 >> 172.21.15.3:0/2538533298 conn(0x7f0ae80fe0a0 msgr2=0x7f0ae81004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:52.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 -- 172.21.15.3:0/2538533298 shutdown_connections 2023-08-11T14:01:52.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.009+0000 7f0aee9ff700 1 -- 172.21.15.3:0/2538533298 wait complete. 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.010+0000 7f0aee9ff700 1 Processor -- start 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.010+0000 7f0aee9ff700 1 -- start start 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.010+0000 7f0aee9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aee9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae8103d20 0x7f0ae8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aee9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 0x7f0ae819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aee9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae81a0340 con 0x7f0ae81046c0 2023-08-11T14:01:52.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aee9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae81a0480 con 0x7f0ae8102b30 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aee9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae81a0610 con 0x7f0ae8103d20 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0ae7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 0x7f0ae819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae8103d20 0x7f0ae8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0ae7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae81987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48804/0 (socket says 172.21.15.3:48804) 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0ae7fff700 1 -- 172.21.15.3:0/486301439 learned_addr learned my addr 172.21.15.3:0/486301439 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:52.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 -- 172.21.15.3:0/486301439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae8103d20 msgr2=0x7f0ae8198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae8103d20 0x7f0ae8198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 -- 172.21.15.3:0/486301439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 msgr2=0x7f0ae81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.011+0000 7f0aecf9c700 1 -- 172.21.15.3:0/486301439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8009520 con 0x7f0ae81046c0 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0aecf9c700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 0x7f0ae819fd70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000efc0 tx=0x7f0ae000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018070 con 0x7f0ae81046c0 2023-08-11T14:01:52.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae000f040 con 0x7f0ae81046c0 2023-08-11T14:01:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc0095f0 con 0x7f0ae81046c0 2023-08-11T14:01:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000cd40 con 0x7f0ae81046c0 2023-08-11T14:01:52.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.012+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae81a0bc0 con 0x7f0ae81046c0 2023-08-11T14:01:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.013+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae810e390 con 0x7f0ae81046c0 2023-08-11T14:01:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.015+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0ae0003a00 con 0x7f0ae81046c0 2023-08-11T14:01:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.016+0000 7f0ae57fa700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 0x7f0ad00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.016+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ae00caff0 con 0x7f0ae81046c0 2023-08-11T14:01:52.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.016+0000 7f0ae7fff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 0x7f0ad00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.017+0000 7f0ae7fff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 0x7f0ad00a2460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8005950 tx=0x7f0ad8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.020+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0ae0094c20 con 0x7f0ae81046c0 2023-08-11T14:01:52.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.232+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ae810e5a0 con 0x7f0ae81046c0 2023-08-11T14:01:52.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.234+0000 7f0ae57fa700 1 -- 172.21.15.3:0/486301439 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7f0ae0098240 con 0x7f0ae81046c0 2023-08-11T14:01:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:52.236 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 msgr2=0x7f0ad00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 0x7f0ad00a2460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8005950 tx=0x7f0ad8009260 comp rx=0 tx=0).stop 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 msgr2=0x7f0ae819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 0x7f0ae819fd70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000efc0 tx=0x7f0ae000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 shutdown_connections 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae81046c0 0x7f0ae819fd70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0ad009ffa0 0x7f0ad00a2460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ae8102b30 0x7f0ae81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 --2- 172.21.15.3:0/486301439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae8103d20 0x7f0ae8198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 >> 172.21.15.3:0/486301439 conn(0x7f0ae80fe0a0 msgr2=0x7f0ae81002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 shutdown_connections 2023-08-11T14:01:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.236+0000 7f0aee9ff700 1 -- 172.21.15.3:0/486301439 wait complete. 2023-08-11T14:01:52.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:01:52.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:01:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 -- 172.21.15.3:0/802003406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc801024a0 msgr2=0x7fbc80108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 --2- 172.21.15.3:0/802003406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc801024a0 0x7fbc80108d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800b0d0 tx=0x7fbc7800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 -- 172.21.15.3:0/802003406 shutdown_connections 2023-08-11T14:01:52.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 --2- 172.21.15.3:0/802003406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc801024a0 0x7fbc80108d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 --2- 172.21.15.3:0/802003406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 --2- 172.21.15.3:0/802003406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc80101b00 0x7fbc80101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 -- 172.21.15.3:0/802003406 >> 172.21.15.3:0/802003406 conn(0x7fbc800fbe60 msgr2=0x7fbc800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 -- 172.21.15.3:0/802003406 shutdown_connections 2023-08-11T14:01:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.465+0000 7fbc877ff700 1 -- 172.21.15.3:0/802003406 wait complete. 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 Processor -- start 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 -- start start 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc801024a0 0x7fbc8019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801a04b0 con 0x7fbc80101b00 2023-08-11T14:01:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801a05f0 con 0x7fbc80100910 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.466+0000 7fbc877ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801a0780 con 0x7fbc801024a0 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46110/0 (socket says 172.21.15.3:46110) 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 -- 172.21.15.3:0/507963039 learned_addr learned my addr 172.21.15.3:0/507963039 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc8559b700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc85d9c700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc801024a0 0x7fbc8019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 -- 172.21.15.3:0/507963039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc801024a0 msgr2=0x7fbc8019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc801024a0 0x7fbc8019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 -- 172.21.15.3:0/507963039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 msgr2=0x7fbc80198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 -- 172.21.15.3:0/507963039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009d00 con 0x7fbc80101b00 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc8559b700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.467+0000 7fbc84d9a700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000e9a0 tx=0x7fbc7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.468+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70009790 con 0x7fbc80101b00 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.468+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc70010430 con 0x7fbc80101b00 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.468+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70018680 con 0x7fbc80101b00 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.468+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc801a0a60 con 0x7fbc80101b00 2023-08-11T14:01:52.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.468+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc801a0f70 con 0x7fbc80101b00 2023-08-11T14:01:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.469+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8010c180 con 0x7fbc80101b00 2023-08-11T14:01:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.471+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbc700105a0 con 0x7fbc80101b00 2023-08-11T14:01:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.472+0000 7fbc767fc700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 0x7fbc6c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.472+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbc70014070 con 0x7fbc80101b00 2023-08-11T14:01:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.472+0000 7fbc8559b700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 0x7fbc6c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.473+0000 7fbc8559b700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 0x7fbc6c0a2490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc80101960 tx=0x7fbc7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.476+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbc70051c30 con 0x7fbc80101b00 2023-08-11T14:01:52.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:52 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/230189093' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:52.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:52 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/486301439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:52.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/230189093' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:52.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/486301439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.687+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc801a1160 con 0x7fbc80101b00 2023-08-11T14:01:52.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.687+0000 7fbc767fc700 1 -- 172.21.15.3:0/507963039 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc70095100 con 0x7fbc80101b00 2023-08-11T14:01:52.689 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:01:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 msgr2=0x7fbc6c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 0x7fbc6c0a2490 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc80101960 tx=0x7fbc7c009260 comp rx=0 tx=0).stop 2023-08-11T14:01:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 msgr2=0x7fbc80198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000e9a0 tx=0x7fbc7000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 shutdown_connections 2023-08-11T14:01:52.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc801024a0 0x7fbc8019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbc6c09ffd0 0x7fbc6c0a2490 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc80100910 0x7fbc80198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 --2- 172.21.15.3:0/507963039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc80101b00 0x7fbc80198e70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.689+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 >> 172.21.15.3:0/507963039 conn(0x7fbc800fbe60 msgr2=0x7fbc80107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:52.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.690+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 shutdown_connections 2023-08-11T14:01:52.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.690+0000 7fbc877ff700 1 -- 172.21.15.3:0/507963039 wait complete. 2023-08-11T14:01:52.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:01:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:52 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/230189093' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:52 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/486301439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 -- 172.21.15.3:0/625732644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 msgr2=0x7f65a8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 --2- 172.21.15.3:0/625732644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a8108d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b0d0 tx=0x7f65a000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 -- 172.21.15.3:0/625732644 shutdown_connections 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 --2- 172.21.15.3:0/625732644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a8108d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 --2- 172.21.15.3:0/625732644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8100910 0x7f65a8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 --2- 172.21.15.3:0/625732644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8101b00 0x7f65a8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 -- 172.21.15.3:0/625732644 >> 172.21.15.3:0/625732644 conn(0x7f65a80fbe60 msgr2=0x7f65a80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 -- 172.21.15.3:0/625732644 shutdown_connections 2023-08-11T14:01:52.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.915+0000 7f65b03ff700 1 -- 172.21.15.3:0/625732644 wait complete. 2023-08-11T14:01:52.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.916+0000 7f65b03ff700 1 Processor -- start 2023-08-11T14:01:52.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.916+0000 7f65b03ff700 1 -- start start 2023-08-11T14:01:52.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 0x7f65a8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 0x7f65a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a0490 con 0x7f65a8101b00 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a05d0 con 0x7f65a81024a0 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65b03ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a0760 con 0x7f65a8100910 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 0x7f65a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ae19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 0x7f65a8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ae19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 0x7f65a8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57264/0 (socket says 172.21.15.3:57264) 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ae19b700 1 -- 172.21.15.3:0/1644782393 learned_addr learned my addr 172.21.15.3:0/1644782393 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ae99c700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 -- 172.21.15.3:0/1644782393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 msgr2=0x7f65a8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 0x7f65a8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 -- 172.21.15.3:0/1644782393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 msgr2=0x7f65a819fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:52.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a819fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ad99a700 1 -- 172.21.15.3:0/1644782393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0009d00 con 0x7f65a8101b00 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.917+0000 7f65ae99c700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f65ad99a700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 0x7f65a8198e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f659800e9f0 tx=0x7f659800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800eee0 con 0x7f65a8101b00 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a81a0a40 con 0x7f65a8101b00 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f659800cbd0 con 0x7f65a8101b00 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598018680 con 0x7f65a8101b00 2023-08-11T14:01:52.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.918+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a81a0f50 con 0x7f65a8101b00 2023-08-11T14:01:52.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.919+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a8192f20 con 0x7f65a8101b00 2023-08-11T14:01:52.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.921+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f659800cd40 con 0x7f65a8101b00 2023-08-11T14:01:52.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.922+0000 7f659f7fe700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 0x7f65940a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:52.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.923+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6598014070 con 0x7f65a8101b00 2023-08-11T14:01:52.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.923+0000 7f65ae19b700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 0x7f65940a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:52.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.923+0000 7f65ae19b700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 0x7f65940a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005950 tx=0x7f65a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:52.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:52.926+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6598051c30 con 0x7f65a8101b00 2023-08-11T14:01:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.138+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65a810c310 con 0x7f65a8101b00 2023-08-11T14:01:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.139+0000 7f659f7fe700 1 -- 172.21.15.3:0/1644782393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6598095110 con 0x7f65a8101b00 2023-08-11T14:01:53.141 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:01:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 msgr2=0x7f65940a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 0x7f65940a2410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005950 tx=0x7f65a4009260 comp rx=0 tx=0).stop 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 msgr2=0x7f65a8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 0x7f65a8198e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f659800e9f0 tx=0x7f659800ed00 comp rx=0 tx=0).stop 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 shutdown_connections 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f659409ff50 0x7f65940a2410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65a81024a0 0x7f65a819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100910 0x7f65a8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 --2- 172.21.15.3:0/1644782393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65a8101b00 0x7f65a8198e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 >> 172.21.15.3:0/1644782393 conn(0x7f65a80fbe60 msgr2=0x7f65a81073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 shutdown_connections 2023-08-11T14:01:53.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.141+0000 7f65b03ff700 1 -- 172.21.15.3:0/1644782393 wait complete. 2023-08-11T14:01:53.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:01:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 -- 172.21.15.3:0/3826049373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0073380 msgr2=0x7ff9d0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/3826049373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0073380 0x7ff9d0073800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 -- 172.21.15.3:0/3826049373 shutdown_connections 2023-08-11T14:01:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/3826049373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/3826049373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0074960 0x7ff9d0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/3826049373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0073380 0x7ff9d0073800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 -- 172.21.15.3:0/3826049373 >> 172.21.15.3:0/3826049373 conn(0x7ff9d00fbe60 msgr2=0x7ff9d00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 -- 172.21.15.3:0/3826049373 shutdown_connections 2023-08-11T14:01:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.370+0000 7ff9d4bff700 1 -- 172.21.15.3:0/3826049373 wait complete. 2023-08-11T14:01:53.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 Processor -- start 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 -- start start 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 0x7ff9d0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a03d0 con 0x7ff9d0074960 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a0510 con 0x7ff9d01087d0 2023-08-11T14:01:53.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.371+0000 7ff9d4bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01a06a0 con 0x7ff9d0073380 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46132/0 (socket says 172.21.15.3:46132) 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9c7fff700 1 -- 172.21.15.3:0/2570604215 learned_addr learned my addr 172.21.15.3:0/2570604215 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48864/0 (socket says 172.21.15.3:48864) 2023-08-11T14:01:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ce59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 0x7ff9d0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 -- 172.21.15.3:0/2570604215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 msgr2=0x7ff9d0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 0x7ff9d0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 -- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 msgr2=0x7ff9d0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ced9d700 1 -- 172.21.15.3:0/2570604215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80095f0 con 0x7ff9d01087d0 2023-08-11T14:01:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9ce59c700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 0x7ff9d0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.372+0000 7ff9c7fff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9ced9d700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800efc0 tx=0x7ff9c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8018070 con 0x7ff9d01087d0 2023-08-11T14:01:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d01a0920 con 0x7ff9d01087d0 2023-08-11T14:01:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c800f040 con 0x7ff9d01087d0 2023-08-11T14:01:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800cd40 con 0x7ff9d01087d0 2023-08-11T14:01:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.373+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d01a0de0 con 0x7ff9d01087d0 2023-08-11T14:01:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.375+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0106e80 con 0x7ff9d01087d0 2023-08-11T14:01:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.377+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff9c8003a00 con 0x7ff9d01087d0 2023-08-11T14:01:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.379+0000 7ff9c77fe700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 0x7ff9bc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.379+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff9c80cb3e0 con 0x7ff9d01087d0 2023-08-11T14:01:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.379+0000 7ff9ce59c700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 0x7ff9bc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.380+0000 7ff9ce59c700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 0x7ff9bc0a2410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011490 tx=0x7ff9c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.381+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff9c8095010 con 0x7ff9d01087d0 2023-08-11T14:01:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:53 smithi003 ceph-mon[97195]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/507963039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1644782393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.594+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9d010e4e0 con 0x7ff9d01087d0 2023-08-11T14:01:53.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.595+0000 7ff9c77fe700 1 -- 172.21.15.3:0/2570604215 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9c8098630 con 0x7ff9d01087d0 2023-08-11T14:01:53.597 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:01:53.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 msgr2=0x7ff9bc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 0x7ff9bc0a2410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011490 tx=0x7ff9c0009260 comp rx=0 tx=0).stop 2023-08-11T14:01:53.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 msgr2=0x7ff9d019fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800efc0 tx=0x7ff9c800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 shutdown_connections 2023-08-11T14:01:53.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff9bc09ff50 0x7ff9bc0a2410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9d01087d0 0x7ff9d019fe00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9d0073380 0x7ff9d0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 --2- 172.21.15.3:0/2570604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9d0074960 0x7ff9d0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 >> 172.21.15.3:0/2570604215 conn(0x7ff9d00fbe60 msgr2=0x7ff9d00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:53.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 shutdown_connections 2023-08-11T14:01:53.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.598+0000 7ff9d4bff700 1 -- 172.21.15.3:0/2570604215 wait complete. 2023-08-11T14:01:53.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:01:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:53 smithi103 ceph-mon[130434]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/507963039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1644782393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:53 smithi187 ceph-mon[130973]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/507963039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1644782393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:01:53.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 -- 172.21.15.3:0/2171424556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c100910 msgr2=0x7fb76c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2171424556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c100910 0x7fb76c100d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7680099c0 tx=0x7fb768009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 -- 172.21.15.3:0/2171424556 shutdown_connections 2023-08-11T14:01:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2171424556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c1024a0 0x7fb76c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2171424556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c100910 0x7fb76c100d30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2171424556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c101b00 0x7fb76c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 -- 172.21.15.3:0/2171424556 >> 172.21.15.3:0/2171424556 conn(0x7fb76c0fbe60 msgr2=0x7fb76c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 -- 172.21.15.3:0/2171424556 shutdown_connections 2023-08-11T14:01:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.823+0000 7fb7739ff700 1 -- 172.21.15.3:0/2171424556 wait complete. 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.824+0000 7fb7739ff700 1 Processor -- start 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.824+0000 7fb7739ff700 1 -- start start 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c100910 0x7fb76c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 0x7fb76c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0739f0 con 0x7fb76c101b00 2023-08-11T14:01:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c073b60 con 0x7fb76c1024a0 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb7739ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c1a06f0 con 0x7fb76c100910 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb770f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb770f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46148/0 (socket says 172.21.15.3:46148) 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb770f9a700 1 -- 172.21.15.3:0/2056339241 learned_addr learned my addr 172.21.15.3:0/2056339241 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb77179b700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c100910 0x7fb76c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb771f9c700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 0x7fb76c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.825+0000 7fb770f9a700 1 -- 172.21.15.3:0/2056339241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c100910 msgr2=0x7fb76c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb770f9a700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c100910 0x7fb76c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb770f9a700 1 -- 172.21.15.3:0/2056339241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 msgr2=0x7fb76c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb770f9a700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 0x7fb76c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb770f9a700 1 -- 172.21.15.3:0/2056339241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7680095f0 con 0x7fb76c101b00 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb771f9c700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 0x7fb76c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:53.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb770f9a700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00e9a0 tx=0x7fb75c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:53.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c009790 con 0x7fb76c101b00 2023-08-11T14:01:53.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c1a09d0 con 0x7fb76c101b00 2023-08-11T14:01:53.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c00cbd0 con 0x7fb76c101b00 2023-08-11T14:01:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c018680 con 0x7fb76c101b00 2023-08-11T14:01:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.826+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c1a0f30 con 0x7fb76c101b00 2023-08-11T14:01:53.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.827+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76c10c170 con 0x7fb76c101b00 2023-08-11T14:01:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.829+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb75c00cd40 con 0x7fb76c101b00 2023-08-11T14:01:53.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.830+0000 7fb7627fc700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 0x7fb7580a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:53.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.830+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb75c014070 con 0x7fb76c101b00 2023-08-11T14:01:53.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.831+0000 7fb77179b700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 0x7fb7580a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:53.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.831+0000 7fb77179b700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 0x7fb7580a2410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb768009990 tx=0x7fb768005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:53.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:53.834+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb75c051c30 con 0x7fb76c101b00 2023-08-11T14:01:54.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.046+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb76c1078c0 con 0x7fb76c101b00 2023-08-11T14:01:54.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.047+0000 7fb7627fc700 1 -- 172.21.15.3:0/2056339241 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb75c095270 con 0x7fb76c101b00 2023-08-11T14:01:54.049 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:01:54.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 msgr2=0x7fb7580a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 0x7fb7580a2410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb768009990 tx=0x7fb768005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 msgr2=0x7fb76c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00e9a0 tx=0x7fb75c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 shutdown_connections 2023-08-11T14:01:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb75809ff50 0x7fb7580a2410 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb76c1024a0 0x7fb76c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb76c100910 0x7fb76c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 --2- 172.21.15.3:0/2056339241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb76c101b00 0x7fb76c072d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 >> 172.21.15.3:0/2056339241 conn(0x7fb76c0fbe60 msgr2=0x7fb76c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:54.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.049+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 shutdown_connections 2023-08-11T14:01:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.050+0000 7fb7739ff700 1 -- 172.21.15.3:0/2056339241 wait complete. 2023-08-11T14:01:54.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T14:01:54.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 -- 172.21.15.3:0/1795889598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 msgr2=0x7f9988100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 --2- 172.21.15.3:0/1795889598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988100d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f99840099c0 tx=0x7f9984009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 -- 172.21.15.3:0/1795889598 shutdown_connections 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 --2- 172.21.15.3:0/1795889598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 0x7f9988108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 --2- 172.21.15.3:0/1795889598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988100d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 --2- 172.21.15.3:0/1795889598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 0x7f9988101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 -- 172.21.15.3:0/1795889598 >> 172.21.15.3:0/1795889598 conn(0x7f99880fbea0 msgr2=0x7f99880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 -- 172.21.15.3:0/1795889598 shutdown_connections 2023-08-11T14:01:54.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.274+0000 7f998f7ff700 1 -- 172.21.15.3:0/1795889598 wait complete. 2023-08-11T14:01:54.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.275+0000 7f998f7ff700 1 Processor -- start 2023-08-11T14:01:54.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 -- start start 2023-08-11T14:01:54.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 0x7f9988198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 0x7f998819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881a03f0 con 0x7f9988100930 2023-08-11T14:01:54.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881a0530 con 0x7f99881024c0 2023-08-11T14:01:54.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998f7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881a06c0 con 0x7f9988101b20 2023-08-11T14:01:54.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998d59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998d59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46158/0 (socket says 172.21.15.3:46158) 2023-08-11T14:01:54.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998d59b700 1 -- 172.21.15.3:0/2124760848 learned_addr learned my addr 172.21.15.3:0/2124760848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:54.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998cd9a700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 0x7f9988198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.276+0000 7f998dd9c700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 0x7f998819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 -- 172.21.15.3:0/2124760848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 msgr2=0x7f9988198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 0x7f9988198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 -- 172.21.15.3:0/2124760848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 msgr2=0x7f998819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 0x7f998819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 -- 172.21.15.3:0/2124760848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009520 con 0x7f9988100930 2023-08-11T14:01:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998d59b700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9984009fd0 tx=0x7f998400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998401d070 con 0x7f9988100930 2023-08-11T14:01:54.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9984005050 con 0x7f9988100930 2023-08-11T14:01:54.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99840095f0 con 0x7f9988100930 2023-08-11T14:01:54.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984004a70 con 0x7f9988100930 2023-08-11T14:01:54.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.277+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99881a0bd0 con 0x7f9988100930 2023-08-11T14:01:54.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.278+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9988192f80 con 0x7f9988100930 2023-08-11T14:01:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.280+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9984005450 con 0x7f9988100930 2023-08-11T14:01:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.282+0000 7f997a7fc700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 0x7f99740a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.282+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f99840cc260 con 0x7f9988100930 2023-08-11T14:01:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.282+0000 7f998cd9a700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 0x7f99740a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.283+0000 7f998cd9a700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 0x7f99740a2410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f997c009df0 tx=0x7f997c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:54.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.286+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9984095e30 con 0x7f9988100930 2023-08-11T14:01:54.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.517+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f998810c190 con 0x7f9988100930 2023-08-11T14:01:54.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.517+0000 7f997a7fc700 1 -- 172.21.15.3:0/2124760848 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+580 (secure 0 0 0) 0x7f9984099450 con 0x7f9988100930 2023-08-11T14:01:54.523 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2366653083 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:54.524 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:54.524 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:54.524 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1070970995 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1776709742 2023-08-12T13:55:02.078980+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3848927136 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3010670797 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:54.525 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1680525538 2023-08-12T13:54:34.603107+0000 2023-08-11T14:01:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 msgr2=0x7f99740a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 0x7f99740a2410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f997c009df0 tx=0x7f997c009260 comp rx=0 tx=0).stop 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 msgr2=0x7f9988198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9984009fd0 tx=0x7f998400f690 comp rx=0 tx=0).stop 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 shutdown_connections 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f997409ff50 0x7f99740a2410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99881024c0 0x7f998819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988100930 0x7f9988198870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 --2- 172.21.15.3:0/2124760848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9988101b20 0x7f9988198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 >> 172.21.15.3:0/2124760848 conn(0x7f99880fbea0 msgr2=0x7f9988107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:54.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 shutdown_connections 2023-08-11T14:01:54.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.520+0000 7f998f7ff700 1 -- 172.21.15.3:0/2124760848 wait complete. 2023-08-11T14:01:54.528 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2023-08-11T14:01:54.540 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-08-11T14:01:54.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:54.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-08-11T14:01:54.606 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2570604215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.606 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2056339241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.612 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[134455]: fuse finished with error 0 and tester_r 0 2023-08-11T14:01:54.616 INFO:teuthology.orchestra.run:waiting for 300 2023-08-11T14:01:54.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:01:54.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:01:54.675 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-08-11T14:01:54.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2570604215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2056339241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:54.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2570604215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2056339241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:01:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 -- 172.21.15.3:0/2569371108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794010cef0 msgr2=0x7f794010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 --2- 172.21.15.3:0/2569371108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794010cef0 0x7f794010f2e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f793800b0a0 tx=0x7f793800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 -- 172.21.15.3:0/2569371108 shutdown_connections 2023-08-11T14:01:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 --2- 172.21.15.3:0/2569371108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f794010cef0 0x7f794010f2e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 --2- 172.21.15.3:0/2569371108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7940074960 0x7f7940072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 --2- 172.21.15.3:0/2569371108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 0x7f7940073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 -- 172.21.15.3:0/2569371108 >> 172.21.15.3:0/2569371108 conn(0x7f79400fbe50 msgr2=0x7f79400fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 -- 172.21.15.3:0/2569371108 shutdown_connections 2023-08-11T14:01:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.940+0000 7f7945bff700 1 -- 172.21.15.3:0/2569371108 wait complete. 2023-08-11T14:01:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 Processor -- start 2023-08-11T14:01:54.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 -- start start 2023-08-11T14:01:54.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 0x7f794019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f794010cef0 0x7f79401a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401a4840 con 0x7f7940074960 2023-08-11T14:01:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401a49b0 con 0x7f7940073380 2023-08-11T14:01:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.941+0000 7f7945bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401a4b70 con 0x7f794010cef0 2023-08-11T14:01:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46164/0 (socket says 172.21.15.3:46164) 2023-08-11T14:01:54.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 -- 172.21.15.3:0/832662893 learned_addr learned my addr 172.21.15.3:0/832662893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:54.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793f7fe700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 0x7f794019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793ffff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f794010cef0 0x7f79401a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 -- 172.21.15.3:0/832662893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f794010cef0 msgr2=0x7f79401a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f794010cef0 0x7f79401a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 -- 172.21.15.3:0/832662893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 msgr2=0x7f794019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:54.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 0x7f794019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 -- 172.21.15.3:0/832662893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009d00 con 0x7f7940074960 2023-08-11T14:01:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.942+0000 7f793effd700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f793400e970 tx=0x7f793400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.943+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934009790 con 0x7f7940074960 2023-08-11T14:01:54.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.943+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793400c7e0 con 0x7f7940074960 2023-08-11T14:01:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.943+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934003c70 con 0x7f7940074960 2023-08-11T14:01:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.943+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79401a4e50 con 0x7f7940074960 2023-08-11T14:01:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.943+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401a5220 con 0x7f7940074960 2023-08-11T14:01:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.944+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940102a40 con 0x7f7940074960 2023-08-11T14:01:54.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.946+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f793400cc50 con 0x7f7940074960 2023-08-11T14:01:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.947+0000 7f793cff9700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 0x7f79280a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:54.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.947+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7934014070 con 0x7f7940074960 2023-08-11T14:01:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.947+0000 7f793f7fe700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 0x7f79280a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.948+0000 7f793f7fe700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 0x7f79280a2460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f794019e1d0 tx=0x7f7930009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:54.951+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7934095190 con 0x7f7940074960 2023-08-11T14:01:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.163+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7940112760 con 0x7f7940074960 2023-08-11T14:01:55.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.165+0000 7f793cff9700 1 -- 172.21.15.3:0/832662893 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7f7934003dd0 con 0x7f7940074960 2023-08-11T14:01:55.167 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:55.168 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 msgr2=0x7f79280a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 0x7f79280a2460 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f794019e1d0 tx=0x7f7930009260 comp rx=0 tx=0).stop 2023-08-11T14:01:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 msgr2=0x7f794019d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f793400e970 tx=0x7f793400ec80 comp rx=0 tx=0).stop 2023-08-11T14:01:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 shutdown_connections 2023-08-11T14:01:55.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f794010cef0 0x7f79401a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f792809ffa0 0x7f79280a2460 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7940073380 0x7f794019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 --2- 172.21.15.3:0/832662893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7940074960 0x7f794019d240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 >> 172.21.15.3:0/832662893 conn(0x7f79400fbe50 msgr2=0x7f79400fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:55.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 shutdown_connections 2023-08-11T14:01:55.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.168+0000 7f7945bff700 1 -- 172.21.15.3:0/832662893 wait complete. 2023-08-11T14:01:55.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:01:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 -- 172.21.15.3:0/615626769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb41024b0 msgr2=0x7f7eb4108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/615626769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb41024b0 0x7f7eb4108d60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b0d0 tx=0x7f7eac00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 -- 172.21.15.3:0/615626769 shutdown_connections 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/615626769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb41024b0 0x7f7eb4108d60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/615626769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb4100920 0x7f7eb4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/615626769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 0x7f7eb4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.395+0000 7f7eb9fff700 1 -- 172.21.15.3:0/615626769 >> 172.21.15.3:0/615626769 conn(0x7f7eb40fbe50 msgr2=0x7f7eb40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.396+0000 7f7eb9fff700 1 -- 172.21.15.3:0/615626769 shutdown_connections 2023-08-11T14:01:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.396+0000 7f7eb9fff700 1 -- 172.21.15.3:0/615626769 wait complete. 2023-08-11T14:01:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.396+0000 7f7eb9fff700 1 Processor -- start 2023-08-11T14:01:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 -- start start 2023-08-11T14:01:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 0x7f7eb4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb41024b0 0x7f7eb419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41a0480 con 0x7f7eb4100920 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41a05c0 con 0x7f7eb4101b10 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb9fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb41a0750 con 0x7f7eb41024b0 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb2ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 0x7f7eb4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46180/0 (socket says 172.21.15.3:46180) 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb37fe700 1 -- 172.21.15.3:0/3930912455 learned_addr learned my addr 172.21.15.3:0/3930912455 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.397+0000 7f7eb3fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb41024b0 0x7f7eb419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 -- 172.21.15.3:0/3930912455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb41024b0 msgr2=0x7f7eb419feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb41024b0 0x7f7eb419feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 -- 172.21.15.3:0/3930912455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 msgr2=0x7f7eb4198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 0x7f7eb4198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 -- 172.21.15.3:0/3930912455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009d00 con 0x7f7eb4100920 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb37fe700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400e970 tx=0x7f7ea400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4009790 con 0x7f7eb4100920 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea400c7e0 con 0x7f7eb4100920 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4003c70 con 0x7f7eb4100920 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb41a0a30 con 0x7f7eb4100920 2023-08-11T14:01:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.398+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb41a0ef0 con 0x7f7eb4100920 2023-08-11T14:01:55.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.399+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb410c190 con 0x7f7eb4100920 2023-08-11T14:01:55.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.401+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7ea400cc50 con 0x7f7eb4100920 2023-08-11T14:01:55.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.403+0000 7f7eb0ff9700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 0x7f7e9c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.403+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7ea4014070 con 0x7f7eb4100920 2023-08-11T14:01:55.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.403+0000 7f7eb2ffd700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 0x7f7e9c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.404+0000 7f7eb2ffd700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 0x7f7e9c0a2460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009c60 tx=0x7f7ea8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:55.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.407+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7ea4095190 con 0x7f7eb4100920 2023-08-11T14:01:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:55 smithi003 ceph-mon[97195]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:55 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2124760848' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:55.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:55 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/832662893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:55.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.620+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eb410c400 con 0x7f7eb4100920 2023-08-11T14:01:55.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.622+0000 7f7eb0ff9700 1 -- 172.21.15.3:0/3930912455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23914 (secure 0 0 0) 0x7f7ea4003dd0 con 0x7f7eb4100920 2023-08-11T14:01:55.624 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:01:55.624 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:01:37.883781+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 msgr2=0x7f7e9c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 0x7f7e9c0a2460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009c60 tx=0x7f7ea8009260 comp rx=0 tx=0).stop 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 msgr2=0x7f7eb4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400e970 tx=0x7f7ea400ec80 comp rx=0 tx=0).stop 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 shutdown_connections 2023-08-11T14:01:55.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb41024b0 0x7f7eb419feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eb4100920 0x7f7eb4198900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e9c09ffa0 0x7f7e9c0a2460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 --2- 172.21.15.3:0/3930912455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb4101b10 0x7f7eb4198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 >> 172.21.15.3:0/3930912455 conn(0x7f7eb40fbe50 msgr2=0x7f7eb40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 shutdown_connections 2023-08-11T14:01:55.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.624+0000 7f7eb9fff700 1 -- 172.21.15.3:0/3930912455 wait complete. 2023-08-11T14:01:55.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-11T14:01:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:55 smithi103 ceph-mon[130434]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:55.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:55 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2124760848' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:55.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:55 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/832662893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:55.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:55 smithi187 ceph-mon[130973]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:55.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:55 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2124760848' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:01:55.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:55 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/832662893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:55.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.850+0000 7fb088dff700 1 -- 172.21.15.3:0/3806416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 msgr2=0x7fb084100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.850+0000 7fb088dff700 1 --2- 172.21.15.3:0/3806416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb084100d30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb074009920 tx=0x7fb074009c30 comp rx=0 tx=0).stop 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 -- 172.21.15.3:0/3806416 shutdown_connections 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 --2- 172.21.15.3:0/3806416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 --2- 172.21.15.3:0/3806416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb084100d30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 --2- 172.21.15.3:0/3806416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 -- 172.21.15.3:0/3806416 >> 172.21.15.3:0/3806416 conn(0x7fb0840fbe60 msgr2=0x7fb0840fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 -- 172.21.15.3:0/3806416 shutdown_connections 2023-08-11T14:01:55.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.851+0000 7fb088dff700 1 -- 172.21.15.3:0/3806416 wait complete. 2023-08-11T14:01:55.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 Processor -- start 2023-08-11T14:01:55.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 -- start start 2023-08-11T14:01:55.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb0840746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0840739f0 con 0x7fb0841024a0 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb084073b60 con 0x7fb084100910 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb088dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841a07e0 con 0x7fb084101b00 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46194/0 (socket says 172.21.15.3:46194) 2023-08-11T14:01:55.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.852+0000 7fb082d9d700 1 -- 172.21.15.3:0/3423628645 learned_addr learned my addr 172.21.15.3:0/3423628645 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb08259c700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb0840746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb07bfff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 -- 172.21.15.3:0/3423628645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 msgr2=0x7fb084072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 -- 172.21.15.3:0/3423628645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 msgr2=0x7fb0840746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb0840746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 -- 172.21.15.3:0/3423628645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009520 con 0x7fb0841024a0 2023-08-11T14:01:55.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb07bfff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb082d9d700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c110 tx=0x7fb07c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c017070 con 0x7fb0841024a0 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0841a0a60 con 0x7fb0841024a0 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb07c00e050 con 0x7fb0841024a0 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.853+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c007940 con 0x7fb0841024a0 2023-08-11T14:01:55.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.854+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841a0f20 con 0x7fb0841024a0 2023-08-11T14:01:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.855+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb084108090 con 0x7fb0841024a0 2023-08-11T14:01:55.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.857+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb07c004ad0 con 0x7fb0841024a0 2023-08-11T14:01:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.858+0000 7fb07b7fe700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 0x7fb0700a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.859+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb07c0cb950 con 0x7fb0841024a0 2023-08-11T14:01:55.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.859+0000 7fb08259c700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 0x7fb0700a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:55.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.859+0000 7fb08259c700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 0x7fb0700a2410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098f0 tx=0x7fb074019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:55.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:55.861+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb07c095490 con 0x7fb0841024a0 2023-08-11T14:01:56.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.118+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb08402cdc0 con 0x7fb0841024a0 2023-08-11T14:01:56.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.329+0000 7fb07b7fe700 1 -- 172.21.15.3:0/3423628645 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb07c098ab0 con 0x7fb0841024a0 2023-08-11T14:01:56.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 msgr2=0x7fb0700a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 0x7fb0700a2410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098f0 tx=0x7fb074019040 comp rx=0 tx=0).stop 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 msgr2=0x7fb084073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c110 tx=0x7fb07c00c420 comp rx=0 tx=0).stop 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 shutdown_connections 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0841024a0 0x7fb084073250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb07009ff50 0x7fb0700a2410 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb084100910 0x7fb0840746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 --2- 172.21.15.3:0/3423628645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb084101b00 0x7fb084072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 >> 172.21.15.3:0/3423628645 conn(0x7fb0840fbe60 msgr2=0x7fb0841051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 shutdown_connections 2023-08-11T14:01:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.332+0000 7fb088dff700 1 -- 172.21.15.3:0/3423628645 wait complete. 2023-08-11T14:01:56.352 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-08-11T14:01:56.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-08-11T14:01:56.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 -- 172.21.15.3:0/3354239641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8180100910 msgr2=0x7f8180100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:56.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 --2- 172.21.15.3:0/3354239641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8180100910 0x7f8180100d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f81700099c0 tx=0x7f8170009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 -- 172.21.15.3:0/3354239641 shutdown_connections 2023-08-11T14:01:56.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 --2- 172.21.15.3:0/3354239641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81801024a0 0x7f8180108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 --2- 172.21.15.3:0/3354239641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8180100910 0x7f8180100d30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 --2- 172.21.15.3:0/3354239641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 -- 172.21.15.3:0/3354239641 >> 172.21.15.3:0/3354239641 conn(0x7f81800fbe60 msgr2=0x7f81800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:56.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.560+0000 7f81853ff700 1 -- 172.21.15.3:0/3354239641 shutdown_connections 2023-08-11T14:01:56.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.561+0000 7f81853ff700 1 -- 172.21.15.3:0/3354239641 wait complete. 2023-08-11T14:01:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.561+0000 7f81853ff700 1 Processor -- start 2023-08-11T14:01:56.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 -- start start 2023-08-11T14:01:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8180100910 0x7f8180198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:56.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81801a0490 con 0x7f81801024a0 2023-08-11T14:01:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81801a05d0 con 0x7f8180100910 2023-08-11T14:01:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f81853ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81801a0760 con 0x7f8180101b00 2023-08-11T14:01:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f817e7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:56.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46218/0 (socket says 172.21.15.3:46218) 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f817f7fe700 1 -- 172.21.15.3:0/168970615 learned_addr learned my addr 172.21.15.3:0/168970615 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.562+0000 7f817effd700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8180100910 0x7f8180198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 -- 172.21.15.3:0/168970615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 msgr2=0x7f8180198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 -- 172.21.15.3:0/168970615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8180100910 msgr2=0x7f8180198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8180100910 0x7f8180198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:56.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 -- 172.21.15.3:0/168970615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700095f0 con 0x7f81801024a0 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817e7fc700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f817f7fe700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8178007ee0 tx=0x7f817800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800dd20 con 0x7f81801024a0 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f817800f040 con 0x7f81801024a0 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81801a0a40 con 0x7f81801024a0 2023-08-11T14:01:56.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780048a0 con 0x7f81801024a0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.563+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81801a0f50 con 0x7f81801024a0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.564+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818010c170 con 0x7f81801024a0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.566+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f8178003a00 con 0x7f81801024a0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.567+0000 7f8167fff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 0x7f81680a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.568+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81780cabf0 con 0x7f81801024a0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.568+0000 7f817effd700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 0x7f81680a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:56.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.568+0000 7f817effd700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 0x7f81680a2490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8170009990 tx=0x7f8170005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:56.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.571+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8178094730 con 0x7f81801024a0 2023-08-11T14:01:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:56 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3930912455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:56 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T14:01:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:56 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3930912455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:56 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T14:01:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:56 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3930912455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:01:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:56 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-08-11T14:01:56.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:56.830+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f81801a1140 con 0x7f81801024a0 2023-08-11T14:01:57.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.283+0000 7f8167fff700 1 -- 172.21.15.3:0/168970615 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8178097d50 con 0x7f81801024a0 2023-08-11T14:01:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 msgr2=0x7f81680a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 0x7f81680a2490 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8170009990 tx=0x7f8170005fb0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 msgr2=0x7f818019fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8178007ee0 tx=0x7f817800d3b0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 shutdown_connections 2023-08-11T14:01:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81801024a0 0x7f818019fec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f816809ffd0 0x7f81680a2490 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8180100910 0x7f8180198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 --2- 172.21.15.3:0/168970615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8180101b00 0x7f8180198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 >> 172.21.15.3:0/168970615 conn(0x7f81800fbe60 msgr2=0x7f81800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:57.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.286+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 shutdown_connections 2023-08-11T14:01:57.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.287+0000 7f81853ff700 1 -- 172.21.15.3:0/168970615 wait complete. 2023-08-11T14:01:57.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-08-11T14:01:57.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.525+0000 7f7d379ff700 1 -- 172.21.15.3:0/3635718266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 msgr2=0x7f7d30100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.525+0000 7f7d379ff700 1 --2- 172.21.15.3:0/3635718266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d30100d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0099c0 tx=0x7f7d2c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 -- 172.21.15.3:0/3635718266 shutdown_connections 2023-08-11T14:01:57.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 --2- 172.21.15.3:0/3635718266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d301024a0 0x7f7d30108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 --2- 172.21.15.3:0/3635718266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d30100d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 --2- 172.21.15.3:0/3635718266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d30101b00 0x7f7d30101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 -- 172.21.15.3:0/3635718266 >> 172.21.15.3:0/3635718266 conn(0x7f7d300fbe60 msgr2=0x7f7d300fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 -- 172.21.15.3:0/3635718266 shutdown_connections 2023-08-11T14:01:57.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.526+0000 7f7d379ff700 1 -- 172.21.15.3:0/3635718266 wait complete. 2023-08-11T14:01:57.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 Processor -- start 2023-08-11T14:01:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 -- start start 2023-08-11T14:01:57.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30101b00 0x7f7d30198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d3019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:57.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a03f0 con 0x7f7d30101b00 2023-08-11T14:01:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a0530 con 0x7f7d30100910 2023-08-11T14:01:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d379ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a06c0 con 0x7f7d301024a0 2023-08-11T14:01:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d3579b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:57.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d3579b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49016/0 (socket says 172.21.15.3:49016) 2023-08-11T14:01:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d3579b700 1 -- 172.21.15.3:0/1651149132 learned_addr learned my addr 172.21.15.3:0/1651149132 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:57.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.527+0000 7f7d34f9a700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30101b00 0x7f7d30198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d35f9c700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d3019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:57.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 -- 172.21.15.3:0/1651149132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 msgr2=0x7f7d3019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d3019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 -- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30101b00 msgr2=0x7f7d30198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30101b00 0x7f7d30198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:57.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 -- 172.21.15.3:0/1651149132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20009520 con 0x7f7d30100910 2023-08-11T14:01:57.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d3579b700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fd0 tx=0x7f7d2c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:57.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c01d070 con 0x7f7d30100910 2023-08-11T14:01:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c0095f0 con 0x7f7d30100910 2023-08-11T14:01:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2c005510 con 0x7f7d30100910 2023-08-11T14:01:57.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c004b00 con 0x7f7d30100910 2023-08-11T14:01:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.528+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301a0c30 con 0x7f7d30100910 2023-08-11T14:01:57.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.529+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3010c170 con 0x7f7d30100910 2023-08-11T14:01:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.531+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7d2c00fb80 con 0x7f7d30100910 2023-08-11T14:01:57.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.533+0000 7f7d267fc700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.533+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7d2c0cc690 con 0x7f7d30100910 2023-08-11T14:01:57.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.533+0000 7f7d34f9a700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.534+0000 7f7d34f9a700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009df0 tx=0x7f7d20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:57.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.535+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7d2c096260 con 0x7f7d30100910 2023-08-11T14:01:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-08-11T14:01:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-11T14:01:57.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:57 smithi103 ceph-mon[130434]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-11T14:01:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T14:01:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-08-11T14:01:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-11T14:01:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-11T14:01:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-08-11T14:01:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-11T14:01:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:57 smithi003 ceph-mon[97195]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-11T14:01:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3423628645' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/168970615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-08-11T14:01:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-08-11T14:01:57.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:57 smithi187 ceph-mon[130973]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-08-11T14:01:57.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:57.794+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f7d3010c330 con 0x7f7d30100910 2023-08-11T14:01:58.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.292+0000 7f7d267fc700 1 -- 172.21.15.3:0/1651149132 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7d2c099880 con 0x7f7d30100910 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 msgr2=0x7f7d1c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009df0 tx=0x7f7d20009260 comp rx=0 tx=0).stop 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 msgr2=0x7f7d30198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fd0 tx=0x7f7d2c00f690 comp rx=0 tx=0).stop 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 shutdown_connections 2023-08-11T14:01:58.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d3019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d1c09ff50 0x7f7d1c0a2410 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30100910 0x7f7d30198870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 --2- 172.21.15.3:0/1651149132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30101b00 0x7f7d30198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 >> 172.21.15.3:0/1651149132 conn(0x7f7d300fbe60 msgr2=0x7f7d300fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 shutdown_connections 2023-08-11T14:01:58.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.295+0000 7f7d379ff700 1 -- 172.21.15.3:0/1651149132 wait complete. 2023-08-11T14:01:58.322 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-08-11T14:01:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 -- 172.21.15.3:0/2808647933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd174101b00 msgr2=0x7fd174101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/2808647933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd174101b00 0x7fd174101f60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd1640099c0 tx=0x7fd164009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 -- 172.21.15.3:0/2808647933 shutdown_connections 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/2808647933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1741024a0 0x7fd174108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/2808647933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174100910 0x7fd174100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/2808647933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd174101b00 0x7fd174101f60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 -- 172.21.15.3:0/2808647933 >> 172.21.15.3:0/2808647933 conn(0x7fd1740fbe60 msgr2=0x7fd1740fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 -- 172.21.15.3:0/2808647933 shutdown_connections 2023-08-11T14:01:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.531+0000 7fd17c1ff700 1 -- 172.21.15.3:0/2808647933 wait complete. 2023-08-11T14:01:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.532+0000 7fd17c1ff700 1 Processor -- start 2023-08-11T14:01:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.532+0000 7fd17c1ff700 1 -- start start 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd174100910 0x7fd1740746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174101b00 0x7fd174072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740739f0 con 0x7fd1741024a0 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd174073b60 con 0x7fd174101b00 2023-08-11T14:01:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17c1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1741a06c0 con 0x7fd174100910 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46268/0 (socket says 172.21.15.3:46268) 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 -- 172.21.15.3:0/1698285648 learned_addr learned my addr 172.21.15.3:0/1698285648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd179f9b700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd174100910 0x7fd1740746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17979a700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174101b00 0x7fd174072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 -- 172.21.15.3:0/1698285648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd174100910 msgr2=0x7fd1740746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd174100910 0x7fd1740746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 -- 172.21.15.3:0/1698285648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174101b00 msgr2=0x7fd174072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174101b00 0x7fd174072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.533+0000 7fd17a79c700 1 -- 172.21.15.3:0/1698285648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1640095f0 con 0x7fd1741024a0 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd17a79c700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00ef30 tx=0x7fd16c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c018070 con 0x7fd1741024a0 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1741a0940 con 0x7fd1741024a0 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd16c00f040 con 0x7fd1741024a0 2023-08-11T14:01:58.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00cd00 con 0x7fd1741024a0 2023-08-11T14:01:58.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.534+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741a0e00 con 0x7fd1741024a0 2023-08-11T14:01:58.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.535+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd174107890 con 0x7fd1741024a0 2023-08-11T14:01:58.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.537+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd16c003a00 con 0x7fd1741024a0 2023-08-11T14:01:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.538+0000 7fd16affd700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 0x7fd1600a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.538+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd16c0cafe0 con 0x7fd1741024a0 2023-08-11T14:01:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.539+0000 7fd179f9b700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 0x7fd1600a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.539+0000 7fd179f9b700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 0x7fd1600a2410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd174101960 tx=0x7fd170009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:58.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.543+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd16c094bb0 con 0x7fd1741024a0 2023-08-11T14:01:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1651149132' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:01:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T14:01:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-11T14:01:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-08-11T14:01:58.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-08-11T14:01:58.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:58 smithi103 ceph-mon[130434]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-08-11T14:01:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1651149132' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-08-11T14:01:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:58 smithi003 ceph-mon[97195]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-08-11T14:01:58.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1651149132' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-08-11T14:01:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-08-11T14:01:58.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:58 smithi187 ceph-mon[130973]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-08-11T14:01:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:58.800+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd17402cc20 con 0x7fd1741024a0 2023-08-11T14:01:59.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.298+0000 7fd16affd700 1 -- 172.21.15.3:0/1698285648 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd16c0981d0 con 0x7fd1741024a0 2023-08-11T14:01:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 msgr2=0x7fd1600a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 0x7fd1600a2410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd174101960 tx=0x7fd170009260 comp rx=0 tx=0).stop 2023-08-11T14:01:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 msgr2=0x7fd174073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00ef30 tx=0x7fd16c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 shutdown_connections 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1741024a0 0x7fd174073250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd174100910 0x7fd1740746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16009ff50 0x7fd1600a2410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 --2- 172.21.15.3:0/1698285648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd174101b00 0x7fd174072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.301+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 >> 172.21.15.3:0/1698285648 conn(0x7fd1740fbe60 msgr2=0x7fd1740fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.302+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 shutdown_connections 2023-08-11T14:01:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.302+0000 7fd17c1ff700 1 -- 172.21.15.3:0/1698285648 wait complete. 2023-08-11T14:01:59.327 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-08-11T14:01:59.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/4066603291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 msgr2=0x7fcd34100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/4066603291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd34100d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd300099c0 tx=0x7fcd30009cd0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/4066603291 shutdown_connections 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/4066603291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 0x7fcd34108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/4066603291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd34100d30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/4066603291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/4066603291 >> 172.21.15.3:0/4066603291 conn(0x7fcd340fbe60 msgr2=0x7fcd340fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:01:59.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/4066603291 shutdown_connections 2023-08-11T14:01:59.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.535+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/4066603291 wait complete. 2023-08-11T14:01:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.536+0000 7fcd3b9ff700 1 Processor -- start 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.536+0000 7fcd3b9ff700 1 -- start start 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 0x7fcd34073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340739f0 con 0x7fcd34100910 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34073b60 con 0x7fcd34101b00 2023-08-11T14:01:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3b9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341a0690 con 0x7fcd341024a0 2023-08-11T14:01:59.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:59.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd38f9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:59.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46280/0 (socket says 172.21.15.3:46280) 2023-08-11T14:01:59.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 -- 172.21.15.3:0/3438889344 learned_addr learned my addr 172.21.15.3:0/3438889344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd39f9c700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 0x7fcd34073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 -- 172.21.15.3:0/3438889344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 msgr2=0x7fcd34073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 0x7fcd34073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 -- 172.21.15.3:0/3438889344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 msgr2=0x7fcd34072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd3979b700 1 -- 172.21.15.3:0/3438889344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300095f0 con 0x7fcd34100910 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.537+0000 7fcd38f9a700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:01:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd3979b700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009990 tx=0x7fcd3000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3001d070 con 0x7fcd34100910 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd30005030 con 0x7fcd34100910 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004a70 con 0x7fcd34100910 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd341a0910 con 0x7fcd34100910 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.538+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd341a0ce0 con 0x7fcd34100910 2023-08-11T14:01:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.539+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd34107890 con 0x7fcd34100910 2023-08-11T14:01:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.541+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcd30005410 con 0x7fcd34100910 2023-08-11T14:01:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.542+0000 7fcd2a7fc700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 0x7fcd200a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:01:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.543+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcd300cc1b0 con 0x7fcd34100910 2023-08-11T14:01:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.543+0000 7fcd38f9a700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 0x7fcd200a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:01:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.543+0000 7fcd38f9a700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 0x7fcd200a2410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011490 tx=0x7fcd24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:01:59.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.546+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fcd30095d80 con 0x7fcd34100910 2023-08-11T14:01:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:active 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-08-11T14:01:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:01:59 smithi103 ceph-mon[130434]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-08-11T14:01:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:active 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-08-11T14:01:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-mon[97195]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-08-11T14:01:59.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:01:59.790+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd3402cd60 con 0x7fcd2009ff50 2023-08-11T14:01:59.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] up:active 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1698285648' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2023-08-11T14:01:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:01:59 smithi187 ceph-mon[130973]: daemon mds.c is now active in filesystem cephfs as rank 3 2023-08-11T14:02:00.291 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:01:59.797+0000 7ff17e5f5700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T14:02:00.292 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:01:59.797+0000 7ff17e5f5700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T14:02:00.292 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:01:59.797+0000 7ff17e5f5700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T14:02:00.292 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:01:59.797+0000 7ff17e5f5700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T14:02:00.292 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:01:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:01:59.797+0000 7ff17e5f5700 -1 client.0 error registering admin socket command: (17) File exists 2023-08-11T14:02:00.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.341+0000 7fcd2a7fc700 1 -- 172.21.15.3:0/3438889344 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcd3402cd60 con 0x7fcd2009ff50 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 msgr2=0x7fcd200a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 0x7fcd200a2410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011490 tx=0x7fcd24009260 comp rx=0 tx=0).stop 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 msgr2=0x7fcd340746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009990 tx=0x7fcd3000f690 comp rx=0 tx=0).stop 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 shutdown_connections 2023-08-11T14:02:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd341024a0 0x7fcd34073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd34100910 0x7fcd340746d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd2009ff50 0x7fcd200a2410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 --2- 172.21.15.3:0/3438889344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd34101b00 0x7fcd34072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 >> 172.21.15.3:0/3438889344 conn(0x7fcd340fbe60 msgr2=0x7fcd340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 shutdown_connections 2023-08-11T14:02:00.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.344+0000 7fcd3b9ff700 1 -- 172.21.15.3:0/3438889344 wait complete. 2023-08-11T14:02:00.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-08-11T14:02:00.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.572+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1039336081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68103d30 msgr2=0x7f5c68104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.572+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1039336081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68103d30 0x7f5c68104190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0099c0 tx=0x7f5c5c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1039336081 shutdown_connections 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1039336081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c681046d0 0x7f5c6810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1039336081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68102b40 0x7f5c68102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1039336081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c68103d30 0x7f5c68104190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1039336081 >> 172.21.15.3:0/1039336081 conn(0x7f5c680fe0d0 msgr2=0x7f5c681004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1039336081 shutdown_connections 2023-08-11T14:02:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.573+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1039336081 wait complete. 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 Processor -- start 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 -- start start 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c68102b40 0x7f5c681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68103d30 0x7f5c68198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681a0420 con 0x7f5c681046d0 2023-08-11T14:02:00.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681a0560 con 0x7f5c68102b40 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c6ddff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c681a06f0 con 0x7f5c68103d30 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c677fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c68102b40 0x7f5c681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c67fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46312/0 (socket says 172.21.15.3:46312) 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c67fff700 1 -- 172.21.15.3:0/1625045824 learned_addr learned my addr 172.21.15.3:0/1625045824 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.574+0000 7f5c66ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68103d30 0x7f5c68198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 -- 172.21.15.3:0/1625045824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68103d30 msgr2=0x7f5c68198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68103d30 0x7f5c68198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 -- 172.21.15.3:0/1625045824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c68102b40 msgr2=0x7f5c681988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c68102b40 0x7f5c681988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 -- 172.21.15.3:0/1625045824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0095f0 con 0x7f5c681046d0 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c67fff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ef30 tx=0x7f5c6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60018070 con 0x7f5c681046d0 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c681a0970 con 0x7f5c681046d0 2023-08-11T14:02:00.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6000f040 con 0x7f5c681046d0 2023-08-11T14:02:00.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.575+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000cd00 con 0x7f5c681046d0 2023-08-11T14:02:00.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.576+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c681a0e30 con 0x7f5c681046d0 2023-08-11T14:02:00.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.576+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6810a130 con 0x7f5c681046d0 2023-08-11T14:02:00.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.579+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5c60003a00 con 0x7f5c681046d0 2023-08-11T14:02:00.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.580+0000 7f5c64ff9700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 0x7f5c500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:00.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.580+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c600caf60 con 0x7f5c681046d0 2023-08-11T14:02:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.580+0000 7f5c677fe700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 0x7f5c500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:00.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.581+0000 7f5c677fe700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 0x7f5c500a2410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c68103b90 tx=0x7f5c58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:00.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.584+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5c60094b30 con 0x7f5c681046d0 2023-08-11T14:02:00.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.828+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6802cdc0 con 0x7f5c5009ff50 2023-08-11T14:02:00.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.832+0000 7f5c64ff9700 1 -- 172.21.15.3:0/1625045824 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f5c6802cdc0 con 0x7f5c5009ff50 2023-08-11T14:02:00.834 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388 2023-08-11T14:02:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 msgr2=0x7f5c500a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 0x7f5c500a2410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c68103b90 tx=0x7f5c58009260 comp rx=0 tx=0).stop 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 msgr2=0x7f5c6819fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ef30 tx=0x7f5c6000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 shutdown_connections 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c681046d0 0x7f5c6819fe50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c5009ff50 0x7f5c500a2410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c68102b40 0x7f5c681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 --2- 172.21.15.3:0/1625045824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c68103d30 0x7f5c68198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:00.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 >> 172.21.15.3:0/1625045824 conn(0x7f5c680fe0d0 msgr2=0x7f5c68107270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:00.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 shutdown_connections 2023-08-11T14:02:00.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:00.835+0000 7f5c6ddff700 1 -- 172.21.15.3:0/1625045824 wait complete. 2023-08-11T14:02:00.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-08-11T14:02:01.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 -- 172.21.15.3:0/2509824888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0581024a0 msgr2=0x7fc058108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/2509824888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0581024a0 0x7fc058108d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc05000b0d0 tx=0x7fc05000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 -- 172.21.15.3:0/2509824888 shutdown_connections 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/2509824888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0581024a0 0x7fc058108d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/2509824888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc058100910 0x7fc058100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/2509824888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 0x7fc058101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 -- 172.21.15.3:0/2509824888 >> 172.21.15.3:0/2509824888 conn(0x7fc0580fbe60 msgr2=0x7fc0580fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:01.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 -- 172.21.15.3:0/2509824888 shutdown_connections 2023-08-11T14:02:01.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.062+0000 7fc05d3ff700 1 -- 172.21.15.3:0/2509824888 wait complete. 2023-08-11T14:02:01.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.063+0000 7fc05d3ff700 1 Processor -- start 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.063+0000 7fc05d3ff700 1 -- start start 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.063+0000 7fc05d3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.063+0000 7fc05d3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 0x7fc058072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.063+0000 7fc05d3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 0x7fc058073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc05d3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580739f0 con 0x7fc058100910 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc05d3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058073b60 con 0x7fc058101b00 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc05d3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581a0790 con 0x7fc0581024a0 2023-08-11T14:02:01.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46334/0 (socket says 172.21.15.3:46334) 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 -- 172.21.15.3:0/3520031813 learned_addr learned my addr 172.21.15.3:0/3520031813 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc0567fc700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 0x7fc058072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc0577fe700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 0x7fc058073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 -- 172.21.15.3:0/3520031813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 msgr2=0x7fc058073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 0x7fc058073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 -- 172.21.15.3:0/3520031813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 msgr2=0x7fc058072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 0x7fc058072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 -- 172.21.15.3:0/3520031813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050009d00 con 0x7fc058100910 2023-08-11T14:02:01.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc0577fe700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 0x7fc058073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:02:01.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.064+0000 7fc056ffd700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e970 tx=0x7fc04800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:01.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.065+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048009790 con 0x7fc058100910 2023-08-11T14:02:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.065+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0480044e0 con 0x7fc058100910 2023-08-11T14:02:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.065+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048003e30 con 0x7fc058100910 2023-08-11T14:02:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.065+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0581a0a70 con 0x7fc058100910 2023-08-11T14:02:01.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.065+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581a0f30 con 0x7fc058100910 2023-08-11T14:02:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.066+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc058107840 con 0x7fc058100910 2023-08-11T14:02:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.068+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc048003f90 con 0x7fc058100910 2023-08-11T14:02:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.069+0000 7fc03ffff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 0x7fc0400a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.069+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc048014070 con 0x7fc058100910 2023-08-11T14:02:01.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.070+0000 7fc0567fc700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 0x7fc0400a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.070+0000 7fc0567fc700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 0x7fc0400a2410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009bf0 tx=0x7fc04c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:01.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.071+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc048095380 con 0x7fc058100910 2023-08-11T14:02:01.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.319+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fc05802cf20 con 0x7fc04009ff50 2023-08-11T14:02:01.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.345+0000 7fc03ffff700 1 -- 172.21.15.3:0/3520031813 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc05802cf20 con 0x7fc04009ff50 2023-08-11T14:02:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.347+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 msgr2=0x7fc0400a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.347+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 0x7fc0400a2410 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009bf0 tx=0x7fc04c009260 comp rx=0 tx=0).stop 2023-08-11T14:02:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.347+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 msgr2=0x7fc0580746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.347+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e970 tx=0x7fc04800ed30 comp rx=0 tx=0).stop 2023-08-11T14:02:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 shutdown_connections 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0581024a0 0x7fc058073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc058100910 0x7fc0580746d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc04009ff50 0x7fc0400a2410 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 --2- 172.21.15.3:0/3520031813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc058101b00 0x7fc058072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 >> 172.21.15.3:0/3520031813 conn(0x7fc0580fbe60 msgr2=0x7fc0580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 shutdown_connections 2023-08-11T14:02:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.348+0000 7fc05d3ff700 1 -- 172.21.15.3:0/3520031813 wait complete. 2023-08-11T14:02:01.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-08-11T14:02:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.576+0000 7f50c1bff700 1 -- 172.21.15.3:0/409666254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 msgr2=0x7f50bc101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.576+0000 7f50c1bff700 1 --2- 172.21.15.3:0/409666254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc101f60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f50b00099c0 tx=0x7f50b0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 -- 172.21.15.3:0/409666254 shutdown_connections 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 --2- 172.21.15.3:0/409666254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 0x7f50bc108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 --2- 172.21.15.3:0/409666254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 --2- 172.21.15.3:0/409666254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc101f60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 -- 172.21.15.3:0/409666254 >> 172.21.15.3:0/409666254 conn(0x7f50bc0fbe60 msgr2=0x7f50bc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 -- 172.21.15.3:0/409666254 shutdown_connections 2023-08-11T14:02:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.577+0000 7f50c1bff700 1 -- 172.21.15.3:0/409666254 wait complete. 2023-08-11T14:02:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 Processor -- start 2023-08-11T14:02:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 -- start start 2023-08-11T14:02:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 0x7f50bc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1a03e0 con 0x7f50bc101b00 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1a0520 con 0x7f50bc1024a0 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.578+0000 7f50c1bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1a06b0 con 0x7f50bc100910 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50bb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41208/0 (socket says 172.21.15.3:41208) 2023-08-11T14:02:01.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 -- 172.21.15.3:0/671025051 learned_addr learned my addr 172.21.15.3:0/671025051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50bbfff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 0x7f50bc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 -- 172.21.15.3:0/671025051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 msgr2=0x7f50bc198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 -- 172.21.15.3:0/671025051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 msgr2=0x7f50bc19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 0x7f50bc19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 -- 172.21.15.3:0/671025051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00095f0 con 0x7f50bc101b00 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50bb7fe700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.579+0000 7f50baffd700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009990 tx=0x7f50b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.580+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b001d070 con 0x7f50bc101b00 2023-08-11T14:02:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.580+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b0005030 con 0x7f50bc101b00 2023-08-11T14:02:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.580+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004a70 con 0x7f50bc101b00 2023-08-11T14:02:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.580+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc1a0930 con 0x7f50bc101b00 2023-08-11T14:02:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.580+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1a0d00 con 0x7f50bc101b00 2023-08-11T14:02:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.581+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc10c170 con 0x7f50bc101b00 2023-08-11T14:02:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.583+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f50b0005410 con 0x7f50bc101b00 2023-08-11T14:02:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.584+0000 7f50b8ff9700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 0x7f50a40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.584+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f50b00cc5e0 con 0x7f50bc101b00 2023-08-11T14:02:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.584+0000 7f50bb7fe700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 0x7f50a40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.585+0000 7f50bb7fe700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 0x7f50a40a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50bc101960 tx=0x7f50ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.587+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f50b00961b0 con 0x7f50bc101b00 2023-08-11T14:02:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-08-11T14:02:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:active 2023-08-11T14:02:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-08-11T14:02:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-11T14:02:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: Cluster is now healthy 2023-08-11T14:02:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-08-11T14:02:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-08-11T14:02:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:01 smithi103 ceph-mon[130434]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-08-11T14:02:01.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:01.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-08-11T14:02:01.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:active 2023-08-11T14:02:01.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-08-11T14:02:01.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-11T14:02:01.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: Cluster is now healthy 2023-08-11T14:02:01.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-08-11T14:02:01.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-08-11T14:02:01.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:01 smithi003 ceph-mon[97195]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-08-11T14:02:01.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:01.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-08-11T14:02:01.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] up:active 2023-08-11T14:02:01.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-08-11T14:02:01.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-08-11T14:02:01.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: Cluster is now healthy 2023-08-11T14:02:01.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2023-08-11T14:02:01.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-08-11T14:02:01.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:01 smithi187 ceph-mon[130973]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-08-11T14:02:01.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.833+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50bc1084a0 con 0x7f50a409ffd0 2023-08-11T14:02:01.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.836+0000 7f50b8ff9700 1 -- 172.21.15.3:0/671025051 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f50bc1084a0 con 0x7f50a409ffd0 2023-08-11T14:02:01.838 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1 2023-08-11T14:02:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 msgr2=0x7f50a40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 0x7f50a40a2490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50bc101960 tx=0x7f50ac009260 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 msgr2=0x7f50bc198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009990 tx=0x7f50b000f690 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 shutdown_connections 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f50a409ffd0 0x7f50a40a2490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1024a0 0x7f50bc19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50bc100910 0x7f50bc198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 --2- 172.21.15.3:0/671025051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc101b00 0x7f50bc198da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 >> 172.21.15.3:0/671025051 conn(0x7f50bc0fbe60 msgr2=0x7f50bc0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:01.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 shutdown_connections 2023-08-11T14:02:01.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:01.839+0000 7f50c1bff700 1 -- 172.21.15.3:0/671025051 wait complete. 2023-08-11T14:02:01.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:02:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 -- 172.21.15.3:0/2447081796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 msgr2=0x7f6440100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 --2- 172.21.15.3:0/2447081796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f6440100d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f64300099c0 tx=0x7f6430009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 -- 172.21.15.3:0/2447081796 shutdown_connections 2023-08-11T14:02:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 --2- 172.21.15.3:0/2447081796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 --2- 172.21.15.3:0/2447081796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f6440100d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 --2- 172.21.15.3:0/2447081796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 0x7f6440101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 -- 172.21.15.3:0/2447081796 >> 172.21.15.3:0/2447081796 conn(0x7f64400fbe50 msgr2=0x7f64400fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:02.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 -- 172.21.15.3:0/2447081796 shutdown_connections 2023-08-11T14:02:02.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.061+0000 7f6445dff700 1 -- 172.21.15.3:0/2447081796 wait complete. 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.062+0000 7f6445dff700 1 Processor -- start 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 -- start start 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f64400746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 0x7f6440072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64400739f0 con 0x7f64401024b0 2023-08-11T14:02:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6440073b60 con 0x7f6440100920 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f6445dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a0600 con 0x7f6440101b10 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f643ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f643ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41232/0 (socket says 172.21.15.3:41232) 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f643ffff700 1 -- 172.21.15.3:0/3829656123 learned_addr learned my addr 172.21.15.3:0/3829656123 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f643effd700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 0x7f6440072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.063+0000 7f643f7fe700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f64400746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 -- 172.21.15.3:0/3829656123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 msgr2=0x7f6440072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 0x7f6440072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 -- 172.21.15.3:0/3829656123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 msgr2=0x7f64400746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f64400746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 -- 172.21.15.3:0/3829656123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300095f0 con 0x7f64401024b0 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643ffff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f643800ef30 tx=0x7f643800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438018070 con 0x7f64401024b0 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643800f040 con 0x7f64401024b0 2023-08-11T14:02:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64401a0880 con 0x7f64401024b0 2023-08-11T14:02:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643800cd00 con 0x7f64401024b0 2023-08-11T14:02:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.064+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401a0d90 con 0x7f64401024b0 2023-08-11T14:02:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.065+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6440107850 con 0x7f64401024b0 2023-08-11T14:02:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.067+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6438003a00 con 0x7f64401024b0 2023-08-11T14:02:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.069+0000 7f643cff9700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 0x7f64280a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.069+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f64380cb920 con 0x7f64401024b0 2023-08-11T14:02:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.069+0000 7f643f7fe700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 0x7f64280a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.070+0000 7f643f7fe700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 0x7f64280a24e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.071+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f64380954f0 con 0x7f64401024b0 2023-08-11T14:02:02.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.340+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f644010c380 con 0x7f64401024b0 2023-08-11T14:02:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.342+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f6438003cb0 con 0x7f64401024b0 2023-08-11T14:02:02.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.345+0000 7f643cff9700 1 -- 172.21.15.3:0/3829656123 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f64380d3090 con 0x7f64401024b0 2023-08-11T14:02:02.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:02.348 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 msgr2=0x7f64280a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 0x7f64280a24e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430005fb0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 msgr2=0x7f6440073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f643800ef30 tx=0x7f643800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 shutdown_connections 2023-08-11T14:02:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64401024b0 0x7f6440073250 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f64280a0020 0x7f64280a24e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6440100920 0x7f64400746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 --2- 172.21.15.3:0/3829656123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101b10 0x7f6440072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 >> 172.21.15.3:0/3829656123 conn(0x7f64400fbe50 msgr2=0x7f64400fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.348+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 shutdown_connections 2023-08-11T14:02:02.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.349+0000 7f6445dff700 1 -- 172.21.15.3:0/3829656123 wait complete. 2023-08-11T14:02:02.352 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:02:02.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:02:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 -- 172.21.15.3:0/1201222420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4520101b10 msgr2=0x7f4520101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 --2- 172.21.15.3:0/1201222420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4520101b10 0x7f4520101f70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45100099c0 tx=0x7f4510009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 -- 172.21.15.3:0/1201222420 shutdown_connections 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 --2- 172.21.15.3:0/1201222420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45201024b0 0x7f4520108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 --2- 172.21.15.3:0/1201222420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520100920 0x7f4520100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 --2- 172.21.15.3:0/1201222420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4520101b10 0x7f4520101f70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 -- 172.21.15.3:0/1201222420 >> 172.21.15.3:0/1201222420 conn(0x7f45200fbe50 msgr2=0x7f45200fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 -- 172.21.15.3:0/1201222420 shutdown_connections 2023-08-11T14:02:02.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.571+0000 7f45283ff700 1 -- 172.21.15.3:0/1201222420 wait complete. 2023-08-11T14:02:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.572+0000 7f45283ff700 1 Processor -- start 2023-08-11T14:02:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 -- start start 2023-08-11T14:02:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 0x7f4520196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 0x7f452019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019e140 con 0x7f45201024b0 2023-08-11T14:02:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019e280 con 0x7f4520101b10 2023-08-11T14:02:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f45283ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019e410 con 0x7f4520100920 2023-08-11T14:02:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f452699c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 0x7f452019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f452619b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f452619b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35808/0 (socket says 172.21.15.3:35808) 2023-08-11T14:02:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.573+0000 7f452619b700 1 -- 172.21.15.3:0/3062430468 learned_addr learned my addr 172.21.15.3:0/3062430468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452599a700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 0x7f4520196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 -- 172.21.15.3:0/3062430468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 msgr2=0x7f45201965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 -- 172.21.15.3:0/3062430468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 msgr2=0x7f4520196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 0x7f4520196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 -- 172.21.15.3:0/3062430468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45100095f0 con 0x7f45201024b0 2023-08-11T14:02:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452619b700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452599a700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 0x7f4520196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f452699c700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 0x7f452019db70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f451800efc0 tx=0x7f451800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518018070 con 0x7f45201024b0 2023-08-11T14:02:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4520072b50 con 0x7f45201024b0 2023-08-11T14:02:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451800f040 con 0x7f45201024b0 2023-08-11T14:02:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.574+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451800cd40 con 0x7f45201024b0 2023-08-11T14:02:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.575+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4520073010 con 0x7f45201024b0 2023-08-11T14:02:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.576+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452010c130 con 0x7f45201024b0 2023-08-11T14:02:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.577+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f4518003a00 con 0x7f45201024b0 2023-08-11T14:02:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.579+0000 7f45177fe700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 0x7f450c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.580+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f45180cb300 con 0x7f45201024b0 2023-08-11T14:02:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.580+0000 7f452619b700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 0x7f450c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.580+0000 7f452619b700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 0x7f450c0a24e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f451c005950 tx=0x7f451c012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.584+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4518094c60 con 0x7f45201024b0 2023-08-11T14:02:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:active 2023-08-11T14:02:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: fsmap cephfs:5 7 up:standby 5 up:active 2023-08-11T14:02:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:02:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2023-08-11T14:02:02.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:02 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:active 2023-08-11T14:02:02.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: fsmap cephfs:5 7 up:standby 5 up:active 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2023-08-11T14:02:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-08-11T14:02:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-08-11T14:02:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-08-11T14:02:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2023-08-11T14:02:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:02 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] up:active 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: fsmap cephfs:5 7 up:standby 5 up:active 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]: dispatch 2023-08-11T14:02:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:02 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-08-11T14:02:02.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.797+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f452010c320 con 0x7f45201024b0 2023-08-11T14:02:02.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.799+0000 7f45177fe700 1 -- 172.21.15.3:0/3062430468 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24357 (secure 0 0 0) 0x7f4518083020 con 0x7f45201024b0 2023-08-11T14:02:02.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:02.801 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":88,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:02:02.315079+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:02:02.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 msgr2=0x7f450c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 0x7f450c0a24e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f451c005950 tx=0x7f451c012040 comp rx=0 tx=0).stop 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 msgr2=0x7f452019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 0x7f452019db70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f451800efc0 tx=0x7f451800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 shutdown_connections 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45201024b0 0x7f452019db70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4520100920 0x7f45201965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f450c0a0020 0x7f450c0a24e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 --2- 172.21.15.3:0/3062430468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4520101b10 0x7f4520196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 >> 172.21.15.3:0/3062430468 conn(0x7f45200fbe50 msgr2=0x7f45200fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 shutdown_connections 2023-08-11T14:02:02.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:02.801+0000 7f45283ff700 1 -- 172.21.15.3:0/3062430468 wait complete. 2023-08-11T14:02:02.822 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-11T14:02:02.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:03.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 -- 172.21.15.3:0/2881291055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8101024c0 msgr2=0x7fe810108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2881291055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8101024c0 0x7fe810108d70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b0d0 tx=0x7fe80800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 -- 172.21.15.3:0/2881291055 shutdown_connections 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2881291055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8101024c0 0x7fe810108d70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2881291055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810100930 0x7fe810100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2881291055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 0x7fe810101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 -- 172.21.15.3:0/2881291055 >> 172.21.15.3:0/2881291055 conn(0x7fe8100fbe60 msgr2=0x7fe8100fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:03.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 -- 172.21.15.3:0/2881291055 shutdown_connections 2023-08-11T14:02:03.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.025+0000 7fe8151ff700 1 -- 172.21.15.3:0/2881291055 wait complete. 2023-08-11T14:02:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 Processor -- start 2023-08-11T14:02:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 -- start start 2023-08-11T14:02:03.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 0x7fe810072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8101024c0 0x7fe810073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8100739f0 con 0x7fe810100930 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810073b60 con 0x7fe810101b20 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.026+0000 7fe8151ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8101a06d0 con 0x7fe8101024c0 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41264/0 (socket says 172.21.15.3:41264) 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 -- 172.21.15.3:0/2917184720 learned_addr learned my addr 172.21.15.3:0/2917184720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80f59e700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8101024c0 0x7fe810073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80e59c700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 0x7fe810072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 -- 172.21.15.3:0/2917184720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8101024c0 msgr2=0x7fe810073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8101024c0 0x7fe810073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 -- 172.21.15.3:0/2917184720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 msgr2=0x7fe810072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 0x7fe810072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 -- 172.21.15.3:0/2917184720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009d00 con 0x7fe810100930 2023-08-11T14:02:03.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.027+0000 7fe80ed9d700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe80000e970 tx=0x7fe80000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:03.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.028+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800009790 con 0x7fe810100930 2023-08-11T14:02:03.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.028+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80000c7e0 con 0x7fe810100930 2023-08-11T14:02:03.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.028+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8101a09b0 con 0x7fe810100930 2023-08-11T14:02:03.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.028+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800003c70 con 0x7fe810100930 2023-08-11T14:02:03.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.028+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8101a0f10 con 0x7fe810100930 2023-08-11T14:02:03.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.029+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81010c150 con 0x7fe810100930 2023-08-11T14:02:03.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.031+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fe80000cc50 con 0x7fe810100930 2023-08-11T14:02:03.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.033+0000 7fe7f7fff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 0x7fe7f80a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.033+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe800014070 con 0x7fe810100930 2023-08-11T14:02:03.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.033+0000 7fe80e59c700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 0x7fe7f80a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.034+0000 7fe80e59c700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 0x7fe7f80a2560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe804007be0 tx=0x7fe8040073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:03.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.037+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe800095160 con 0x7fe810100930 2023-08-11T14:02:03.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.245+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe810060d60 con 0x7fe7f80a00a0 2023-08-11T14:02:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.261+0000 7fe7f7fff700 1 -- 172.21.15.3:0/2917184720 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422748 (secure 0 0 0) 0x7fe810060d60 con 0x7fe7f80a00a0 2023-08-11T14:02:03.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 msgr2=0x7fe7f80a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 0x7fe7f80a2560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe804007be0 tx=0x7fe8040073d0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 msgr2=0x7fe8100746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe80000e970 tx=0x7fe80000ec80 comp rx=0 tx=0).stop 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 shutdown_connections 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8101024c0 0x7fe810073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810100930 0x7fe8100746d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7f80a00a0 0x7fe7f80a2560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 --2- 172.21.15.3:0/2917184720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810101b20 0x7fe810072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 >> 172.21.15.3:0/2917184720 conn(0x7fe8100fbe60 msgr2=0x7fe8100fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 shutdown_connections 2023-08-11T14:02:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.266+0000 7fe8151ff700 1 -- 172.21.15.3:0/2917184720 wait complete. 2023-08-11T14:02:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:03.287 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2023-08-11T14:02:02.327182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600355,"num_objects":81,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":151,"num_read_kb":132,"num_write":345,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337472,"kb_used_data":13280,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604352,"statfs":{"total":1151940427776,"available":1151594856448,"internally_reserved":0,"allocated":13598720,"data_stored":7395280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5488,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":25,"num_read_kb":23,"num_write":116,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.163672"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454489+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:01:38.454489+0000","last_peered":"2023-08-11T14:01:38.454489+0000","last_clean":"2023-08-11T14:01:38.454489+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:01:38.454489+0000","last_undegraded":"2023-08-11T14:01:38.454489+0000","last_fullsized":"2023-08-11T14:01:38.454489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:15.565502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453396+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:01:38.453396+0000","last_peered":"2023-08-11T14:01:38.453396+0000","last_clean":"2023-08-11T14:01:38.453396+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:01:38.453396+0000","last_undegraded":"2023-08-11T14:01:38.453396+0000","last_fullsized":"2023-08-11T14:01:38.453396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:05.487390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453350+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:01:38.453350+0000","last_peered":"2023-08-11T14:01:38.453350+0000","last_clean":"2023-08-11T14:01:38.453350+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:01:38.453350+0000","last_undegraded":"2023-08-11T14:01:38.453350+0000","last_fullsized":"2023-08-11T14:01:38.453350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:32:07.002359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453773+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:01:38.453773+0000","last_peered":"2023-08-11T14:01:38.453773+0000","last_clean":"2023-08-11T14:01:38.453773+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:01:38.453773+0000","last_undegraded":"2023-08-11T14:01:38.453773+0000","last_fullsized":"2023-08-11T14:01:38.453773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:24.933688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357698+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:01:38.357698+0000","last_peered":"2023-08-11T14:01:38.357698+0000","last_clean":"2023-08-11T14:01:38.357698+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:01:38.357698+0000","last_undegraded":"2023-08-11T14:01:38.357698+0000","last_fullsized":"2023-08-11T14:01:38.357698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:30:43.465070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454651+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:01:38.454651+0000","last_peered":"2023-08-11T14:01:38.454651+0000","last_clean":"2023-08-11T14:01:38.454651+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:01:38.454651+0000","last_undegraded":"2023-08-11T14:01:38.454651+0000","last_fullsized":"2023-08-11T14:01:38.454651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:11:08.034766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453427+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:01:38.453427+0000","last_peered":"2023-08-11T14:01:38.453427+0000","last_clean":"2023-08-11T14:01:38.453427+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:01:38.453427+0000","last_undegraded":"2023-08-11T14:01:38.453427+0000","last_fullsized":"2023-08-11T14:01:38.453427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:45:09.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454191+0000","last_change":"2023-08-11T14:01:33.312566+0000","last_active":"2023-08-11T14:01:38.454191+0000","last_peered":"2023-08-11T14:01:38.454191+0000","last_clean":"2023-08-11T14:01:38.454191+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454191+0000","last_undegraded":"2023-08-11T14:01:38.454191+0000","last_fullsized":"2023-08-11T14:01:38.454191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:49:47.771091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358208+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:01:38.358208+0000","last_peered":"2023-08-11T14:01:38.358208+0000","last_clean":"2023-08-11T14:01:38.358208+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:01:38.358208+0000","last_undegraded":"2023-08-11T14:01:38.358208+0000","last_fullsized":"2023-08-11T14:01:38.358208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:30.807708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455163+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:01:38.455163+0000","last_peered":"2023-08-11T14:01:38.455163+0000","last_clean":"2023-08-11T14:01:38.455163+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:01:38.455163+0000","last_undegraded":"2023-08-11T14:01:38.455163+0000","last_fullsized":"2023-08-11T14:01:38.455163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:15.132967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454608+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:01:38.454608+0000","last_peered":"2023-08-11T14:01:38.454608+0000","last_clean":"2023-08-11T14:01:38.454608+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:01:38.454608+0000","last_undegraded":"2023-08-11T14:01:38.454608+0000","last_fullsized":"2023-08-11T14:01:38.454608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:36.691099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453612+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:01:38.453612+0000","last_peered":"2023-08-11T14:01:38.453612+0000","last_clean":"2023-08-11T14:01:38.453612+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:01:38.453612+0000","last_undegraded":"2023-08-11T14:01:38.453612+0000","last_fullsized":"2023-08-11T14:01:38.453612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:41.734924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357734+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:01:38.357734+0000","last_peered":"2023-08-11T14:01:38.357734+0000","last_clean":"2023-08-11T14:01:38.357734+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:01:38.357734+0000","last_undegraded":"2023-08-11T14:01:38.357734+0000","last_fullsized":"2023-08-11T14:01:38.357734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:49:21.624209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453057+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:01:38.453057+0000","last_peered":"2023-08-11T14:01:38.453057+0000","last_clean":"2023-08-11T14:01:38.453057+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:01:38.453057+0000","last_undegraded":"2023-08-11T14:01:38.453057+0000","last_fullsized":"2023-08-11T14:01:38.453057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:54:14.411087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453329+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:01:38.453329+0000","last_peered":"2023-08-11T14:01:38.453329+0000","last_clean":"2023-08-11T14:01:38.453329+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:01:38.453329+0000","last_undegraded":"2023-08-11T14:01:38.453329+0000","last_fullsized":"2023-08-11T14:01:38.453329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:45.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357450+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:01:38.357450+0000","last_peered":"2023-08-11T14:01:38.357450+0000","last_clean":"2023-08-11T14:01:38.357450+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:01:38.357450+0000","last_undegraded":"2023-08-11T14:01:38.357450+0000","last_fullsized":"2023-08-11T14:01:38.357450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:49.542397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454690+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:01:38.454690+0000","last_peered":"2023-08-11T14:01:38.454690+0000","last_clean":"2023-08-11T14:01:38.454690+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:01:38.454690+0000","last_undegraded":"2023-08-11T14:01:38.454690+0000","last_fullsized":"2023-08-11T14:01:38.454690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:06:02.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455475+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:01:38.455475+0000","last_peered":"2023-08-11T14:01:38.455475+0000","last_clean":"2023-08-11T14:01:38.455475+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:01:38.455475+0000","last_undegraded":"2023-08-11T14:01:38.455475+0000","last_fullsized":"2023-08-11T14:01:38.455475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:52:58.018698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454406+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:01:38.454406+0000","last_peered":"2023-08-11T14:01:38.454406+0000","last_clean":"2023-08-11T14:01:38.454406+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:01:38.454406+0000","last_undegraded":"2023-08-11T14:01:38.454406+0000","last_fullsized":"2023-08-11T14:01:38.454406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:02:50.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455090+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:01:38.455090+0000","last_peered":"2023-08-11T14:01:38.455090+0000","last_clean":"2023-08-11T14:01:38.455090+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:01:38.455090+0000","last_undegraded":"2023-08-11T14:01:38.455090+0000","last_fullsized":"2023-08-11T14:01:38.455090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:39.906536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454623+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:01:38.454623+0000","last_peered":"2023-08-11T14:01:38.454623+0000","last_clean":"2023-08-11T14:01:38.454623+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:01:38.454623+0000","last_undegraded":"2023-08-11T14:01:38.454623+0000","last_fullsized":"2023-08-11T14:01:38.454623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:44.659968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454673+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:01:38.454673+0000","last_peered":"2023-08-11T14:01:38.454673+0000","last_clean":"2023-08-11T14:01:38.454673+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:01:38.454673+0000","last_undegraded":"2023-08-11T14:01:38.454673+0000","last_fullsized":"2023-08-11T14:01:38.454673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:09:31.987246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453033+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:01:38.453033+0000","last_peered":"2023-08-11T14:01:38.453033+0000","last_clean":"2023-08-11T14:01:38.453033+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:01:38.453033+0000","last_undegraded":"2023-08-11T14:01:38.453033+0000","last_fullsized":"2023-08-11T14:01:38.453033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:24:09.010833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454851+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:01:38.454851+0000","last_peered":"2023-08-11T14:01:38.454851+0000","last_clean":"2023-08-11T14:01:38.454851+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:01:38.454851+0000","last_undegraded":"2023-08-11T14:01:38.454851+0000","last_fullsized":"2023-08-11T14:01:38.454851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:09.972413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453569+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:01:38.453569+0000","last_peered":"2023-08-11T14:01:38.453569+0000","last_clean":"2023-08-11T14:01:38.453569+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:01:38.453569+0000","last_undegraded":"2023-08-11T14:01:38.453569+0000","last_fullsized":"2023-08-11T14:01:38.453569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:31.879999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453610+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:01:38.453610+0000","last_peered":"2023-08-11T14:01:38.453610+0000","last_clean":"2023-08-11T14:01:38.453610+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:01:38.453610+0000","last_undegraded":"2023-08-11T14:01:38.453610+0000","last_fullsized":"2023-08-11T14:01:38.453610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:00.077438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322139+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:00.322139+0000","last_peered":"2023-08-11T14:02:00.322139+0000","last_clean":"2023-08-11T14:02:00.322139+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:00.322139+0000","last_undegraded":"2023-08-11T14:02:00.322139+0000","last_fullsized":"2023-08-11T14:02:00.322139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:53.482576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357683+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:01:38.357683+0000","last_peered":"2023-08-11T14:01:38.357683+0000","last_clean":"2023-08-11T14:01:38.357683+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:01:38.357683+0000","last_undegraded":"2023-08-11T14:01:38.357683+0000","last_fullsized":"2023-08-11T14:01:38.357683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:45:28.652581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.452997+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:01:38.452997+0000","last_peered":"2023-08-11T14:01:38.452997+0000","last_clean":"2023-08-11T14:01:38.452997+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:01:38.452997+0000","last_undegraded":"2023-08-11T14:01:38.452997+0000","last_fullsized":"2023-08-11T14:01:38.452997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:46.488285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320442+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:00.320442+0000","last_peered":"2023-08-11T14:02:00.320442+0000","last_clean":"2023-08-11T14:02:00.320442+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:00.320442+0000","last_undegraded":"2023-08-11T14:02:00.320442+0000","last_fullsized":"2023-08-11T14:02:00.320442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:37:57.485057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455704+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:01:38.455704+0000","last_peered":"2023-08-11T14:01:38.455704+0000","last_clean":"2023-08-11T14:01:38.455704+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:01:38.455704+0000","last_undegraded":"2023-08-11T14:01:38.455704+0000","last_fullsized":"2023-08-11T14:01:38.455704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:25.911259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454143+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:01:38.454143+0000","last_peered":"2023-08-11T14:01:38.454143+0000","last_clean":"2023-08-11T14:01:38.454143+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:01:38.454143+0000","last_undegraded":"2023-08-11T14:01:38.454143+0000","last_fullsized":"2023-08-11T14:01:38.454143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:00.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321984+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:01.321984+0000","last_peered":"2023-08-11T14:02:01.321984+0000","last_clean":"2023-08-11T14:02:01.321984+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:01.321984+0000","last_undegraded":"2023-08-11T14:02:01.321984+0000","last_fullsized":"2023-08-11T14:02:01.321984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:10.771366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357613+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:01:38.357613+0000","last_peered":"2023-08-11T14:01:38.357613+0000","last_clean":"2023-08-11T14:01:38.357613+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:01:38.357613+0000","last_undegraded":"2023-08-11T14:01:38.357613+0000","last_fullsized":"2023-08-11T14:01:38.357613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:46:18.787688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454174+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:01:38.454174+0000","last_peered":"2023-08-11T14:01:38.454174+0000","last_clean":"2023-08-11T14:01:38.454174+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:01:38.454174+0000","last_undegraded":"2023-08-11T14:01:38.454174+0000","last_fullsized":"2023-08-11T14:01:38.454174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:21.945068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305879+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:01:58.305879+0000","last_peered":"2023-08-11T14:01:58.305879+0000","last_clean":"2023-08-11T14:01:58.305879+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:01:58.305879+0000","last_undegraded":"2023-08-11T14:01:58.305879+0000","last_fullsized":"2023-08-11T14:01:58.305879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:33.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455026+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:01:38.455026+0000","last_peered":"2023-08-11T14:01:38.455026+0000","last_clean":"2023-08-11T14:01:38.455026+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:01:38.455026+0000","last_undegraded":"2023-08-11T14:01:38.455026+0000","last_fullsized":"2023-08-11T14:01:38.455026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:26:04.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454393+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:01:38.454393+0000","last_peered":"2023-08-11T14:01:38.454393+0000","last_clean":"2023-08-11T14:01:38.454393+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:01:38.454393+0000","last_undegraded":"2023-08-11T14:01:38.454393+0000","last_fullsized":"2023-08-11T14:01:38.454393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:09.108932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.307340+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:01:58.307340+0000","last_peered":"2023-08-11T14:01:58.307340+0000","last_clean":"2023-08-11T14:01:58.307340+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:01:58.307340+0000","last_undegraded":"2023-08-11T14:01:58.307340+0000","last_fullsized":"2023-08-11T14:01:58.307340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:28.668048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454198+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:01:38.454198+0000","last_peered":"2023-08-11T14:01:38.454198+0000","last_clean":"2023-08-11T14:01:38.454198+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:01:38.454198+0000","last_undegraded":"2023-08-11T14:01:38.454198+0000","last_fullsized":"2023-08-11T14:01:38.454198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:02.471143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453242+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:01:38.453242+0000","last_peered":"2023-08-11T14:01:38.453242+0000","last_clean":"2023-08-11T14:01:38.453242+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:01:38.453242+0000","last_undegraded":"2023-08-11T14:01:38.453242+0000","last_fullsized":"2023-08-11T14:01:38.453242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:21:10.060538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321844+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:00.321844+0000","last_peered":"2023-08-11T14:02:00.321844+0000","last_clean":"2023-08-11T14:02:00.321844+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:00.321844+0000","last_undegraded":"2023-08-11T14:02:00.321844+0000","last_fullsized":"2023-08-11T14:02:00.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:29.646533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455392+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:01:38.455392+0000","last_peered":"2023-08-11T14:01:38.455392+0000","last_clean":"2023-08-11T14:01:38.455392+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:01:38.455392+0000","last_undegraded":"2023-08-11T14:01:38.455392+0000","last_fullsized":"2023-08-11T14:01:38.455392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:44.217117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454637+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:01:38.454637+0000","last_peered":"2023-08-11T14:01:38.454637+0000","last_clean":"2023-08-11T14:01:38.454637+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:01:38.454637+0000","last_undegraded":"2023-08-11T14:01:38.454637+0000","last_fullsized":"2023-08-11T14:01:38.454637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:14:38.485351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.292369+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:01:57.292369+0000","last_peered":"2023-08-11T14:01:57.292369+0000","last_clean":"2023-08-11T14:01:57.292369+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:01:57.292369+0000","last_undegraded":"2023-08-11T14:01:57.292369+0000","last_fullsized":"2023-08-11T14:01:57.292369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:34:53.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358143+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:01:38.358143+0000","last_peered":"2023-08-11T14:01:38.358143+0000","last_clean":"2023-08-11T14:01:38.358143+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:01:38.358143+0000","last_undegraded":"2023-08-11T14:01:38.358143+0000","last_fullsized":"2023-08-11T14:01:38.358143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:56.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454492+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:01:38.454492+0000","last_peered":"2023-08-11T14:01:38.454492+0000","last_clean":"2023-08-11T14:01:38.454492+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:01:38.454492+0000","last_undegraded":"2023-08-11T14:01:38.454492+0000","last_fullsized":"2023-08-11T14:01:38.454492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:41.341028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293732+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:01:57.293732+0000","last_peered":"2023-08-11T14:01:57.293732+0000","last_clean":"2023-08-11T14:01:57.293732+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:01:57.293732+0000","last_undegraded":"2023-08-11T14:01:57.293732+0000","last_fullsized":"2023-08-11T14:01:57.293732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:00.643436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453798+0000","last_change":"2023-08-11T14:01:33.308455+0000","last_active":"2023-08-11T14:01:38.453798+0000","last_peered":"2023-08-11T14:01:38.453798+0000","last_clean":"2023-08-11T14:01:38.453798+0000","last_became_active":"2023-08-11T14:01:33.308084+0000","last_became_peered":"2023-08-11T14:01:33.308084+0000","last_unstale":"2023-08-11T14:01:38.453798+0000","last_undegraded":"2023-08-11T14:01:38.453798+0000","last_fullsized":"2023-08-11T14:01:38.453798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:33.705786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453779+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:01:38.453779+0000","last_peered":"2023-08-11T14:01:38.453779+0000","last_clean":"2023-08-11T14:01:38.453779+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:01:38.453779+0000","last_undegraded":"2023-08-11T14:01:38.453779+0000","last_fullsized":"2023-08-11T14:01:38.453779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:41.727745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312606+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:01:59.312606+0000","last_peered":"2023-08-11T14:01:59.312606+0000","last_clean":"2023-08-11T14:01:59.312606+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:01:59.312606+0000","last_undegraded":"2023-08-11T14:01:59.312606+0000","last_fullsized":"2023-08-11T14:01:59.312606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:33:25.490224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357940+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:01:38.357940+0000","last_peered":"2023-08-11T14:01:38.357940+0000","last_clean":"2023-08-11T14:01:38.357940+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:01:38.357940+0000","last_undegraded":"2023-08-11T14:01:38.357940+0000","last_fullsized":"2023-08-11T14:01:38.357940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:05:22.390481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455321+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:01:38.455321+0000","last_peered":"2023-08-11T14:01:38.455321+0000","last_clean":"2023-08-11T14:01:38.455321+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:01:38.455321+0000","last_undegraded":"2023-08-11T14:01:38.455321+0000","last_fullsized":"2023-08-11T14:01:38.455321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:08.091777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.295025+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:01:57.295025+0000","last_peered":"2023-08-11T14:01:57.295025+0000","last_clean":"2023-08-11T14:01:57.295025+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:01:57.295025+0000","last_undegraded":"2023-08-11T14:01:57.295025+0000","last_fullsized":"2023-08-11T14:01:57.295025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:38:00.566235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357914+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:01:38.357914+0000","last_peered":"2023-08-11T14:01:38.357914+0000","last_clean":"2023-08-11T14:01:38.357914+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:01:38.357914+0000","last_undegraded":"2023-08-11T14:01:38.357914+0000","last_fullsized":"2023-08-11T14:01:38.357914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:21:16.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455907+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:01:38.455907+0000","last_peered":"2023-08-11T14:01:38.455907+0000","last_clean":"2023-08-11T14:01:38.455907+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:01:38.455907+0000","last_undegraded":"2023-08-11T14:01:38.455907+0000","last_fullsized":"2023-08-11T14:01:38.455907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:56.419641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305981+0000","last_change":"2023-08-11T14:01:32.306792+0000","last_active":"2023-08-11T14:01:58.305981+0000","last_peered":"2023-08-11T14:01:58.305981+0000","last_clean":"2023-08-11T14:01:58.305981+0000","last_became_active":"2023-08-11T14:01:32.306447+0000","last_became_peered":"2023-08-11T14:01:32.306447+0000","last_unstale":"2023-08-11T14:01:58.305981+0000","last_undegraded":"2023-08-11T14:01:58.305981+0000","last_fullsized":"2023-08-11T14:01:58.305981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:09.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357607+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:01:38.357607+0000","last_peered":"2023-08-11T14:01:38.357607+0000","last_clean":"2023-08-11T14:01:38.357607+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:01:38.357607+0000","last_undegraded":"2023-08-11T14:01:38.357607+0000","last_fullsized":"2023-08-11T14:01:38.357607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:58:53.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453894+0000","last_change":"2023-08-11T14:01:37.375702+0000","last_active":"2023-08-11T14:01:38.453894+0000","last_peered":"2023-08-11T14:01:38.453894+0000","last_clean":"2023-08-11T14:01:38.453894+0000","last_became_active":"2023-08-11T14:01:37.375199+0000","last_became_peered":"2023-08-11T14:01:37.375199+0000","last_unstale":"2023-08-11T14:01:38.453894+0000","last_undegraded":"2023-08-11T14:01:38.453894+0000","last_fullsized":"2023-08-11T14:01:38.453894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:14.150791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321941+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:00.321941+0000","last_peered":"2023-08-11T14:02:00.321941+0000","last_clean":"2023-08-11T14:02:00.321941+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:00.321941+0000","last_undegraded":"2023-08-11T14:02:00.321941+0000","last_fullsized":"2023-08-11T14:02:00.321941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:02:07.982583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453519+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:01:38.453519+0000","last_peered":"2023-08-11T14:01:38.453519+0000","last_clean":"2023-08-11T14:01:38.453519+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:01:38.453519+0000","last_undegraded":"2023-08-11T14:01:38.453519+0000","last_fullsized":"2023-08-11T14:01:38.453519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:57.895242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454218+0000","last_change":"2023-08-11T14:01:37.377532+0000","last_active":"2023-08-11T14:01:38.454218+0000","last_peered":"2023-08-11T14:01:38.454218+0000","last_clean":"2023-08-11T14:01:38.454218+0000","last_became_active":"2023-08-11T14:01:37.377142+0000","last_became_peered":"2023-08-11T14:01:37.377142+0000","last_unstale":"2023-08-11T14:01:38.454218+0000","last_undegraded":"2023-08-11T14:01:38.454218+0000","last_fullsized":"2023-08-11T14:01:38.454218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:13.506622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321844+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:01.321844+0000","last_peered":"2023-08-11T14:02:01.321844+0000","last_clean":"2023-08-11T14:02:01.321844+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:01.321844+0000","last_undegraded":"2023-08-11T14:02:01.321844+0000","last_fullsized":"2023-08-11T14:02:01.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:27.821349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453776+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:01:38.453776+0000","last_peered":"2023-08-11T14:01:38.453776+0000","last_clean":"2023-08-11T14:01:38.453776+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:01:38.453776+0000","last_undegraded":"2023-08-11T14:01:38.453776+0000","last_fullsized":"2023-08-11T14:01:38.453776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:13:35.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357992+0000","last_change":"2023-08-11T14:01:37.381723+0000","last_active":"2023-08-11T14:01:38.357992+0000","last_peered":"2023-08-11T14:01:38.357992+0000","last_clean":"2023-08-11T14:01:38.357992+0000","last_became_active":"2023-08-11T14:01:37.381306+0000","last_became_peered":"2023-08-11T14:01:37.381306+0000","last_unstale":"2023-08-11T14:01:38.357992+0000","last_undegraded":"2023-08-11T14:01:38.357992+0000","last_fullsized":"2023-08-11T14:01:38.357992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:27.079422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.318041+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:00.318041+0000","last_peered":"2023-08-11T14:02:00.318041+0000","last_clean":"2023-08-11T14:02:00.318041+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:00.318041+0000","last_undegraded":"2023-08-11T14:02:00.318041+0000","last_fullsized":"2023-08-11T14:02:00.318041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:33:54.507816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454542+0000","last_change":"2023-08-11T14:01:33.304616+0000","last_active":"2023-08-11T14:01:38.454542+0000","last_peered":"2023-08-11T14:01:38.454542+0000","last_clean":"2023-08-11T14:01:38.454542+0000","last_became_active":"2023-08-11T14:01:33.303833+0000","last_became_peered":"2023-08-11T14:01:33.303833+0000","last_unstale":"2023-08-11T14:01:38.454542+0000","last_undegraded":"2023-08-11T14:01:38.454542+0000","last_fullsized":"2023-08-11T14:01:38.454542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:07.125672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454385+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:01:38.454385+0000","last_peered":"2023-08-11T14:01:38.454385+0000","last_clean":"2023-08-11T14:01:38.454385+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:01:38.454385+0000","last_undegraded":"2023-08-11T14:01:38.454385+0000","last_fullsized":"2023-08-11T14:01:38.454385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:21.288285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.230701+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:01.230701+0000","last_peered":"2023-08-11T14:02:01.230701+0000","last_clean":"2023-08-11T14:02:01.230701+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:01.230701+0000","last_undegraded":"2023-08-11T14:02:01.230701+0000","last_fullsized":"2023-08-11T14:02:01.230701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:14.365596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453956+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:01:38.453956+0000","last_peered":"2023-08-11T14:01:38.453956+0000","last_clean":"2023-08-11T14:01:38.453956+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:01:38.453956+0000","last_undegraded":"2023-08-11T14:01:38.453956+0000","last_fullsized":"2023-08-11T14:01:38.453956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:22.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357583+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:01:38.357583+0000","last_peered":"2023-08-11T14:01:38.357583+0000","last_clean":"2023-08-11T14:01:38.357583+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:01:38.357583+0000","last_undegraded":"2023-08-11T14:01:38.357583+0000","last_fullsized":"2023-08-11T14:01:38.357583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:04:46.172105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454795+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:01:38.454795+0000","last_peered":"2023-08-11T14:01:38.454795+0000","last_clean":"2023-08-11T14:01:38.454795+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:01:38.454795+0000","last_undegraded":"2023-08-11T14:01:38.454795+0000","last_fullsized":"2023-08-11T14:01:38.454795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:31:14.848929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454940+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:01:38.454940+0000","last_peered":"2023-08-11T14:01:38.454940+0000","last_clean":"2023-08-11T14:01:38.454940+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:01:38.454940+0000","last_undegraded":"2023-08-11T14:01:38.454940+0000","last_fullsized":"2023-08-11T14:01:38.454940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:55:34.033468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357752+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:01:38.357752+0000","last_peered":"2023-08-11T14:01:38.357752+0000","last_clean":"2023-08-11T14:01:38.357752+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:01:38.357752+0000","last_undegraded":"2023-08-11T14:01:38.357752+0000","last_fullsized":"2023-08-11T14:01:38.357752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:12.528692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453781+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:01:38.453781+0000","last_peered":"2023-08-11T14:01:38.453781+0000","last_clean":"2023-08-11T14:01:38.453781+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:01:38.453781+0000","last_undegraded":"2023-08-11T14:01:38.453781+0000","last_fullsized":"2023-08-11T14:01:38.453781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:37.699260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455015+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:01:38.455015+0000","last_peered":"2023-08-11T14:01:38.455015+0000","last_clean":"2023-08-11T14:01:38.455015+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:01:38.455015+0000","last_undegraded":"2023-08-11T14:01:38.455015+0000","last_fullsized":"2023-08-11T14:01:38.455015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:05.858370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357865+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:01:38.357865+0000","last_peered":"2023-08-11T14:01:38.357865+0000","last_clean":"2023-08-11T14:01:38.357865+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:01:38.357865+0000","last_undegraded":"2023-08-11T14:01:38.357865+0000","last_fullsized":"2023-08-11T14:01:38.357865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:18.092395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455326+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:01:38.455326+0000","last_peered":"2023-08-11T14:01:38.455326+0000","last_clean":"2023-08-11T14:01:38.455326+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:01:38.455326+0000","last_undegraded":"2023-08-11T14:01:38.455326+0000","last_fullsized":"2023-08-11T14:01:38.455326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:57.290710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.308543+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:01:58.308543+0000","last_peered":"2023-08-11T14:01:58.308543+0000","last_clean":"2023-08-11T14:01:58.308543+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:01:58.308543+0000","last_undegraded":"2023-08-11T14:01:58.308543+0000","last_fullsized":"2023-08-11T14:01:58.308543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:33.988051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454372+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:01:38.454372+0000","last_peered":"2023-08-11T14:01:38.454372+0000","last_clean":"2023-08-11T14:01:38.454372+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:01:38.454372+0000","last_undegraded":"2023-08-11T14:01:38.454372+0000","last_fullsized":"2023-08-11T14:01:38.454372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:32:11.305694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454548+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:01:38.454548+0000","last_peered":"2023-08-11T14:01:38.454548+0000","last_clean":"2023-08-11T14:01:38.454548+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:01:38.454548+0000","last_undegraded":"2023-08-11T14:01:38.454548+0000","last_fullsized":"2023-08-11T14:01:38.454548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:38.933163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.319779+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:01.319779+0000","last_peered":"2023-08-11T14:02:01.319779+0000","last_clean":"2023-08-11T14:02:01.319779+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:01.319779+0000","last_undegraded":"2023-08-11T14:02:01.319779+0000","last_fullsized":"2023-08-11T14:02:01.319779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:56:44.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453444+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:01:38.453444+0000","last_peered":"2023-08-11T14:01:38.453444+0000","last_clean":"2023-08-11T14:01:38.453444+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:01:38.453444+0000","last_undegraded":"2023-08-11T14:01:38.453444+0000","last_fullsized":"2023-08-11T14:01:38.453444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:10.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455405+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:01:38.455405+0000","last_peered":"2023-08-11T14:01:38.455405+0000","last_clean":"2023-08-11T14:01:38.455405+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:01:38.455405+0000","last_undegraded":"2023-08-11T14:01:38.455405+0000","last_fullsized":"2023-08-11T14:01:38.455405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:31:57.406400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357235+0000","last_change":"2023-08-11T14:01:33.305155+0000","last_active":"2023-08-11T14:01:38.357235+0000","last_peered":"2023-08-11T14:01:38.357235+0000","last_clean":"2023-08-11T14:01:38.357235+0000","last_became_active":"2023-08-11T14:01:33.304789+0000","last_became_peered":"2023-08-11T14:01:33.304789+0000","last_unstale":"2023-08-11T14:01:38.357235+0000","last_undegraded":"2023-08-11T14:01:38.357235+0000","last_fullsized":"2023-08-11T14:01:38.357235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:54.803726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305859+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:01:58.305859+0000","last_peered":"2023-08-11T14:01:58.305859+0000","last_clean":"2023-08-11T14:01:58.305859+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:01:58.305859+0000","last_undegraded":"2023-08-11T14:01:58.305859+0000","last_fullsized":"2023-08-11T14:01:58.305859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:21:11.143846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454625+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:01:38.454625+0000","last_peered":"2023-08-11T14:01:38.454625+0000","last_clean":"2023-08-11T14:01:38.454625+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:01:38.454625+0000","last_undegraded":"2023-08-11T14:01:38.454625+0000","last_fullsized":"2023-08-11T14:01:38.454625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:40:26.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320780+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:00.320780+0000","last_peered":"2023-08-11T14:02:00.320780+0000","last_clean":"2023-08-11T14:02:00.320780+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:00.320780+0000","last_undegraded":"2023-08-11T14:02:00.320780+0000","last_fullsized":"2023-08-11T14:02:00.320780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:17:38.449276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357848+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:01:38.357848+0000","last_peered":"2023-08-11T14:01:38.357848+0000","last_clean":"2023-08-11T14:01:38.357848+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:01:38.357848+0000","last_undegraded":"2023-08-11T14:01:38.357848+0000","last_fullsized":"2023-08-11T14:01:38.357848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:53:41.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454047+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:01:38.454047+0000","last_peered":"2023-08-11T14:01:38.454047+0000","last_clean":"2023-08-11T14:01:38.454047+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:01:38.454047+0000","last_undegraded":"2023-08-11T14:01:38.454047+0000","last_fullsized":"2023-08-11T14:01:38.454047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:47:11.994511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455558+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:01:38.455558+0000","last_peered":"2023-08-11T14:01:38.455558+0000","last_clean":"2023-08-11T14:01:38.455558+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:01:38.455558+0000","last_undegraded":"2023-08-11T14:01:38.455558+0000","last_fullsized":"2023-08-11T14:01:38.455558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:26:48.337276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455078+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:01:38.455078+0000","last_peered":"2023-08-11T14:01:38.455078+0000","last_clean":"2023-08-11T14:01:38.455078+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:01:38.455078+0000","last_undegraded":"2023-08-11T14:01:38.455078+0000","last_fullsized":"2023-08-11T14:01:38.455078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:14:16.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454784+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:01:38.454784+0000","last_peered":"2023-08-11T14:01:38.454784+0000","last_clean":"2023-08-11T14:01:38.454784+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:01:38.454784+0000","last_undegraded":"2023-08-11T14:01:38.454784+0000","last_fullsized":"2023-08-11T14:01:38.454784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:33.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320128+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:00.320128+0000","last_peered":"2023-08-11T14:02:00.320128+0000","last_clean":"2023-08-11T14:02:00.320128+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:00.320128+0000","last_undegraded":"2023-08-11T14:02:00.320128+0000","last_fullsized":"2023-08-11T14:02:00.320128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:50.342786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453594+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:01:38.453594+0000","last_peered":"2023-08-11T14:01:38.453594+0000","last_clean":"2023-08-11T14:01:38.453594+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:01:38.453594+0000","last_undegraded":"2023-08-11T14:01:38.453594+0000","last_fullsized":"2023-08-11T14:01:38.453594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:05.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453251+0000","last_change":"2023-08-11T14:01:37.586607+0000","last_active":"2023-08-11T14:01:38.453251+0000","last_peered":"2023-08-11T14:01:38.453251+0000","last_clean":"2023-08-11T14:01:38.453251+0000","last_became_active":"2023-08-11T14:01:37.586160+0000","last_became_peered":"2023-08-11T14:01:37.586160+0000","last_unstale":"2023-08-11T14:01:38.453251+0000","last_undegraded":"2023-08-11T14:01:38.453251+0000","last_fullsized":"2023-08-11T14:01:38.453251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:11.941518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293980+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:01:57.293980+0000","last_peered":"2023-08-11T14:01:57.293980+0000","last_clean":"2023-08-11T14:01:57.293980+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:01:57.293980+0000","last_undegraded":"2023-08-11T14:01:57.293980+0000","last_fullsized":"2023-08-11T14:01:57.293980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:59:18.708334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357500+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:01:38.357500+0000","last_peered":"2023-08-11T14:01:38.357500+0000","last_clean":"2023-08-11T14:01:38.357500+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:01:38.357500+0000","last_undegraded":"2023-08-11T14:01:38.357500+0000","last_fullsized":"2023-08-11T14:01:38.357500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:49:09.080233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454487+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:01:38.454487+0000","last_peered":"2023-08-11T14:01:38.454487+0000","last_clean":"2023-08-11T14:01:38.454487+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:01:38.454487+0000","last_undegraded":"2023-08-11T14:01:38.454487+0000","last_fullsized":"2023-08-11T14:01:38.454487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:55:51.552618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322084+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:00.322084+0000","last_peered":"2023-08-11T14:02:00.322084+0000","last_clean":"2023-08-11T14:02:00.322084+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:00.322084+0000","last_undegraded":"2023-08-11T14:02:00.322084+0000","last_fullsized":"2023-08-11T14:02:00.322084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:13:22.765102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453667+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:01:38.453667+0000","last_peered":"2023-08-11T14:01:38.453667+0000","last_clean":"2023-08-11T14:01:38.453667+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:01:38.453667+0000","last_undegraded":"2023-08-11T14:01:38.453667+0000","last_fullsized":"2023-08-11T14:01:38.453667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:10:22.858975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453688+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:01:38.453688+0000","last_peered":"2023-08-11T14:01:38.453688+0000","last_clean":"2023-08-11T14:01:38.453688+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:01:38.453688+0000","last_undegraded":"2023-08-11T14:01:38.453688+0000","last_fullsized":"2023-08-11T14:01:38.453688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:18.721663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.470170+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:01:58.470170+0000","last_peered":"2023-08-11T14:01:58.470170+0000","last_clean":"2023-08-11T14:01:58.470170+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:01:58.470170+0000","last_undegraded":"2023-08-11T14:01:58.470170+0000","last_fullsized":"2023-08-11T14:01:58.470170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:54:36.626882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453866+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:01:38.453866+0000","last_peered":"2023-08-11T14:01:38.453866+0000","last_clean":"2023-08-11T14:01:38.453866+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:01:38.453866+0000","last_undegraded":"2023-08-11T14:01:38.453866+0000","last_fullsized":"2023-08-11T14:01:38.453866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:57.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453150+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:01:38.453150+0000","last_peered":"2023-08-11T14:01:38.453150+0000","last_clean":"2023-08-11T14:01:38.453150+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:01:38.453150+0000","last_undegraded":"2023-08-11T14:01:38.453150+0000","last_fullsized":"2023-08-11T14:01:38.453150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:41.914296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.294819+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:01:57.294819+0000","last_peered":"2023-08-11T14:01:57.294819+0000","last_clean":"2023-08-11T14:01:57.294819+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:01:57.294819+0000","last_undegraded":"2023-08-11T14:01:57.294819+0000","last_fullsized":"2023-08-11T14:01:57.294819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:25.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453769+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:01:38.453769+0000","last_peered":"2023-08-11T14:01:38.453769+0000","last_clean":"2023-08-11T14:01:38.453769+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:01:38.453769+0000","last_undegraded":"2023-08-11T14:01:38.453769+0000","last_fullsized":"2023-08-11T14:01:38.453769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:24.669978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357592+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:01:38.357592+0000","last_peered":"2023-08-11T14:01:38.357592+0000","last_clean":"2023-08-11T14:01:38.357592+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:01:38.357592+0000","last_undegraded":"2023-08-11T14:01:38.357592+0000","last_fullsized":"2023-08-11T14:01:38.357592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:22.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312575+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:01:59.312575+0000","last_peered":"2023-08-11T14:01:59.312575+0000","last_clean":"2023-08-11T14:01:59.312575+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:01:59.312575+0000","last_undegraded":"2023-08-11T14:01:59.312575+0000","last_fullsized":"2023-08-11T14:01:59.312575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:59.189153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454907+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:01:38.454907+0000","last_peered":"2023-08-11T14:01:38.454907+0000","last_clean":"2023-08-11T14:01:38.454907+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:01:38.454907+0000","last_undegraded":"2023-08-11T14:01:38.454907+0000","last_fullsized":"2023-08-11T14:01:38.454907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:01.579515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453184+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:01:38.453184+0000","last_peered":"2023-08-11T14:01:38.453184+0000","last_clean":"2023-08-11T14:01:38.453184+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:01:38.453184+0000","last_undegraded":"2023-08-11T14:01:38.453184+0000","last_fullsized":"2023-08-11T14:01:38.453184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:17:44.247485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320548+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:00.320548+0000","last_peered":"2023-08-11T14:02:00.320548+0000","last_clean":"2023-08-11T14:02:00.320548+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:00.320548+0000","last_undegraded":"2023-08-11T14:02:00.320548+0000","last_fullsized":"2023-08-11T14:02:00.320548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:22.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453958+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:01:38.453958+0000","last_peered":"2023-08-11T14:01:38.453958+0000","last_clean":"2023-08-11T14:01:38.453958+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:01:38.453958+0000","last_undegraded":"2023-08-11T14:01:38.453958+0000","last_fullsized":"2023-08-11T14:01:38.453958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:03.238481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453996+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:01:38.453996+0000","last_peered":"2023-08-11T14:01:38.453996+0000","last_clean":"2023-08-11T14:01:38.453996+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:01:38.453996+0000","last_undegraded":"2023-08-11T14:01:38.453996+0000","last_fullsized":"2023-08-11T14:01:38.453996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:16:23.427336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358000+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:01:38.358000+0000","last_peered":"2023-08-11T14:01:38.358000+0000","last_clean":"2023-08-11T14:01:38.358000+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:01:38.358000+0000","last_undegraded":"2023-08-11T14:01:38.358000+0000","last_fullsized":"2023-08-11T14:01:38.358000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:54.195587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454315+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:01:38.454315+0000","last_peered":"2023-08-11T14:01:38.454315+0000","last_clean":"2023-08-11T14:01:38.454315+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:01:38.454315+0000","last_undegraded":"2023-08-11T14:01:38.454315+0000","last_fullsized":"2023-08-11T14:01:38.454315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:23.010885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453705+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:01:38.453705+0000","last_peered":"2023-08-11T14:01:38.453705+0000","last_clean":"2023-08-11T14:01:38.453705+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:01:38.453705+0000","last_undegraded":"2023-08-11T14:01:38.453705+0000","last_fullsized":"2023-08-11T14:01:38.453705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:29.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455259+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:01:38.455259+0000","last_peered":"2023-08-11T14:01:38.455259+0000","last_clean":"2023-08-11T14:01:38.455259+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:01:38.455259+0000","last_undegraded":"2023-08-11T14:01:38.455259+0000","last_fullsized":"2023-08-11T14:01:38.455259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:25:44.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454215+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:01:38.454215+0000","last_peered":"2023-08-11T14:01:38.454215+0000","last_clean":"2023-08-11T14:01:38.454215+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:01:38.454215+0000","last_undegraded":"2023-08-11T14:01:38.454215+0000","last_fullsized":"2023-08-11T14:01:38.454215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:25.219837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453473+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:01:38.453473+0000","last_peered":"2023-08-11T14:01:38.453473+0000","last_clean":"2023-08-11T14:01:38.453473+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:01:38.453473+0000","last_undegraded":"2023-08-11T14:01:38.453473+0000","last_fullsized":"2023-08-11T14:01:38.453473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:09:10.549528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357742+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:01:38.357742+0000","last_peered":"2023-08-11T14:01:38.357742+0000","last_clean":"2023-08-11T14:01:38.357742+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:01:38.357742+0000","last_undegraded":"2023-08-11T14:01:38.357742+0000","last_fullsized":"2023-08-11T14:01:38.357742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:33:26.233928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454286+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:01:38.454286+0000","last_peered":"2023-08-11T14:01:38.454286+0000","last_clean":"2023-08-11T14:01:38.454286+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:01:38.454286+0000","last_undegraded":"2023-08-11T14:01:38.454286+0000","last_fullsized":"2023-08-11T14:01:38.454286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:09.697084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454819+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:01:38.454819+0000","last_peered":"2023-08-11T14:01:38.454819+0000","last_clean":"2023-08-11T14:01:38.454819+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:01:38.454819+0000","last_undegraded":"2023-08-11T14:01:38.454819+0000","last_fullsized":"2023-08-11T14:01:38.454819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:08:07.050432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455253+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:01:38.455253+0000","last_peered":"2023-08-11T14:01:38.455253+0000","last_clean":"2023-08-11T14:01:38.455253+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:01:38.455253+0000","last_undegraded":"2023-08-11T14:01:38.455253+0000","last_fullsized":"2023-08-11T14:01:38.455253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:01:53.038590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455190+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:01:38.455190+0000","last_peered":"2023-08-11T14:01:38.455190+0000","last_clean":"2023-08-11T14:01:38.455190+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:01:38.455190+0000","last_undegraded":"2023-08-11T14:01:38.455190+0000","last_fullsized":"2023-08-11T14:01:38.455190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:01:59.437947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454868+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:01:38.454868+0000","last_peered":"2023-08-11T14:01:38.454868+0000","last_clean":"2023-08-11T14:01:38.454868+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:01:38.454868+0000","last_undegraded":"2023-08-11T14:01:38.454868+0000","last_fullsized":"2023-08-11T14:01:38.454868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:30.935593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357848+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:01:38.357848+0000","last_peered":"2023-08-11T14:01:38.357848+0000","last_clean":"2023-08-11T14:01:38.357848+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:01:38.357848+0000","last_undegraded":"2023-08-11T14:01:38.357848+0000","last_fullsized":"2023-08-11T14:01:38.357848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:11.005735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357783+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:01:38.357783+0000","last_peered":"2023-08-11T14:01:38.357783+0000","last_clean":"2023-08-11T14:01:38.357783+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:01:38.357783+0000","last_undegraded":"2023-08-11T14:01:38.357783+0000","last_fullsized":"2023-08-11T14:01:38.357783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:32.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453988+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:01:38.453988+0000","last_peered":"2023-08-11T14:01:38.453988+0000","last_clean":"2023-08-11T14:01:38.453988+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:01:38.453988+0000","last_undegraded":"2023-08-11T14:01:38.453988+0000","last_fullsized":"2023-08-11T14:01:38.453988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:53.361416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9864,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":25,"num_read_kb":23,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":28404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.754}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1061588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.504}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.39100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72899999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841562,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":466584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.624}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.745}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.49299999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.751},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.96999999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.94399999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364834,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1057687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.498}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.622},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":465890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":469369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.84199999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.89100000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.75},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65800000000000003}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.872}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":471614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.876},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.439}]},{"osd":6,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":466208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.86899999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.745}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":470986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.439}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79900000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1063084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:03.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:03.337 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: 2023-08-11T14:02:03.020+0000 7f80c8aa2700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-08-11T14:02:03.338 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: 2023-08-11T14:02:03.020+0000 7f80c8aa2700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-08-11T14:02:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 -- 172.21.15.3:0/1869857096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c102b30 msgr2=0x7f5f2c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1869857096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c102b30 0x7f5f2c102f50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f280099c0 tx=0x7f5f28009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 -- 172.21.15.3:0/1869857096 shutdown_connections 2023-08-11T14:02:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1869857096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c1046c0 0x7f5f2c10af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1869857096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c102b30 0x7f5f2c102f50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1869857096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c103d20 0x7f5f2c104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 -- 172.21.15.3:0/1869857096 >> 172.21.15.3:0/1869857096 conn(0x7f5f2c0fe0a0 msgr2=0x7f5f2c1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 -- 172.21.15.3:0/1869857096 shutdown_connections 2023-08-11T14:02:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.539+0000 7f5f345ff700 1 -- 172.21.15.3:0/1869857096 wait complete. 2023-08-11T14:02:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-08-11T14:02:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T14:02:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T14:02:03.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-08-11T14:02:03.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T14:02:03.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T14:02:03.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3829656123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:03 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3062430468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.540+0000 7f5f345ff700 1 Processor -- start 2023-08-11T14:02:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 -- start start 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c102b30 0x7f5f2c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c1046c0 0x7f5f2c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1a0350 con 0x7f5f2c103d20 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1a0490 con 0x7f5f2c1046c0 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f345ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1a0620 con 0x7f5f2c102b30 2023-08-11T14:02:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f31b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f31b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41278/0 (socket says 172.21.15.3:41278) 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f31b9a700 1 -- 172.21.15.3:0/1987916933 learned_addr learned my addr 172.21.15.3:0/1987916933 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f3239b700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c102b30 0x7f5f2c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.541+0000 7f5f32b9c700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c1046c0 0x7f5f2c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 -- 172.21.15.3:0/1987916933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c102b30 msgr2=0x7f5f2c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c102b30 0x7f5f2c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 -- 172.21.15.3:0/1987916933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c1046c0 msgr2=0x7f5f2c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c1046c0 0x7f5f2c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 -- 172.21.15.3:0/1987916933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280095f0 con 0x7f5f2c103d20 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f31b9a700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d980 tx=0x7f5f1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004220 con 0x7f5f2c103d20 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f1c007640 con 0x7f5f2c103d20 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1a0900 con 0x7f5f2c103d20 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c005690 con 0x7f5f2c103d20 2023-08-11T14:02:03.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.542+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1a0e10 con 0x7f5f2c103d20 2023-08-11T14:02:03.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.543+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c109a10 con 0x7f5f2c103d20 2023-08-11T14:02:03.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.545+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f5f1c004750 con 0x7f5f2c103d20 2023-08-11T14:02:03.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.546+0000 7f5f237fe700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 0x7f5f180a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:03.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.547+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5f1c0cb500 con 0x7f5f2c103d20 2023-08-11T14:02:03.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.547+0000 7f5f3239b700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 0x7f5f180a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:03.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.547+0000 7f5f3239b700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 0x7f5f180a24e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009990 tx=0x7f5f28005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:03.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.550+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5f1c094e30 con 0x7f5f2c103d20 2023-08-11T14:02:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-08-11T14:02:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3829656123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:03 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3062430468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.756+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f2c02ce70 con 0x7f5f180a0020 2023-08-11T14:02:03.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.769+0000 7f5f237fe700 1 -- 172.21.15.3:0/1987916933 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422630 (secure 0 0 0) 0x7f5f2c02ce70 con 0x7f5f180a0020 2023-08-11T14:02:03.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 msgr2=0x7f5f180a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 0x7f5f180a24e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009990 tx=0x7f5f28005fb0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 msgr2=0x7f5f2c198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d980 tx=0x7f5f1c00dc90 comp rx=0 tx=0).stop 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 shutdown_connections 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5f180a0020 0x7f5f180a24e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2c1046c0 0x7f5f2c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f2c102b30 0x7f5f2c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 --2- 172.21.15.3:0/1987916933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f2c103d20 0x7f5f2c198d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 >> 172.21.15.3:0/1987916933 conn(0x7f5f2c0fe0a0 msgr2=0x7f5f2c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 shutdown_connections 2023-08-11T14:02:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:03.774+0000 7f5f345ff700 1 -- 172.21.15.3:0/1987916933 wait complete. 2023-08-11T14:02:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:02:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-08-11T14:02:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-08-11T14:02:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 10]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-08-11T14:02:03.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-08-11T14:02:03.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: osdmap e88: 12 total, 12 up, 12 in 2023-08-11T14:02:03.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-08-11T14:02:03.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3829656123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:03 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3062430468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:03.797 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2023-08-11T14:02:03.329621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600445,"num_objects":88,"num_object_clones":0,"num_object_copies":265,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":154,"num_read_kb":135,"num_write":366,"num_write_kb":2999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337520,"kb_used_data":13324,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604304,"statfs":{"total":1151940427776,"available":1151594807296,"internally_reserved":0,"allocated":13643776,"data_stored":7413278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5578,"num_objects":64,"num_object_clones":0,"num_object_copies":193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":28,"num_read_kb":26,"num_write":137,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.165572"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454489+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:01:38.454489+0000","last_peered":"2023-08-11T14:01:38.454489+0000","last_clean":"2023-08-11T14:01:38.454489+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:01:38.454489+0000","last_undegraded":"2023-08-11T14:01:38.454489+0000","last_fullsized":"2023-08-11T14:01:38.454489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:15.565502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453396+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:01:38.453396+0000","last_peered":"2023-08-11T14:01:38.453396+0000","last_clean":"2023-08-11T14:01:38.453396+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:01:38.453396+0000","last_undegraded":"2023-08-11T14:01:38.453396+0000","last_fullsized":"2023-08-11T14:01:38.453396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:05.487390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453350+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:01:38.453350+0000","last_peered":"2023-08-11T14:01:38.453350+0000","last_clean":"2023-08-11T14:01:38.453350+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:01:38.453350+0000","last_undegraded":"2023-08-11T14:01:38.453350+0000","last_fullsized":"2023-08-11T14:01:38.453350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:32:07.002359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453773+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:01:38.453773+0000","last_peered":"2023-08-11T14:01:38.453773+0000","last_clean":"2023-08-11T14:01:38.453773+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:01:38.453773+0000","last_undegraded":"2023-08-11T14:01:38.453773+0000","last_fullsized":"2023-08-11T14:01:38.453773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:24.933688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357698+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:01:38.357698+0000","last_peered":"2023-08-11T14:01:38.357698+0000","last_clean":"2023-08-11T14:01:38.357698+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:01:38.357698+0000","last_undegraded":"2023-08-11T14:01:38.357698+0000","last_fullsized":"2023-08-11T14:01:38.357698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:30:43.465070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454651+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:01:38.454651+0000","last_peered":"2023-08-11T14:01:38.454651+0000","last_clean":"2023-08-11T14:01:38.454651+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:01:38.454651+0000","last_undegraded":"2023-08-11T14:01:38.454651+0000","last_fullsized":"2023-08-11T14:01:38.454651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:11:08.034766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453427+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:01:38.453427+0000","last_peered":"2023-08-11T14:01:38.453427+0000","last_clean":"2023-08-11T14:01:38.453427+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:01:38.453427+0000","last_undegraded":"2023-08-11T14:01:38.453427+0000","last_fullsized":"2023-08-11T14:01:38.453427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:45:09.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:38.454237+0000","last_change":"2023-08-11T14:02:03.242225+0000","last_active":"2023-08-11T14:02:03.242225+0000","last_peered":"2023-08-11T14:01:38.454237+0000","last_clean":"2023-08-11T14:01:38.454237+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454237+0000","last_undegraded":"2023-08-11T14:01:38.454237+0000","last_fullsized":"2023-08-11T14:01:38.454237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358208+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:01:38.358208+0000","last_peered":"2023-08-11T14:01:38.358208+0000","last_clean":"2023-08-11T14:01:38.358208+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:01:38.358208+0000","last_undegraded":"2023-08-11T14:01:38.358208+0000","last_fullsized":"2023-08-11T14:01:38.358208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:01:30.807708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455163+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:01:38.455163+0000","last_peered":"2023-08-11T14:01:38.455163+0000","last_clean":"2023-08-11T14:01:38.455163+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:01:38.455163+0000","last_undegraded":"2023-08-11T14:01:38.455163+0000","last_fullsized":"2023-08-11T14:01:38.455163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:18:15.132967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454608+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:01:38.454608+0000","last_peered":"2023-08-11T14:01:38.454608+0000","last_clean":"2023-08-11T14:01:38.454608+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:01:38.454608+0000","last_undegraded":"2023-08-11T14:01:38.454608+0000","last_fullsized":"2023-08-11T14:01:38.454608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:36.691099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453612+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:01:38.453612+0000","last_peered":"2023-08-11T14:01:38.453612+0000","last_clean":"2023-08-11T14:01:38.453612+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:01:38.453612+0000","last_undegraded":"2023-08-11T14:01:38.453612+0000","last_fullsized":"2023-08-11T14:01:38.453612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:41.734924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357734+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:01:38.357734+0000","last_peered":"2023-08-11T14:01:38.357734+0000","last_clean":"2023-08-11T14:01:38.357734+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:01:38.357734+0000","last_undegraded":"2023-08-11T14:01:38.357734+0000","last_fullsized":"2023-08-11T14:01:38.357734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:49:21.624209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453057+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:01:38.453057+0000","last_peered":"2023-08-11T14:01:38.453057+0000","last_clean":"2023-08-11T14:01:38.453057+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:01:38.453057+0000","last_undegraded":"2023-08-11T14:01:38.453057+0000","last_fullsized":"2023-08-11T14:01:38.453057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:54:14.411087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453329+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:01:38.453329+0000","last_peered":"2023-08-11T14:01:38.453329+0000","last_clean":"2023-08-11T14:01:38.453329+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:01:38.453329+0000","last_undegraded":"2023-08-11T14:01:38.453329+0000","last_fullsized":"2023-08-11T14:01:38.453329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:45.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357450+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:01:38.357450+0000","last_peered":"2023-08-11T14:01:38.357450+0000","last_clean":"2023-08-11T14:01:38.357450+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:01:38.357450+0000","last_undegraded":"2023-08-11T14:01:38.357450+0000","last_fullsized":"2023-08-11T14:01:38.357450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:25:49.542397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454690+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:01:38.454690+0000","last_peered":"2023-08-11T14:01:38.454690+0000","last_clean":"2023-08-11T14:01:38.454690+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:01:38.454690+0000","last_undegraded":"2023-08-11T14:01:38.454690+0000","last_fullsized":"2023-08-11T14:01:38.454690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:06:02.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345350+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:02.345350+0000","last_peered":"2023-08-11T14:02:02.345350+0000","last_clean":"2023-08-11T14:02:02.345350+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:02.345350+0000","last_undegraded":"2023-08-11T14:02:02.345350+0000","last_fullsized":"2023-08-11T14:02:02.345350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:08:36.944921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454406+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:01:38.454406+0000","last_peered":"2023-08-11T14:01:38.454406+0000","last_clean":"2023-08-11T14:01:38.454406+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:01:38.454406+0000","last_undegraded":"2023-08-11T14:01:38.454406+0000","last_fullsized":"2023-08-11T14:01:38.454406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:02:50.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455090+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:01:38.455090+0000","last_peered":"2023-08-11T14:01:38.455090+0000","last_clean":"2023-08-11T14:01:38.455090+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:01:38.455090+0000","last_undegraded":"2023-08-11T14:01:38.455090+0000","last_fullsized":"2023-08-11T14:01:38.455090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:39.906536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339916+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:02.339916+0000","last_peered":"2023-08-11T14:02:02.339916+0000","last_clean":"2023-08-11T14:02:02.339916+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:02.339916+0000","last_undegraded":"2023-08-11T14:02:02.339916+0000","last_fullsized":"2023-08-11T14:02:02.339916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:13:11.710971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454673+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:01:38.454673+0000","last_peered":"2023-08-11T14:01:38.454673+0000","last_clean":"2023-08-11T14:01:38.454673+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:01:38.454673+0000","last_undegraded":"2023-08-11T14:01:38.454673+0000","last_fullsized":"2023-08-11T14:01:38.454673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:09:31.987246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453033+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:01:38.453033+0000","last_peered":"2023-08-11T14:01:38.453033+0000","last_clean":"2023-08-11T14:01:38.453033+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:01:38.453033+0000","last_undegraded":"2023-08-11T14:01:38.453033+0000","last_fullsized":"2023-08-11T14:01:38.453033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:24:09.010833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454851+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:01:38.454851+0000","last_peered":"2023-08-11T14:01:38.454851+0000","last_clean":"2023-08-11T14:01:38.454851+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:01:38.454851+0000","last_undegraded":"2023-08-11T14:01:38.454851+0000","last_fullsized":"2023-08-11T14:01:38.454851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:09.972413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453569+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:01:38.453569+0000","last_peered":"2023-08-11T14:01:38.453569+0000","last_clean":"2023-08-11T14:01:38.453569+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:01:38.453569+0000","last_undegraded":"2023-08-11T14:01:38.453569+0000","last_fullsized":"2023-08-11T14:01:38.453569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:31.879999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453610+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:01:38.453610+0000","last_peered":"2023-08-11T14:01:38.453610+0000","last_clean":"2023-08-11T14:01:38.453610+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:01:38.453610+0000","last_undegraded":"2023-08-11T14:01:38.453610+0000","last_fullsized":"2023-08-11T14:01:38.453610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:00.077438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322139+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:00.322139+0000","last_peered":"2023-08-11T14:02:00.322139+0000","last_clean":"2023-08-11T14:02:00.322139+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:00.322139+0000","last_undegraded":"2023-08-11T14:02:00.322139+0000","last_fullsized":"2023-08-11T14:02:00.322139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:53.482576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357683+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:01:38.357683+0000","last_peered":"2023-08-11T14:01:38.357683+0000","last_clean":"2023-08-11T14:01:38.357683+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:01:38.357683+0000","last_undegraded":"2023-08-11T14:01:38.357683+0000","last_fullsized":"2023-08-11T14:01:38.357683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:45:28.652581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.452997+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:01:38.452997+0000","last_peered":"2023-08-11T14:01:38.452997+0000","last_clean":"2023-08-11T14:01:38.452997+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:01:38.452997+0000","last_undegraded":"2023-08-11T14:01:38.452997+0000","last_fullsized":"2023-08-11T14:01:38.452997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:46.488285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320442+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:00.320442+0000","last_peered":"2023-08-11T14:02:00.320442+0000","last_clean":"2023-08-11T14:02:00.320442+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:00.320442+0000","last_undegraded":"2023-08-11T14:02:00.320442+0000","last_fullsized":"2023-08-11T14:02:00.320442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:37:57.485057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455704+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:01:38.455704+0000","last_peered":"2023-08-11T14:01:38.455704+0000","last_clean":"2023-08-11T14:01:38.455704+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:01:38.455704+0000","last_undegraded":"2023-08-11T14:01:38.455704+0000","last_fullsized":"2023-08-11T14:01:38.455704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:25.911259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454143+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:01:38.454143+0000","last_peered":"2023-08-11T14:01:38.454143+0000","last_clean":"2023-08-11T14:01:38.454143+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:01:38.454143+0000","last_undegraded":"2023-08-11T14:01:38.454143+0000","last_fullsized":"2023-08-11T14:01:38.454143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:00.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321984+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:01.321984+0000","last_peered":"2023-08-11T14:02:01.321984+0000","last_clean":"2023-08-11T14:02:01.321984+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:01.321984+0000","last_undegraded":"2023-08-11T14:02:01.321984+0000","last_fullsized":"2023-08-11T14:02:01.321984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:10.771366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357613+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:01:38.357613+0000","last_peered":"2023-08-11T14:01:38.357613+0000","last_clean":"2023-08-11T14:01:38.357613+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:01:38.357613+0000","last_undegraded":"2023-08-11T14:01:38.357613+0000","last_fullsized":"2023-08-11T14:01:38.357613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:46:18.787688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454174+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:01:38.454174+0000","last_peered":"2023-08-11T14:01:38.454174+0000","last_clean":"2023-08-11T14:01:38.454174+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:01:38.454174+0000","last_undegraded":"2023-08-11T14:01:38.454174+0000","last_fullsized":"2023-08-11T14:01:38.454174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:21.945068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305879+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:01:58.305879+0000","last_peered":"2023-08-11T14:01:58.305879+0000","last_clean":"2023-08-11T14:01:58.305879+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:01:58.305879+0000","last_undegraded":"2023-08-11T14:01:58.305879+0000","last_fullsized":"2023-08-11T14:01:58.305879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:33.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455026+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:01:38.455026+0000","last_peered":"2023-08-11T14:01:38.455026+0000","last_clean":"2023-08-11T14:01:38.455026+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:01:38.455026+0000","last_undegraded":"2023-08-11T14:01:38.455026+0000","last_fullsized":"2023-08-11T14:01:38.455026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:26:04.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454393+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:01:38.454393+0000","last_peered":"2023-08-11T14:01:38.454393+0000","last_clean":"2023-08-11T14:01:38.454393+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:01:38.454393+0000","last_undegraded":"2023-08-11T14:01:38.454393+0000","last_fullsized":"2023-08-11T14:01:38.454393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:09.108932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.307340+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:01:58.307340+0000","last_peered":"2023-08-11T14:01:58.307340+0000","last_clean":"2023-08-11T14:01:58.307340+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:01:58.307340+0000","last_undegraded":"2023-08-11T14:01:58.307340+0000","last_fullsized":"2023-08-11T14:01:58.307340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:28.668048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454198+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:01:38.454198+0000","last_peered":"2023-08-11T14:01:38.454198+0000","last_clean":"2023-08-11T14:01:38.454198+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:01:38.454198+0000","last_undegraded":"2023-08-11T14:01:38.454198+0000","last_fullsized":"2023-08-11T14:01:38.454198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:02.471143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453242+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:01:38.453242+0000","last_peered":"2023-08-11T14:01:38.453242+0000","last_clean":"2023-08-11T14:01:38.453242+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:01:38.453242+0000","last_undegraded":"2023-08-11T14:01:38.453242+0000","last_fullsized":"2023-08-11T14:01:38.453242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:21:10.060538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321844+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:00.321844+0000","last_peered":"2023-08-11T14:02:00.321844+0000","last_clean":"2023-08-11T14:02:00.321844+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:00.321844+0000","last_undegraded":"2023-08-11T14:02:00.321844+0000","last_fullsized":"2023-08-11T14:02:00.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:29.646533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339573+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:02.339573+0000","last_peered":"2023-08-11T14:02:02.339573+0000","last_clean":"2023-08-11T14:02:02.339573+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:02.339573+0000","last_undegraded":"2023-08-11T14:02:02.339573+0000","last_fullsized":"2023-08-11T14:02:02.339573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:12.537664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454637+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:01:38.454637+0000","last_peered":"2023-08-11T14:01:38.454637+0000","last_clean":"2023-08-11T14:01:38.454637+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:01:38.454637+0000","last_undegraded":"2023-08-11T14:01:38.454637+0000","last_fullsized":"2023-08-11T14:01:38.454637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:14:38.485351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.292369+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:01:57.292369+0000","last_peered":"2023-08-11T14:01:57.292369+0000","last_clean":"2023-08-11T14:01:57.292369+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:01:57.292369+0000","last_undegraded":"2023-08-11T14:01:57.292369+0000","last_fullsized":"2023-08-11T14:01:57.292369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:34:53.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358143+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:01:38.358143+0000","last_peered":"2023-08-11T14:01:38.358143+0000","last_clean":"2023-08-11T14:01:38.358143+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:01:38.358143+0000","last_undegraded":"2023-08-11T14:01:38.358143+0000","last_fullsized":"2023-08-11T14:01:38.358143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:56.024569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340313+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:02.340313+0000","last_peered":"2023-08-11T14:02:02.340313+0000","last_clean":"2023-08-11T14:02:02.340313+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:02.340313+0000","last_undegraded":"2023-08-11T14:02:02.340313+0000","last_fullsized":"2023-08-11T14:02:02.340313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:37:35.917226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293732+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:01:57.293732+0000","last_peered":"2023-08-11T14:01:57.293732+0000","last_clean":"2023-08-11T14:01:57.293732+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:01:57.293732+0000","last_undegraded":"2023-08-11T14:01:57.293732+0000","last_fullsized":"2023-08-11T14:01:57.293732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:00.643436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453798+0000","last_change":"2023-08-11T14:01:33.308455+0000","last_active":"2023-08-11T14:01:38.453798+0000","last_peered":"2023-08-11T14:01:38.453798+0000","last_clean":"2023-08-11T14:01:38.453798+0000","last_became_active":"2023-08-11T14:01:33.308084+0000","last_became_peered":"2023-08-11T14:01:33.308084+0000","last_unstale":"2023-08-11T14:01:38.453798+0000","last_undegraded":"2023-08-11T14:01:38.453798+0000","last_fullsized":"2023-08-11T14:01:38.453798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:33.705786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453779+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:01:38.453779+0000","last_peered":"2023-08-11T14:01:38.453779+0000","last_clean":"2023-08-11T14:01:38.453779+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:01:38.453779+0000","last_undegraded":"2023-08-11T14:01:38.453779+0000","last_fullsized":"2023-08-11T14:01:38.453779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:41.727745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312606+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:01:59.312606+0000","last_peered":"2023-08-11T14:01:59.312606+0000","last_clean":"2023-08-11T14:01:59.312606+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:01:59.312606+0000","last_undegraded":"2023-08-11T14:01:59.312606+0000","last_fullsized":"2023-08-11T14:01:59.312606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:33:25.490224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357940+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:01:38.357940+0000","last_peered":"2023-08-11T14:01:38.357940+0000","last_clean":"2023-08-11T14:01:38.357940+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:01:38.357940+0000","last_undegraded":"2023-08-11T14:01:38.357940+0000","last_fullsized":"2023-08-11T14:01:38.357940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:05:22.390481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340016+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:02.340016+0000","last_peered":"2023-08-11T14:02:02.340016+0000","last_clean":"2023-08-11T14:02:02.340016+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:02.340016+0000","last_undegraded":"2023-08-11T14:02:02.340016+0000","last_fullsized":"2023-08-11T14:02:02.340016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:07.686519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.295025+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:01:57.295025+0000","last_peered":"2023-08-11T14:01:57.295025+0000","last_clean":"2023-08-11T14:01:57.295025+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:01:57.295025+0000","last_undegraded":"2023-08-11T14:01:57.295025+0000","last_fullsized":"2023-08-11T14:01:57.295025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:38:00.566235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357914+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:01:38.357914+0000","last_peered":"2023-08-11T14:01:38.357914+0000","last_clean":"2023-08-11T14:01:38.357914+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:01:38.357914+0000","last_undegraded":"2023-08-11T14:01:38.357914+0000","last_fullsized":"2023-08-11T14:01:38.357914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:21:16.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455907+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:01:38.455907+0000","last_peered":"2023-08-11T14:01:38.455907+0000","last_clean":"2023-08-11T14:01:38.455907+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:01:38.455907+0000","last_undegraded":"2023-08-11T14:01:38.455907+0000","last_fullsized":"2023-08-11T14:01:38.455907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:56.419641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305981+0000","last_change":"2023-08-11T14:01:32.306792+0000","last_active":"2023-08-11T14:01:58.305981+0000","last_peered":"2023-08-11T14:01:58.305981+0000","last_clean":"2023-08-11T14:01:58.305981+0000","last_became_active":"2023-08-11T14:01:32.306447+0000","last_became_peered":"2023-08-11T14:01:32.306447+0000","last_unstale":"2023-08-11T14:01:58.305981+0000","last_undegraded":"2023-08-11T14:01:58.305981+0000","last_fullsized":"2023-08-11T14:01:58.305981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:09.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357607+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:01:38.357607+0000","last_peered":"2023-08-11T14:01:38.357607+0000","last_clean":"2023-08-11T14:01:38.357607+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:01:38.357607+0000","last_undegraded":"2023-08-11T14:01:38.357607+0000","last_fullsized":"2023-08-11T14:01:38.357607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:58:53.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453894+0000","last_change":"2023-08-11T14:01:37.375702+0000","last_active":"2023-08-11T14:01:38.453894+0000","last_peered":"2023-08-11T14:01:38.453894+0000","last_clean":"2023-08-11T14:01:38.453894+0000","last_became_active":"2023-08-11T14:01:37.375199+0000","last_became_peered":"2023-08-11T14:01:37.375199+0000","last_unstale":"2023-08-11T14:01:38.453894+0000","last_undegraded":"2023-08-11T14:01:38.453894+0000","last_fullsized":"2023-08-11T14:01:38.453894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:14.150791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321941+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:00.321941+0000","last_peered":"2023-08-11T14:02:00.321941+0000","last_clean":"2023-08-11T14:02:00.321941+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:00.321941+0000","last_undegraded":"2023-08-11T14:02:00.321941+0000","last_fullsized":"2023-08-11T14:02:00.321941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:02:07.982583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453519+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:01:38.453519+0000","last_peered":"2023-08-11T14:01:38.453519+0000","last_clean":"2023-08-11T14:01:38.453519+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:01:38.453519+0000","last_undegraded":"2023-08-11T14:01:38.453519+0000","last_fullsized":"2023-08-11T14:01:38.453519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:57.895242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454218+0000","last_change":"2023-08-11T14:01:37.377532+0000","last_active":"2023-08-11T14:01:38.454218+0000","last_peered":"2023-08-11T14:01:38.454218+0000","last_clean":"2023-08-11T14:01:38.454218+0000","last_became_active":"2023-08-11T14:01:37.377142+0000","last_became_peered":"2023-08-11T14:01:37.377142+0000","last_unstale":"2023-08-11T14:01:38.454218+0000","last_undegraded":"2023-08-11T14:01:38.454218+0000","last_fullsized":"2023-08-11T14:01:38.454218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:13.506622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321844+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:01.321844+0000","last_peered":"2023-08-11T14:02:01.321844+0000","last_clean":"2023-08-11T14:02:01.321844+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:01.321844+0000","last_undegraded":"2023-08-11T14:02:01.321844+0000","last_fullsized":"2023-08-11T14:02:01.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:27.821349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453776+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:01:38.453776+0000","last_peered":"2023-08-11T14:01:38.453776+0000","last_clean":"2023-08-11T14:01:38.453776+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:01:38.453776+0000","last_undegraded":"2023-08-11T14:01:38.453776+0000","last_fullsized":"2023-08-11T14:01:38.453776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:13:35.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357992+0000","last_change":"2023-08-11T14:01:37.381723+0000","last_active":"2023-08-11T14:01:38.357992+0000","last_peered":"2023-08-11T14:01:38.357992+0000","last_clean":"2023-08-11T14:01:38.357992+0000","last_became_active":"2023-08-11T14:01:37.381306+0000","last_became_peered":"2023-08-11T14:01:37.381306+0000","last_unstale":"2023-08-11T14:01:38.357992+0000","last_undegraded":"2023-08-11T14:01:38.357992+0000","last_fullsized":"2023-08-11T14:01:38.357992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:55:27.079422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.318041+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:00.318041+0000","last_peered":"2023-08-11T14:02:00.318041+0000","last_clean":"2023-08-11T14:02:00.318041+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:00.318041+0000","last_undegraded":"2023-08-11T14:02:00.318041+0000","last_fullsized":"2023-08-11T14:02:00.318041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:33:54.507816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:33.290559+0000","last_change":"2023-08-11T14:02:03.242268+0000","last_active":"2023-08-11T14:01:32.273793+0000","last_peered":"2023-08-11T14:01:32.273793+0000","last_clean":"2023-08-11T14:01:32.273793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-11T14:01:33.290559+0000","last_undegraded":"2023-08-11T14:01:33.290559+0000","last_fullsized":"2023-08-11T14:01:33.290559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454385+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:01:38.454385+0000","last_peered":"2023-08-11T14:01:38.454385+0000","last_clean":"2023-08-11T14:01:38.454385+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:01:38.454385+0000","last_undegraded":"2023-08-11T14:01:38.454385+0000","last_fullsized":"2023-08-11T14:01:38.454385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:21.288285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.230701+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:01.230701+0000","last_peered":"2023-08-11T14:02:01.230701+0000","last_clean":"2023-08-11T14:02:01.230701+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:01.230701+0000","last_undegraded":"2023-08-11T14:02:01.230701+0000","last_fullsized":"2023-08-11T14:02:01.230701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:14.365596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453956+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:01:38.453956+0000","last_peered":"2023-08-11T14:01:38.453956+0000","last_clean":"2023-08-11T14:01:38.453956+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:01:38.453956+0000","last_undegraded":"2023-08-11T14:01:38.453956+0000","last_fullsized":"2023-08-11T14:01:38.453956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:22.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357583+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:01:38.357583+0000","last_peered":"2023-08-11T14:01:38.357583+0000","last_clean":"2023-08-11T14:01:38.357583+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:01:38.357583+0000","last_undegraded":"2023-08-11T14:01:38.357583+0000","last_fullsized":"2023-08-11T14:01:38.357583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:04:46.172105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.338227+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:02.338227+0000","last_peered":"2023-08-11T14:02:02.338227+0000","last_clean":"2023-08-11T14:02:02.338227+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:02.338227+0000","last_undegraded":"2023-08-11T14:02:02.338227+0000","last_fullsized":"2023-08-11T14:02:02.338227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:37.579833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454940+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:01:38.454940+0000","last_peered":"2023-08-11T14:01:38.454940+0000","last_clean":"2023-08-11T14:01:38.454940+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:01:38.454940+0000","last_undegraded":"2023-08-11T14:01:38.454940+0000","last_fullsized":"2023-08-11T14:01:38.454940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:55:34.033468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357752+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:01:38.357752+0000","last_peered":"2023-08-11T14:01:38.357752+0000","last_clean":"2023-08-11T14:01:38.357752+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:01:38.357752+0000","last_undegraded":"2023-08-11T14:01:38.357752+0000","last_fullsized":"2023-08-11T14:01:38.357752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:06:12.528692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453781+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:01:38.453781+0000","last_peered":"2023-08-11T14:01:38.453781+0000","last_clean":"2023-08-11T14:01:38.453781+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:01:38.453781+0000","last_undegraded":"2023-08-11T14:01:38.453781+0000","last_fullsized":"2023-08-11T14:01:38.453781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:37.699260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455015+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:01:38.455015+0000","last_peered":"2023-08-11T14:01:38.455015+0000","last_clean":"2023-08-11T14:01:38.455015+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:01:38.455015+0000","last_undegraded":"2023-08-11T14:01:38.455015+0000","last_fullsized":"2023-08-11T14:01:38.455015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:05.858370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357865+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:01:38.357865+0000","last_peered":"2023-08-11T14:01:38.357865+0000","last_clean":"2023-08-11T14:01:38.357865+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:01:38.357865+0000","last_undegraded":"2023-08-11T14:01:38.357865+0000","last_fullsized":"2023-08-11T14:01:38.357865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:18.092395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455326+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:01:38.455326+0000","last_peered":"2023-08-11T14:01:38.455326+0000","last_clean":"2023-08-11T14:01:38.455326+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:01:38.455326+0000","last_undegraded":"2023-08-11T14:01:38.455326+0000","last_fullsized":"2023-08-11T14:01:38.455326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:57.290710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.308543+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:01:58.308543+0000","last_peered":"2023-08-11T14:01:58.308543+0000","last_clean":"2023-08-11T14:01:58.308543+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:01:58.308543+0000","last_undegraded":"2023-08-11T14:01:58.308543+0000","last_fullsized":"2023-08-11T14:01:58.308543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:33.988051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454372+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:01:38.454372+0000","last_peered":"2023-08-11T14:01:38.454372+0000","last_clean":"2023-08-11T14:01:38.454372+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:01:38.454372+0000","last_undegraded":"2023-08-11T14:01:38.454372+0000","last_fullsized":"2023-08-11T14:01:38.454372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:32:11.305694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454548+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:01:38.454548+0000","last_peered":"2023-08-11T14:01:38.454548+0000","last_clean":"2023-08-11T14:01:38.454548+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:01:38.454548+0000","last_undegraded":"2023-08-11T14:01:38.454548+0000","last_fullsized":"2023-08-11T14:01:38.454548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:38.933163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.319779+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:01.319779+0000","last_peered":"2023-08-11T14:02:01.319779+0000","last_clean":"2023-08-11T14:02:01.319779+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:01.319779+0000","last_undegraded":"2023-08-11T14:02:01.319779+0000","last_fullsized":"2023-08-11T14:02:01.319779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:56:44.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453444+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:01:38.453444+0000","last_peered":"2023-08-11T14:01:38.453444+0000","last_clean":"2023-08-11T14:01:38.453444+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:01:38.453444+0000","last_undegraded":"2023-08-11T14:01:38.453444+0000","last_fullsized":"2023-08-11T14:01:38.453444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:10.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455405+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:01:38.455405+0000","last_peered":"2023-08-11T14:01:38.455405+0000","last_clean":"2023-08-11T14:01:38.455405+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:01:38.455405+0000","last_undegraded":"2023-08-11T14:01:38.455405+0000","last_fullsized":"2023-08-11T14:01:38.455405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:31:57.406400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357235+0000","last_change":"2023-08-11T14:01:33.305155+0000","last_active":"2023-08-11T14:01:38.357235+0000","last_peered":"2023-08-11T14:01:38.357235+0000","last_clean":"2023-08-11T14:01:38.357235+0000","last_became_active":"2023-08-11T14:01:33.304789+0000","last_became_peered":"2023-08-11T14:01:33.304789+0000","last_unstale":"2023-08-11T14:01:38.357235+0000","last_undegraded":"2023-08-11T14:01:38.357235+0000","last_fullsized":"2023-08-11T14:01:38.357235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:54.803726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305859+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:01:58.305859+0000","last_peered":"2023-08-11T14:01:58.305859+0000","last_clean":"2023-08-11T14:01:58.305859+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:01:58.305859+0000","last_undegraded":"2023-08-11T14:01:58.305859+0000","last_fullsized":"2023-08-11T14:01:58.305859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:21:11.143846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454625+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:01:38.454625+0000","last_peered":"2023-08-11T14:01:38.454625+0000","last_clean":"2023-08-11T14:01:38.454625+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:01:38.454625+0000","last_undegraded":"2023-08-11T14:01:38.454625+0000","last_fullsized":"2023-08-11T14:01:38.454625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:40:26.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320780+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:00.320780+0000","last_peered":"2023-08-11T14:02:00.320780+0000","last_clean":"2023-08-11T14:02:00.320780+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:00.320780+0000","last_undegraded":"2023-08-11T14:02:00.320780+0000","last_fullsized":"2023-08-11T14:02:00.320780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:17:38.449276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357848+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:01:38.357848+0000","last_peered":"2023-08-11T14:01:38.357848+0000","last_clean":"2023-08-11T14:01:38.357848+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:01:38.357848+0000","last_undegraded":"2023-08-11T14:01:38.357848+0000","last_fullsized":"2023-08-11T14:01:38.357848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:53:41.329551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454047+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:01:38.454047+0000","last_peered":"2023-08-11T14:01:38.454047+0000","last_clean":"2023-08-11T14:01:38.454047+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:01:38.454047+0000","last_undegraded":"2023-08-11T14:01:38.454047+0000","last_fullsized":"2023-08-11T14:01:38.454047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:47:11.994511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345212+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:02.345212+0000","last_peered":"2023-08-11T14:02:02.345212+0000","last_clean":"2023-08-11T14:02:02.345212+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:02.345212+0000","last_undegraded":"2023-08-11T14:02:02.345212+0000","last_fullsized":"2023-08-11T14:02:02.345212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:23.734051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455078+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:01:38.455078+0000","last_peered":"2023-08-11T14:01:38.455078+0000","last_clean":"2023-08-11T14:01:38.455078+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:01:38.455078+0000","last_undegraded":"2023-08-11T14:01:38.455078+0000","last_fullsized":"2023-08-11T14:01:38.455078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:14:16.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454784+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:01:38.454784+0000","last_peered":"2023-08-11T14:01:38.454784+0000","last_clean":"2023-08-11T14:01:38.454784+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:01:38.454784+0000","last_undegraded":"2023-08-11T14:01:38.454784+0000","last_fullsized":"2023-08-11T14:01:38.454784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:33.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320128+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:00.320128+0000","last_peered":"2023-08-11T14:02:00.320128+0000","last_clean":"2023-08-11T14:02:00.320128+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:00.320128+0000","last_undegraded":"2023-08-11T14:02:00.320128+0000","last_fullsized":"2023-08-11T14:02:00.320128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:50.342786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453594+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:01:38.453594+0000","last_peered":"2023-08-11T14:01:38.453594+0000","last_clean":"2023-08-11T14:01:38.453594+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:01:38.453594+0000","last_undegraded":"2023-08-11T14:01:38.453594+0000","last_fullsized":"2023-08-11T14:01:38.453594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:05.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453251+0000","last_change":"2023-08-11T14:01:37.586607+0000","last_active":"2023-08-11T14:01:38.453251+0000","last_peered":"2023-08-11T14:01:38.453251+0000","last_clean":"2023-08-11T14:01:38.453251+0000","last_became_active":"2023-08-11T14:01:37.586160+0000","last_became_peered":"2023-08-11T14:01:37.586160+0000","last_unstale":"2023-08-11T14:01:38.453251+0000","last_undegraded":"2023-08-11T14:01:38.453251+0000","last_fullsized":"2023-08-11T14:01:38.453251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:11.941518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293980+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:01:57.293980+0000","last_peered":"2023-08-11T14:01:57.293980+0000","last_clean":"2023-08-11T14:01:57.293980+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:01:57.293980+0000","last_undegraded":"2023-08-11T14:01:57.293980+0000","last_fullsized":"2023-08-11T14:01:57.293980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:59:18.708334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357500+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:01:38.357500+0000","last_peered":"2023-08-11T14:01:38.357500+0000","last_clean":"2023-08-11T14:01:38.357500+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:01:38.357500+0000","last_undegraded":"2023-08-11T14:01:38.357500+0000","last_fullsized":"2023-08-11T14:01:38.357500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:49:09.080233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339784+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:02.339784+0000","last_peered":"2023-08-11T14:02:02.339784+0000","last_clean":"2023-08-11T14:02:02.339784+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:02.339784+0000","last_undegraded":"2023-08-11T14:02:02.339784+0000","last_fullsized":"2023-08-11T14:02:02.339784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:41.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322084+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:00.322084+0000","last_peered":"2023-08-11T14:02:00.322084+0000","last_clean":"2023-08-11T14:02:00.322084+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:00.322084+0000","last_undegraded":"2023-08-11T14:02:00.322084+0000","last_fullsized":"2023-08-11T14:02:00.322084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:13:22.765102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453667+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:01:38.453667+0000","last_peered":"2023-08-11T14:01:38.453667+0000","last_clean":"2023-08-11T14:01:38.453667+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:01:38.453667+0000","last_undegraded":"2023-08-11T14:01:38.453667+0000","last_fullsized":"2023-08-11T14:01:38.453667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:10:22.858975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453688+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:01:38.453688+0000","last_peered":"2023-08-11T14:01:38.453688+0000","last_clean":"2023-08-11T14:01:38.453688+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:01:38.453688+0000","last_undegraded":"2023-08-11T14:01:38.453688+0000","last_fullsized":"2023-08-11T14:01:38.453688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:18.721663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.470170+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:01:58.470170+0000","last_peered":"2023-08-11T14:01:58.470170+0000","last_clean":"2023-08-11T14:01:58.470170+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:01:58.470170+0000","last_undegraded":"2023-08-11T14:01:58.470170+0000","last_fullsized":"2023-08-11T14:01:58.470170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:54:36.626882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453866+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:01:38.453866+0000","last_peered":"2023-08-11T14:01:38.453866+0000","last_clean":"2023-08-11T14:01:38.453866+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:01:38.453866+0000","last_undegraded":"2023-08-11T14:01:38.453866+0000","last_fullsized":"2023-08-11T14:01:38.453866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:57.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453150+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:01:38.453150+0000","last_peered":"2023-08-11T14:01:38.453150+0000","last_clean":"2023-08-11T14:01:38.453150+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:01:38.453150+0000","last_undegraded":"2023-08-11T14:01:38.453150+0000","last_fullsized":"2023-08-11T14:01:38.453150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:41.914296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.294819+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:01:57.294819+0000","last_peered":"2023-08-11T14:01:57.294819+0000","last_clean":"2023-08-11T14:01:57.294819+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:01:57.294819+0000","last_undegraded":"2023-08-11T14:01:57.294819+0000","last_fullsized":"2023-08-11T14:01:57.294819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:25.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453769+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:01:38.453769+0000","last_peered":"2023-08-11T14:01:38.453769+0000","last_clean":"2023-08-11T14:01:38.453769+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:01:38.453769+0000","last_undegraded":"2023-08-11T14:01:38.453769+0000","last_fullsized":"2023-08-11T14:01:38.453769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:24.669978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357592+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:01:38.357592+0000","last_peered":"2023-08-11T14:01:38.357592+0000","last_clean":"2023-08-11T14:01:38.357592+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:01:38.357592+0000","last_undegraded":"2023-08-11T14:01:38.357592+0000","last_fullsized":"2023-08-11T14:01:38.357592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:22.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312575+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:01:59.312575+0000","last_peered":"2023-08-11T14:01:59.312575+0000","last_clean":"2023-08-11T14:01:59.312575+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:01:59.312575+0000","last_undegraded":"2023-08-11T14:01:59.312575+0000","last_fullsized":"2023-08-11T14:01:59.312575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:59.189153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454907+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:01:38.454907+0000","last_peered":"2023-08-11T14:01:38.454907+0000","last_clean":"2023-08-11T14:01:38.454907+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:01:38.454907+0000","last_undegraded":"2023-08-11T14:01:38.454907+0000","last_fullsized":"2023-08-11T14:01:38.454907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:01.579515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453184+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:01:38.453184+0000","last_peered":"2023-08-11T14:01:38.453184+0000","last_clean":"2023-08-11T14:01:38.453184+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:01:38.453184+0000","last_undegraded":"2023-08-11T14:01:38.453184+0000","last_fullsized":"2023-08-11T14:01:38.453184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:17:44.247485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320548+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:00.320548+0000","last_peered":"2023-08-11T14:02:00.320548+0000","last_clean":"2023-08-11T14:02:00.320548+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:00.320548+0000","last_undegraded":"2023-08-11T14:02:00.320548+0000","last_fullsized":"2023-08-11T14:02:00.320548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:22.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453958+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:01:38.453958+0000","last_peered":"2023-08-11T14:01:38.453958+0000","last_clean":"2023-08-11T14:01:38.453958+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:01:38.453958+0000","last_undegraded":"2023-08-11T14:01:38.453958+0000","last_fullsized":"2023-08-11T14:01:38.453958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:03.238481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453996+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:01:38.453996+0000","last_peered":"2023-08-11T14:01:38.453996+0000","last_clean":"2023-08-11T14:01:38.453996+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:01:38.453996+0000","last_undegraded":"2023-08-11T14:01:38.453996+0000","last_fullsized":"2023-08-11T14:01:38.453996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:16:23.427336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.358000+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:01:38.358000+0000","last_peered":"2023-08-11T14:01:38.358000+0000","last_clean":"2023-08-11T14:01:38.358000+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:01:38.358000+0000","last_undegraded":"2023-08-11T14:01:38.358000+0000","last_fullsized":"2023-08-11T14:01:38.358000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:54.195587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454315+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:01:38.454315+0000","last_peered":"2023-08-11T14:01:38.454315+0000","last_clean":"2023-08-11T14:01:38.454315+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:01:38.454315+0000","last_undegraded":"2023-08-11T14:01:38.454315+0000","last_fullsized":"2023-08-11T14:01:38.454315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:23.010885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453705+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:01:38.453705+0000","last_peered":"2023-08-11T14:01:38.453705+0000","last_clean":"2023-08-11T14:01:38.453705+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:01:38.453705+0000","last_undegraded":"2023-08-11T14:01:38.453705+0000","last_fullsized":"2023-08-11T14:01:38.453705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:29.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455259+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:01:38.455259+0000","last_peered":"2023-08-11T14:01:38.455259+0000","last_clean":"2023-08-11T14:01:38.455259+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:01:38.455259+0000","last_undegraded":"2023-08-11T14:01:38.455259+0000","last_fullsized":"2023-08-11T14:01:38.455259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:25:44.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454215+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:01:38.454215+0000","last_peered":"2023-08-11T14:01:38.454215+0000","last_clean":"2023-08-11T14:01:38.454215+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:01:38.454215+0000","last_undegraded":"2023-08-11T14:01:38.454215+0000","last_fullsized":"2023-08-11T14:01:38.454215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:25.219837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453473+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:01:38.453473+0000","last_peered":"2023-08-11T14:01:38.453473+0000","last_clean":"2023-08-11T14:01:38.453473+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:01:38.453473+0000","last_undegraded":"2023-08-11T14:01:38.453473+0000","last_fullsized":"2023-08-11T14:01:38.453473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:09:10.549528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357742+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:01:38.357742+0000","last_peered":"2023-08-11T14:01:38.357742+0000","last_clean":"2023-08-11T14:01:38.357742+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:01:38.357742+0000","last_undegraded":"2023-08-11T14:01:38.357742+0000","last_fullsized":"2023-08-11T14:01:38.357742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:33:26.233928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454286+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:01:38.454286+0000","last_peered":"2023-08-11T14:01:38.454286+0000","last_clean":"2023-08-11T14:01:38.454286+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:01:38.454286+0000","last_undegraded":"2023-08-11T14:01:38.454286+0000","last_fullsized":"2023-08-11T14:01:38.454286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:09.697084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454819+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:01:38.454819+0000","last_peered":"2023-08-11T14:01:38.454819+0000","last_clean":"2023-08-11T14:01:38.454819+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:01:38.454819+0000","last_undegraded":"2023-08-11T14:01:38.454819+0000","last_fullsized":"2023-08-11T14:01:38.454819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:08:07.050432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455253+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:01:38.455253+0000","last_peered":"2023-08-11T14:01:38.455253+0000","last_clean":"2023-08-11T14:01:38.455253+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:01:38.455253+0000","last_undegraded":"2023-08-11T14:01:38.455253+0000","last_fullsized":"2023-08-11T14:01:38.455253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:01:53.038590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455190+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:01:38.455190+0000","last_peered":"2023-08-11T14:01:38.455190+0000","last_clean":"2023-08-11T14:01:38.455190+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:01:38.455190+0000","last_undegraded":"2023-08-11T14:01:38.455190+0000","last_fullsized":"2023-08-11T14:01:38.455190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:01:59.437947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454868+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:01:38.454868+0000","last_peered":"2023-08-11T14:01:38.454868+0000","last_clean":"2023-08-11T14:01:38.454868+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:01:38.454868+0000","last_undegraded":"2023-08-11T14:01:38.454868+0000","last_fullsized":"2023-08-11T14:01:38.454868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:30.935593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357848+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:01:38.357848+0000","last_peered":"2023-08-11T14:01:38.357848+0000","last_clean":"2023-08-11T14:01:38.357848+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:01:38.357848+0000","last_undegraded":"2023-08-11T14:01:38.357848+0000","last_fullsized":"2023-08-11T14:01:38.357848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:11.005735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357783+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:01:38.357783+0000","last_peered":"2023-08-11T14:01:38.357783+0000","last_clean":"2023-08-11T14:01:38.357783+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:01:38.357783+0000","last_undegraded":"2023-08-11T14:01:38.357783+0000","last_fullsized":"2023-08-11T14:01:38.357783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:03:32.528257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453988+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:01:38.453988+0000","last_peered":"2023-08-11T14:01:38.453988+0000","last_clean":"2023-08-11T14:01:38.453988+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:01:38.453988+0000","last_undegraded":"2023-08-11T14:01:38.453988+0000","last_fullsized":"2023-08-11T14:01:38.453988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:53.361416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9954,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":28,"num_read_kb":26,"num_write":184,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":56,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":28742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.754}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678036,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1061588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.504}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.39100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.61799999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.71199999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72899999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.437}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.88},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.751},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.96999999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.94399999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364834,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1057687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.498}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.622},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":465890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":469369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.84199999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.52400000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.89100000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.75},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65800000000000003}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.872}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":471614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.876},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.439}]},{"osd":6,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":466208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.86899999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.745}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":470986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.439}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79900000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1063084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:03.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:04.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 -- 172.21.15.3:0/458696937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 msgr2=0x7f31f4073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/458696937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4073800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009990 tx=0x7f31e4009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 -- 172.21.15.3:0/458696937 shutdown_connections 2023-08-11T14:02:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/458696937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f41087e0 0x7f31f410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/458696937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f4074960 0x7f31f4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/458696937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4073800 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 -- 172.21.15.3:0/458696937 >> 172.21.15.3:0/458696937 conn(0x7f31f40fbe50 msgr2=0x7f31f40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:04.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 -- 172.21.15.3:0/458696937 shutdown_connections 2023-08-11T14:02:04.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.040+0000 7f31fc5ff700 1 -- 172.21.15.3:0/458696937 wait complete. 2023-08-11T14:02:04.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.041+0000 7f31fc5ff700 1 Processor -- start 2023-08-11T14:02:04.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.041+0000 7f31fc5ff700 1 -- start start 2023-08-11T14:02:04.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.041+0000 7f31fc5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.041+0000 7f31fc5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f4074960 0x7f31f4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fc5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f41087e0 0x7f31f419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fc5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41a03b0 con 0x7f31f4073380 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fc5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41a04f0 con 0x7f31f4074960 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fc5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41a0680 con 0x7f31f41087e0 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41298/0 (socket says 172.21.15.3:41298) 2023-08-11T14:02:04.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 -- 172.21.15.3:0/2348774268 learned_addr learned my addr 172.21.15.3:0/2348774268 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fab9c700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f41087e0 0x7f31f419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31f9b9a700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f4074960 0x7f31f4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 -- 172.21.15.3:0/2348774268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f41087e0 msgr2=0x7f31f419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f41087e0 0x7f31f419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 -- 172.21.15.3:0/2348774268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f4074960 msgr2=0x7f31f4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f4074960 0x7f31f4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.042+0000 7f31fa39b700 1 -- 172.21.15.3:0/2348774268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40095f0 con 0x7f31f4073380 2023-08-11T14:02:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31fa39b700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9a0 tx=0x7f31f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0009790 con 0x7f31f4073380 2023-08-11T14:02:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f0010430 con 0x7f31f4073380 2023-08-11T14:02:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0018680 con 0x7f31f4073380 2023-08-11T14:02:04.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f41a0960 con 0x7f31f4073380 2023-08-11T14:02:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.043+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f41a0e20 con 0x7f31f4073380 2023-08-11T14:02:04.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.044+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f4106e90 con 0x7f31f4073380 2023-08-11T14:02:04.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.046+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f31f00105a0 con 0x7f31f4073380 2023-08-11T14:02:04.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.048+0000 7f31eb7fe700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 0x7f31e00a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.048+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f31f0014070 con 0x7f31f4073380 2023-08-11T14:02:04.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.048+0000 7f31f9b9a700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 0x7f31e00a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.048+0000 7f31f9b9a700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 0x7f31e00a2560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b2e0 tx=0x7f31e4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:04.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.051+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f31f0051c30 con 0x7f31f4073380 2023-08-11T14:02:04.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.259+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31f402cea0 con 0x7f31e00a00a0 2023-08-11T14:02:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.272+0000 7f31eb7fe700 1 -- 172.21.15.3:0/2348774268 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422588 (secure 0 0 0) 0x7f31f402cea0 con 0x7f31e00a00a0 2023-08-11T14:02:04.279 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 msgr2=0x7f31e00a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 0x7f31e00a2560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b2e0 tx=0x7f31e4005fb0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 msgr2=0x7f31f4198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9a0 tx=0x7f31f000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 shutdown_connections 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31f41087e0 0x7f31f419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31f4073380 0x7f31f4198830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f31e00a00a0 0x7f31e00a2560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 --2- 172.21.15.3:0/2348774268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f4074960 0x7f31f4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 >> 172.21.15.3:0/2348774268 conn(0x7f31f40fbe50 msgr2=0x7f31f40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 shutdown_connections 2023-08-11T14:02:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.277+0000 7f31fc5ff700 1 -- 172.21.15.3:0/2348774268 wait complete. 2023-08-11T14:02:04.285 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:04.297 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-08-11T14:02:04.166679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640476,"num_objects":95,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":173,"num_read_kb":193,"num_write":393,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":323,"ondisk_log_size":323,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337732,"kb_used_data":13520,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604092,"statfs":{"total":1151940427776,"available":1151594590208,"internally_reserved":0,"allocated":13844480,"data_stored":7543484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45609,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":47,"num_read_kb":84,"num_write":164,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002114"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454489+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:01:38.454489+0000","last_peered":"2023-08-11T14:01:38.454489+0000","last_clean":"2023-08-11T14:01:38.454489+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:01:38.454489+0000","last_undegraded":"2023-08-11T14:01:38.454489+0000","last_fullsized":"2023-08-11T14:01:38.454489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:15.565502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453396+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:01:38.453396+0000","last_peered":"2023-08-11T14:01:38.453396+0000","last_clean":"2023-08-11T14:01:38.453396+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:01:38.453396+0000","last_undegraded":"2023-08-11T14:01:38.453396+0000","last_fullsized":"2023-08-11T14:01:38.453396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:05.487390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453350+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:01:38.453350+0000","last_peered":"2023-08-11T14:01:38.453350+0000","last_clean":"2023-08-11T14:01:38.453350+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:01:38.453350+0000","last_undegraded":"2023-08-11T14:01:38.453350+0000","last_fullsized":"2023-08-11T14:01:38.453350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:32:07.002359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453773+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:01:38.453773+0000","last_peered":"2023-08-11T14:01:38.453773+0000","last_clean":"2023-08-11T14:01:38.453773+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:01:38.453773+0000","last_undegraded":"2023-08-11T14:01:38.453773+0000","last_fullsized":"2023-08-11T14:01:38.453773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:24.933688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357698+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:01:38.357698+0000","last_peered":"2023-08-11T14:01:38.357698+0000","last_clean":"2023-08-11T14:01:38.357698+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:01:38.357698+0000","last_undegraded":"2023-08-11T14:01:38.357698+0000","last_fullsized":"2023-08-11T14:01:38.357698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:30:43.465070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453427+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:01:38.453427+0000","last_peered":"2023-08-11T14:01:38.453427+0000","last_clean":"2023-08-11T14:01:38.453427+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:01:38.453427+0000","last_undegraded":"2023-08-11T14:01:38.453427+0000","last_fullsized":"2023-08-11T14:01:38.453427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:45:09.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:38.454237+0000","last_change":"2023-08-11T14:02:03.242225+0000","last_active":"2023-08-11T14:02:03.242225+0000","last_peered":"2023-08-11T14:01:38.454237+0000","last_clean":"2023-08-11T14:01:38.454237+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454237+0000","last_undegraded":"2023-08-11T14:01:38.454237+0000","last_fullsized":"2023-08-11T14:01:38.454237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347768+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:02.347768+0000","last_peered":"2023-08-11T14:02:02.347768+0000","last_clean":"2023-08-11T14:02:02.347768+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:02.347768+0000","last_undegraded":"2023-08-11T14:02:02.347768+0000","last_fullsized":"2023-08-11T14:02:02.347768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:30:58.508053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454608+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:01:38.454608+0000","last_peered":"2023-08-11T14:01:38.454608+0000","last_clean":"2023-08-11T14:01:38.454608+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:01:38.454608+0000","last_undegraded":"2023-08-11T14:01:38.454608+0000","last_fullsized":"2023-08-11T14:01:38.454608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:36.691099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453612+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:01:38.453612+0000","last_peered":"2023-08-11T14:01:38.453612+0000","last_clean":"2023-08-11T14:01:38.453612+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:01:38.453612+0000","last_undegraded":"2023-08-11T14:01:38.453612+0000","last_fullsized":"2023-08-11T14:01:38.453612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:41.734924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453057+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:01:38.453057+0000","last_peered":"2023-08-11T14:01:38.453057+0000","last_clean":"2023-08-11T14:01:38.453057+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:01:38.453057+0000","last_undegraded":"2023-08-11T14:01:38.453057+0000","last_fullsized":"2023-08-11T14:01:38.453057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:54:14.411087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453329+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:01:38.453329+0000","last_peered":"2023-08-11T14:01:38.453329+0000","last_clean":"2023-08-11T14:01:38.453329+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:01:38.453329+0000","last_undegraded":"2023-08-11T14:01:38.453329+0000","last_fullsized":"2023-08-11T14:01:38.453329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:45.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346972+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:02.346972+0000","last_peered":"2023-08-11T14:02:02.346972+0000","last_clean":"2023-08-11T14:02:02.346972+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:02.346972+0000","last_undegraded":"2023-08-11T14:02:02.346972+0000","last_fullsized":"2023-08-11T14:02:02.346972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:31.791055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454690+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:01:38.454690+0000","last_peered":"2023-08-11T14:01:38.454690+0000","last_clean":"2023-08-11T14:01:38.454690+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:01:38.454690+0000","last_undegraded":"2023-08-11T14:01:38.454690+0000","last_fullsized":"2023-08-11T14:01:38.454690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:06:02.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345350+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:02.345350+0000","last_peered":"2023-08-11T14:02:02.345350+0000","last_clean":"2023-08-11T14:02:02.345350+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:02.345350+0000","last_undegraded":"2023-08-11T14:02:02.345350+0000","last_fullsized":"2023-08-11T14:02:02.345350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:08:36.944921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454406+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:01:38.454406+0000","last_peered":"2023-08-11T14:01:38.454406+0000","last_clean":"2023-08-11T14:01:38.454406+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:01:38.454406+0000","last_undegraded":"2023-08-11T14:01:38.454406+0000","last_fullsized":"2023-08-11T14:01:38.454406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:02:50.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455090+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:01:38.455090+0000","last_peered":"2023-08-11T14:01:38.455090+0000","last_clean":"2023-08-11T14:01:38.455090+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:01:38.455090+0000","last_undegraded":"2023-08-11T14:01:38.455090+0000","last_fullsized":"2023-08-11T14:01:38.455090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:39.906536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339916+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:02.339916+0000","last_peered":"2023-08-11T14:02:02.339916+0000","last_clean":"2023-08-11T14:02:02.339916+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:02.339916+0000","last_undegraded":"2023-08-11T14:02:02.339916+0000","last_fullsized":"2023-08-11T14:02:02.339916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:13:11.710971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454673+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:01:38.454673+0000","last_peered":"2023-08-11T14:01:38.454673+0000","last_clean":"2023-08-11T14:01:38.454673+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:01:38.454673+0000","last_undegraded":"2023-08-11T14:01:38.454673+0000","last_fullsized":"2023-08-11T14:01:38.454673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:09:31.987246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453033+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:01:38.453033+0000","last_peered":"2023-08-11T14:01:38.453033+0000","last_clean":"2023-08-11T14:01:38.453033+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:01:38.453033+0000","last_undegraded":"2023-08-11T14:01:38.453033+0000","last_fullsized":"2023-08-11T14:01:38.453033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:24:09.010833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454851+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:01:38.454851+0000","last_peered":"2023-08-11T14:01:38.454851+0000","last_clean":"2023-08-11T14:01:38.454851+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:01:38.454851+0000","last_undegraded":"2023-08-11T14:01:38.454851+0000","last_fullsized":"2023-08-11T14:01:38.454851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:09.972413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453569+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:01:38.453569+0000","last_peered":"2023-08-11T14:01:38.453569+0000","last_clean":"2023-08-11T14:01:38.453569+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:01:38.453569+0000","last_undegraded":"2023-08-11T14:01:38.453569+0000","last_fullsized":"2023-08-11T14:01:38.453569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:31.879999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453610+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:01:38.453610+0000","last_peered":"2023-08-11T14:01:38.453610+0000","last_clean":"2023-08-11T14:01:38.453610+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:01:38.453610+0000","last_undegraded":"2023-08-11T14:01:38.453610+0000","last_fullsized":"2023-08-11T14:01:38.453610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:00.077438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322139+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:00.322139+0000","last_peered":"2023-08-11T14:02:00.322139+0000","last_clean":"2023-08-11T14:02:00.322139+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:00.322139+0000","last_undegraded":"2023-08-11T14:02:00.322139+0000","last_fullsized":"2023-08-11T14:02:00.322139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:53.482576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347387+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:02.347387+0000","last_peered":"2023-08-11T14:02:02.347387+0000","last_clean":"2023-08-11T14:02:02.347387+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:02.347387+0000","last_undegraded":"2023-08-11T14:02:02.347387+0000","last_fullsized":"2023-08-11T14:02:02.347387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:13.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.452997+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:01:38.452997+0000","last_peered":"2023-08-11T14:01:38.452997+0000","last_clean":"2023-08-11T14:01:38.452997+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:01:38.452997+0000","last_undegraded":"2023-08-11T14:01:38.452997+0000","last_fullsized":"2023-08-11T14:01:38.452997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:46.488285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320442+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:00.320442+0000","last_peered":"2023-08-11T14:02:00.320442+0000","last_clean":"2023-08-11T14:02:00.320442+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:00.320442+0000","last_undegraded":"2023-08-11T14:02:00.320442+0000","last_fullsized":"2023-08-11T14:02:00.320442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:37:57.485057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455704+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:01:38.455704+0000","last_peered":"2023-08-11T14:01:38.455704+0000","last_clean":"2023-08-11T14:01:38.455704+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:01:38.455704+0000","last_undegraded":"2023-08-11T14:01:38.455704+0000","last_fullsized":"2023-08-11T14:01:38.455704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:25.911259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454143+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:01:38.454143+0000","last_peered":"2023-08-11T14:01:38.454143+0000","last_clean":"2023-08-11T14:01:38.454143+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:01:38.454143+0000","last_undegraded":"2023-08-11T14:01:38.454143+0000","last_fullsized":"2023-08-11T14:01:38.454143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:00.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321984+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:01.321984+0000","last_peered":"2023-08-11T14:02:01.321984+0000","last_clean":"2023-08-11T14:02:01.321984+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:01.321984+0000","last_undegraded":"2023-08-11T14:02:01.321984+0000","last_fullsized":"2023-08-11T14:02:01.321984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:10.771366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357613+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:01:38.357613+0000","last_peered":"2023-08-11T14:01:38.357613+0000","last_clean":"2023-08-11T14:01:38.357613+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:01:38.357613+0000","last_undegraded":"2023-08-11T14:01:38.357613+0000","last_fullsized":"2023-08-11T14:01:38.357613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:46:18.787688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454174+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:01:38.454174+0000","last_peered":"2023-08-11T14:01:38.454174+0000","last_clean":"2023-08-11T14:01:38.454174+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:01:38.454174+0000","last_undegraded":"2023-08-11T14:01:38.454174+0000","last_fullsized":"2023-08-11T14:01:38.454174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:21.945068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455026+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:01:38.455026+0000","last_peered":"2023-08-11T14:01:38.455026+0000","last_clean":"2023-08-11T14:01:38.455026+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:01:38.455026+0000","last_undegraded":"2023-08-11T14:01:38.455026+0000","last_fullsized":"2023-08-11T14:01:38.455026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:26:04.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454393+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:01:38.454393+0000","last_peered":"2023-08-11T14:01:38.454393+0000","last_clean":"2023-08-11T14:01:38.454393+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:01:38.454393+0000","last_undegraded":"2023-08-11T14:01:38.454393+0000","last_fullsized":"2023-08-11T14:01:38.454393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:09.108932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454198+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:01:38.454198+0000","last_peered":"2023-08-11T14:01:38.454198+0000","last_clean":"2023-08-11T14:01:38.454198+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:01:38.454198+0000","last_undegraded":"2023-08-11T14:01:38.454198+0000","last_fullsized":"2023-08-11T14:01:38.454198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:02.471143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453242+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:01:38.453242+0000","last_peered":"2023-08-11T14:01:38.453242+0000","last_clean":"2023-08-11T14:01:38.453242+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:01:38.453242+0000","last_undegraded":"2023-08-11T14:01:38.453242+0000","last_fullsized":"2023-08-11T14:01:38.453242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:21:10.060538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321844+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:00.321844+0000","last_peered":"2023-08-11T14:02:00.321844+0000","last_clean":"2023-08-11T14:02:00.321844+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:00.321844+0000","last_undegraded":"2023-08-11T14:02:00.321844+0000","last_fullsized":"2023-08-11T14:02:00.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:29.646533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339573+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:02.339573+0000","last_peered":"2023-08-11T14:02:02.339573+0000","last_clean":"2023-08-11T14:02:02.339573+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:02.339573+0000","last_undegraded":"2023-08-11T14:02:02.339573+0000","last_fullsized":"2023-08-11T14:02:02.339573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:12.537664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454637+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:01:38.454637+0000","last_peered":"2023-08-11T14:01:38.454637+0000","last_clean":"2023-08-11T14:01:38.454637+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:01:38.454637+0000","last_undegraded":"2023-08-11T14:01:38.454637+0000","last_fullsized":"2023-08-11T14:01:38.454637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:14:38.485351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332630+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:03.332630+0000","last_peered":"2023-08-11T14:02:03.332630+0000","last_clean":"2023-08-11T14:02:03.332630+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:03.332630+0000","last_undegraded":"2023-08-11T14:02:03.332630+0000","last_fullsized":"2023-08-11T14:02:03.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347670+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:02.347670+0000","last_peered":"2023-08-11T14:02:02.347670+0000","last_clean":"2023-08-11T14:02:02.347670+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:02.347670+0000","last_undegraded":"2023-08-11T14:02:02.347670+0000","last_fullsized":"2023-08-11T14:02:02.347670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:29.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340313+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:02.340313+0000","last_peered":"2023-08-11T14:02:02.340313+0000","last_clean":"2023-08-11T14:02:02.340313+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:02.340313+0000","last_undegraded":"2023-08-11T14:02:02.340313+0000","last_fullsized":"2023-08-11T14:02:02.340313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:37:35.917226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293732+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:01:57.293732+0000","last_peered":"2023-08-11T14:01:57.293732+0000","last_clean":"2023-08-11T14:01:57.293732+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:01:57.293732+0000","last_undegraded":"2023-08-11T14:01:57.293732+0000","last_fullsized":"2023-08-11T14:01:57.293732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:00.643436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453798+0000","last_change":"2023-08-11T14:01:33.308455+0000","last_active":"2023-08-11T14:01:38.453798+0000","last_peered":"2023-08-11T14:01:38.453798+0000","last_clean":"2023-08-11T14:01:38.453798+0000","last_became_active":"2023-08-11T14:01:33.308084+0000","last_became_peered":"2023-08-11T14:01:33.308084+0000","last_unstale":"2023-08-11T14:01:38.453798+0000","last_undegraded":"2023-08-11T14:01:38.453798+0000","last_fullsized":"2023-08-11T14:01:38.453798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:33.705786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453779+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:01:38.453779+0000","last_peered":"2023-08-11T14:01:38.453779+0000","last_clean":"2023-08-11T14:01:38.453779+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:01:38.453779+0000","last_undegraded":"2023-08-11T14:01:38.453779+0000","last_fullsized":"2023-08-11T14:01:38.453779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:41.727745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312606+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:01:59.312606+0000","last_peered":"2023-08-11T14:01:59.312606+0000","last_clean":"2023-08-11T14:01:59.312606+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:01:59.312606+0000","last_undegraded":"2023-08-11T14:01:59.312606+0000","last_fullsized":"2023-08-11T14:01:59.312606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:33:25.490224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347579+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:02.347579+0000","last_peered":"2023-08-11T14:02:02.347579+0000","last_clean":"2023-08-11T14:02:02.347579+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:02.347579+0000","last_undegraded":"2023-08-11T14:02:02.347579+0000","last_fullsized":"2023-08-11T14:02:02.347579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:04.886749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340016+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:02.340016+0000","last_peered":"2023-08-11T14:02:02.340016+0000","last_clean":"2023-08-11T14:02:02.340016+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:02.340016+0000","last_undegraded":"2023-08-11T14:02:02.340016+0000","last_fullsized":"2023-08-11T14:02:02.340016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:07.686519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'25","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333081+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:03.333081+0000","last_peered":"2023-08-11T14:02:03.333081+0000","last_clean":"2023-08-11T14:02:03.333081+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:03.333081+0000","last_undegraded":"2023-08-11T14:02:03.333081+0000","last_fullsized":"2023-08-11T14:02:03.333081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":48,"num_write":27,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357914+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:01:38.357914+0000","last_peered":"2023-08-11T14:01:38.357914+0000","last_clean":"2023-08-11T14:01:38.357914+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:01:38.357914+0000","last_undegraded":"2023-08-11T14:01:38.357914+0000","last_fullsized":"2023-08-11T14:01:38.357914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:21:16.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455907+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:01:38.455907+0000","last_peered":"2023-08-11T14:01:38.455907+0000","last_clean":"2023-08-11T14:01:38.455907+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:01:38.455907+0000","last_undegraded":"2023-08-11T14:01:38.455907+0000","last_fullsized":"2023-08-11T14:01:38.455907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:56.419641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305981+0000","last_change":"2023-08-11T14:01:32.306792+0000","last_active":"2023-08-11T14:01:58.305981+0000","last_peered":"2023-08-11T14:01:58.305981+0000","last_clean":"2023-08-11T14:01:58.305981+0000","last_became_active":"2023-08-11T14:01:32.306447+0000","last_became_peered":"2023-08-11T14:01:32.306447+0000","last_unstale":"2023-08-11T14:01:58.305981+0000","last_undegraded":"2023-08-11T14:01:58.305981+0000","last_fullsized":"2023-08-11T14:01:58.305981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:09.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453894+0000","last_change":"2023-08-11T14:01:37.375702+0000","last_active":"2023-08-11T14:01:38.453894+0000","last_peered":"2023-08-11T14:01:38.453894+0000","last_clean":"2023-08-11T14:01:38.453894+0000","last_became_active":"2023-08-11T14:01:37.375199+0000","last_became_peered":"2023-08-11T14:01:37.375199+0000","last_unstale":"2023-08-11T14:01:38.453894+0000","last_undegraded":"2023-08-11T14:01:38.453894+0000","last_fullsized":"2023-08-11T14:01:38.453894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:14.150791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321941+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:00.321941+0000","last_peered":"2023-08-11T14:02:00.321941+0000","last_clean":"2023-08-11T14:02:00.321941+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:00.321941+0000","last_undegraded":"2023-08-11T14:02:00.321941+0000","last_fullsized":"2023-08-11T14:02:00.321941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:02:07.982583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453519+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:01:38.453519+0000","last_peered":"2023-08-11T14:01:38.453519+0000","last_clean":"2023-08-11T14:01:38.453519+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:01:38.453519+0000","last_undegraded":"2023-08-11T14:01:38.453519+0000","last_fullsized":"2023-08-11T14:01:38.453519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:57.895242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454218+0000","last_change":"2023-08-11T14:01:37.377532+0000","last_active":"2023-08-11T14:01:38.454218+0000","last_peered":"2023-08-11T14:01:38.454218+0000","last_clean":"2023-08-11T14:01:38.454218+0000","last_became_active":"2023-08-11T14:01:37.377142+0000","last_became_peered":"2023-08-11T14:01:37.377142+0000","last_unstale":"2023-08-11T14:01:38.454218+0000","last_undegraded":"2023-08-11T14:01:38.454218+0000","last_fullsized":"2023-08-11T14:01:38.454218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:13.506622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321844+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:01.321844+0000","last_peered":"2023-08-11T14:02:01.321844+0000","last_clean":"2023-08-11T14:02:01.321844+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:01.321844+0000","last_undegraded":"2023-08-11T14:02:01.321844+0000","last_fullsized":"2023-08-11T14:02:01.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:27.821349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453776+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:01:38.453776+0000","last_peered":"2023-08-11T14:01:38.453776+0000","last_clean":"2023-08-11T14:01:38.453776+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:01:38.453776+0000","last_undegraded":"2023-08-11T14:01:38.453776+0000","last_fullsized":"2023-08-11T14:01:38.453776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:13:35.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.318041+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:00.318041+0000","last_peered":"2023-08-11T14:02:00.318041+0000","last_clean":"2023-08-11T14:02:00.318041+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:00.318041+0000","last_undegraded":"2023-08-11T14:02:00.318041+0000","last_fullsized":"2023-08-11T14:02:00.318041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:33:54.507816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:33.290559+0000","last_change":"2023-08-11T14:02:03.242268+0000","last_active":"2023-08-11T14:01:32.273793+0000","last_peered":"2023-08-11T14:01:32.273793+0000","last_clean":"2023-08-11T14:01:32.273793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-11T14:01:33.290559+0000","last_undegraded":"2023-08-11T14:01:33.290559+0000","last_fullsized":"2023-08-11T14:01:33.290559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454385+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:01:38.454385+0000","last_peered":"2023-08-11T14:01:38.454385+0000","last_clean":"2023-08-11T14:01:38.454385+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:01:38.454385+0000","last_undegraded":"2023-08-11T14:01:38.454385+0000","last_fullsized":"2023-08-11T14:01:38.454385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:21.288285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.230701+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:01.230701+0000","last_peered":"2023-08-11T14:02:01.230701+0000","last_clean":"2023-08-11T14:02:01.230701+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:01.230701+0000","last_undegraded":"2023-08-11T14:02:01.230701+0000","last_fullsized":"2023-08-11T14:02:01.230701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:14.365596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453956+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:01:38.453956+0000","last_peered":"2023-08-11T14:01:38.453956+0000","last_clean":"2023-08-11T14:01:38.453956+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:01:38.453956+0000","last_undegraded":"2023-08-11T14:01:38.453956+0000","last_fullsized":"2023-08-11T14:01:38.453956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:22.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357583+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:01:38.357583+0000","last_peered":"2023-08-11T14:01:38.357583+0000","last_clean":"2023-08-11T14:01:38.357583+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:01:38.357583+0000","last_undegraded":"2023-08-11T14:01:38.357583+0000","last_fullsized":"2023-08-11T14:01:38.357583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:04:46.172105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.338227+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:02.338227+0000","last_peered":"2023-08-11T14:02:02.338227+0000","last_clean":"2023-08-11T14:02:02.338227+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:02.338227+0000","last_undegraded":"2023-08-11T14:02:02.338227+0000","last_fullsized":"2023-08-11T14:02:02.338227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:37.579833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453781+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:01:38.453781+0000","last_peered":"2023-08-11T14:01:38.453781+0000","last_clean":"2023-08-11T14:01:38.453781+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:01:38.453781+0000","last_undegraded":"2023-08-11T14:01:38.453781+0000","last_fullsized":"2023-08-11T14:01:38.453781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:37.699260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455326+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:01:38.455326+0000","last_peered":"2023-08-11T14:01:38.455326+0000","last_clean":"2023-08-11T14:01:38.455326+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:01:38.455326+0000","last_undegraded":"2023-08-11T14:01:38.455326+0000","last_fullsized":"2023-08-11T14:01:38.455326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:57.290710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331730+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:03.331730+0000","last_peered":"2023-08-11T14:02:03.331730+0000","last_clean":"2023-08-11T14:02:03.331730+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:03.331730+0000","last_undegraded":"2023-08-11T14:02:03.331730+0000","last_fullsized":"2023-08-11T14:02:03.331730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454372+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:01:38.454372+0000","last_peered":"2023-08-11T14:01:38.454372+0000","last_clean":"2023-08-11T14:01:38.454372+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:01:38.454372+0000","last_undegraded":"2023-08-11T14:01:38.454372+0000","last_fullsized":"2023-08-11T14:01:38.454372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:32:11.305694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454548+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:01:38.454548+0000","last_peered":"2023-08-11T14:01:38.454548+0000","last_clean":"2023-08-11T14:01:38.454548+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:01:38.454548+0000","last_undegraded":"2023-08-11T14:01:38.454548+0000","last_fullsized":"2023-08-11T14:01:38.454548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:38.933163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.319779+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:01.319779+0000","last_peered":"2023-08-11T14:02:01.319779+0000","last_clean":"2023-08-11T14:02:01.319779+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:01.319779+0000","last_undegraded":"2023-08-11T14:02:01.319779+0000","last_fullsized":"2023-08-11T14:02:01.319779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:56:44.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453444+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:01:38.453444+0000","last_peered":"2023-08-11T14:01:38.453444+0000","last_clean":"2023-08-11T14:01:38.453444+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:01:38.453444+0000","last_undegraded":"2023-08-11T14:01:38.453444+0000","last_fullsized":"2023-08-11T14:01:38.453444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:10.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455405+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:01:38.455405+0000","last_peered":"2023-08-11T14:01:38.455405+0000","last_clean":"2023-08-11T14:01:38.455405+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:01:38.455405+0000","last_undegraded":"2023-08-11T14:01:38.455405+0000","last_fullsized":"2023-08-11T14:01:38.455405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:31:57.406400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305859+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:01:58.305859+0000","last_peered":"2023-08-11T14:01:58.305859+0000","last_clean":"2023-08-11T14:01:58.305859+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:01:58.305859+0000","last_undegraded":"2023-08-11T14:01:58.305859+0000","last_fullsized":"2023-08-11T14:01:58.305859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:21:11.143846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454625+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:01:38.454625+0000","last_peered":"2023-08-11T14:01:38.454625+0000","last_clean":"2023-08-11T14:01:38.454625+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:01:38.454625+0000","last_undegraded":"2023-08-11T14:01:38.454625+0000","last_fullsized":"2023-08-11T14:01:38.454625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:40:26.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320780+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:00.320780+0000","last_peered":"2023-08-11T14:02:00.320780+0000","last_clean":"2023-08-11T14:02:00.320780+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:00.320780+0000","last_undegraded":"2023-08-11T14:02:00.320780+0000","last_fullsized":"2023-08-11T14:02:00.320780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:17:38.449276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454047+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:01:38.454047+0000","last_peered":"2023-08-11T14:01:38.454047+0000","last_clean":"2023-08-11T14:01:38.454047+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:01:38.454047+0000","last_undegraded":"2023-08-11T14:01:38.454047+0000","last_fullsized":"2023-08-11T14:01:38.454047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:47:11.994511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345212+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:02.345212+0000","last_peered":"2023-08-11T14:02:02.345212+0000","last_clean":"2023-08-11T14:02:02.345212+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:02.345212+0000","last_undegraded":"2023-08-11T14:02:02.345212+0000","last_fullsized":"2023-08-11T14:02:02.345212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:23.734051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455078+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:01:38.455078+0000","last_peered":"2023-08-11T14:01:38.455078+0000","last_clean":"2023-08-11T14:01:38.455078+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:01:38.455078+0000","last_undegraded":"2023-08-11T14:01:38.455078+0000","last_fullsized":"2023-08-11T14:01:38.455078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:14:16.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454784+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:01:38.454784+0000","last_peered":"2023-08-11T14:01:38.454784+0000","last_clean":"2023-08-11T14:01:38.454784+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:01:38.454784+0000","last_undegraded":"2023-08-11T14:01:38.454784+0000","last_fullsized":"2023-08-11T14:01:38.454784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:33.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320128+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:00.320128+0000","last_peered":"2023-08-11T14:02:00.320128+0000","last_clean":"2023-08-11T14:02:00.320128+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:00.320128+0000","last_undegraded":"2023-08-11T14:02:00.320128+0000","last_fullsized":"2023-08-11T14:02:00.320128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:50.342786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453594+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:01:38.453594+0000","last_peered":"2023-08-11T14:01:38.453594+0000","last_clean":"2023-08-11T14:01:38.453594+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:01:38.453594+0000","last_undegraded":"2023-08-11T14:01:38.453594+0000","last_fullsized":"2023-08-11T14:01:38.453594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:05.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453251+0000","last_change":"2023-08-11T14:01:37.586607+0000","last_active":"2023-08-11T14:01:38.453251+0000","last_peered":"2023-08-11T14:01:38.453251+0000","last_clean":"2023-08-11T14:01:38.453251+0000","last_became_active":"2023-08-11T14:01:37.586160+0000","last_became_peered":"2023-08-11T14:01:37.586160+0000","last_unstale":"2023-08-11T14:01:38.453251+0000","last_undegraded":"2023-08-11T14:01:38.453251+0000","last_fullsized":"2023-08-11T14:01:38.453251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:11.941518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347080+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:02.347080+0000","last_peered":"2023-08-11T14:02:02.347080+0000","last_clean":"2023-08-11T14:02:02.347080+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:02.347080+0000","last_undegraded":"2023-08-11T14:02:02.347080+0000","last_fullsized":"2023-08-11T14:02:02.347080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:51.895542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357500+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:01:38.357500+0000","last_peered":"2023-08-11T14:01:38.357500+0000","last_clean":"2023-08-11T14:01:38.357500+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:01:38.357500+0000","last_undegraded":"2023-08-11T14:01:38.357500+0000","last_fullsized":"2023-08-11T14:01:38.357500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:49:09.080233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339784+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:02.339784+0000","last_peered":"2023-08-11T14:02:02.339784+0000","last_clean":"2023-08-11T14:02:02.339784+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:02.339784+0000","last_undegraded":"2023-08-11T14:02:02.339784+0000","last_fullsized":"2023-08-11T14:02:02.339784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:41.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322084+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:00.322084+0000","last_peered":"2023-08-11T14:02:00.322084+0000","last_clean":"2023-08-11T14:02:00.322084+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:00.322084+0000","last_undegraded":"2023-08-11T14:02:00.322084+0000","last_fullsized":"2023-08-11T14:02:00.322084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:13:22.765102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453667+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:01:38.453667+0000","last_peered":"2023-08-11T14:01:38.453667+0000","last_clean":"2023-08-11T14:01:38.453667+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:01:38.453667+0000","last_undegraded":"2023-08-11T14:01:38.453667+0000","last_fullsized":"2023-08-11T14:01:38.453667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:10:22.858975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453688+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:01:38.453688+0000","last_peered":"2023-08-11T14:01:38.453688+0000","last_clean":"2023-08-11T14:01:38.453688+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:01:38.453688+0000","last_undegraded":"2023-08-11T14:01:38.453688+0000","last_fullsized":"2023-08-11T14:01:38.453688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:18.721663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.470170+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:01:58.470170+0000","last_peered":"2023-08-11T14:01:58.470170+0000","last_clean":"2023-08-11T14:01:58.470170+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:01:58.470170+0000","last_undegraded":"2023-08-11T14:01:58.470170+0000","last_fullsized":"2023-08-11T14:01:58.470170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:54:36.626882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453866+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:01:38.453866+0000","last_peered":"2023-08-11T14:01:38.453866+0000","last_clean":"2023-08-11T14:01:38.453866+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:01:38.453866+0000","last_undegraded":"2023-08-11T14:01:38.453866+0000","last_fullsized":"2023-08-11T14:01:38.453866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:57.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453150+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:01:38.453150+0000","last_peered":"2023-08-11T14:01:38.453150+0000","last_clean":"2023-08-11T14:01:38.453150+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:01:38.453150+0000","last_undegraded":"2023-08-11T14:01:38.453150+0000","last_fullsized":"2023-08-11T14:01:38.453150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:41.914296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.294819+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:01:57.294819+0000","last_peered":"2023-08-11T14:01:57.294819+0000","last_clean":"2023-08-11T14:01:57.294819+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:01:57.294819+0000","last_undegraded":"2023-08-11T14:01:57.294819+0000","last_fullsized":"2023-08-11T14:01:57.294819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:25.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453769+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:01:38.453769+0000","last_peered":"2023-08-11T14:01:38.453769+0000","last_clean":"2023-08-11T14:01:38.453769+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:01:38.453769+0000","last_undegraded":"2023-08-11T14:01:38.453769+0000","last_fullsized":"2023-08-11T14:01:38.453769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:24.669978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346978+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:02.346978+0000","last_peered":"2023-08-11T14:02:02.346978+0000","last_clean":"2023-08-11T14:02:02.346978+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:02.346978+0000","last_undegraded":"2023-08-11T14:02:02.346978+0000","last_fullsized":"2023-08-11T14:02:02.346978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:00:01.572653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312575+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:01:59.312575+0000","last_peered":"2023-08-11T14:01:59.312575+0000","last_clean":"2023-08-11T14:01:59.312575+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:01:59.312575+0000","last_undegraded":"2023-08-11T14:01:59.312575+0000","last_fullsized":"2023-08-11T14:01:59.312575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:59.189153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454907+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:01:38.454907+0000","last_peered":"2023-08-11T14:01:38.454907+0000","last_clean":"2023-08-11T14:01:38.454907+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:01:38.454907+0000","last_undegraded":"2023-08-11T14:01:38.454907+0000","last_fullsized":"2023-08-11T14:01:38.454907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:01.579515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453184+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:01:38.453184+0000","last_peered":"2023-08-11T14:01:38.453184+0000","last_clean":"2023-08-11T14:01:38.453184+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:01:38.453184+0000","last_undegraded":"2023-08-11T14:01:38.453184+0000","last_fullsized":"2023-08-11T14:01:38.453184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:17:44.247485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320548+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:00.320548+0000","last_peered":"2023-08-11T14:02:00.320548+0000","last_clean":"2023-08-11T14:02:00.320548+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:00.320548+0000","last_undegraded":"2023-08-11T14:02:00.320548+0000","last_fullsized":"2023-08-11T14:02:00.320548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:22.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453958+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:01:38.453958+0000","last_peered":"2023-08-11T14:01:38.453958+0000","last_clean":"2023-08-11T14:01:38.453958+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:01:38.453958+0000","last_undegraded":"2023-08-11T14:01:38.453958+0000","last_fullsized":"2023-08-11T14:01:38.453958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:03.238481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453996+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:01:38.453996+0000","last_peered":"2023-08-11T14:01:38.453996+0000","last_clean":"2023-08-11T14:01:38.453996+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:01:38.453996+0000","last_undegraded":"2023-08-11T14:01:38.453996+0000","last_fullsized":"2023-08-11T14:01:38.453996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:16:23.427336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454315+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:01:38.454315+0000","last_peered":"2023-08-11T14:01:38.454315+0000","last_clean":"2023-08-11T14:01:38.454315+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:01:38.454315+0000","last_undegraded":"2023-08-11T14:01:38.454315+0000","last_fullsized":"2023-08-11T14:01:38.454315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:23.010885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453705+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:01:38.453705+0000","last_peered":"2023-08-11T14:01:38.453705+0000","last_clean":"2023-08-11T14:01:38.453705+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:01:38.453705+0000","last_undegraded":"2023-08-11T14:01:38.453705+0000","last_fullsized":"2023-08-11T14:01:38.453705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:29.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455259+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:01:38.455259+0000","last_peered":"2023-08-11T14:01:38.455259+0000","last_clean":"2023-08-11T14:01:38.455259+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:01:38.455259+0000","last_undegraded":"2023-08-11T14:01:38.455259+0000","last_fullsized":"2023-08-11T14:01:38.455259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:25:44.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454215+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:01:38.454215+0000","last_peered":"2023-08-11T14:01:38.454215+0000","last_clean":"2023-08-11T14:01:38.454215+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:01:38.454215+0000","last_undegraded":"2023-08-11T14:01:38.454215+0000","last_fullsized":"2023-08-11T14:01:38.454215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:25.219837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453473+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:01:38.453473+0000","last_peered":"2023-08-11T14:01:38.453473+0000","last_clean":"2023-08-11T14:01:38.453473+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:01:38.453473+0000","last_undegraded":"2023-08-11T14:01:38.453473+0000","last_fullsized":"2023-08-11T14:01:38.453473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:09:10.549528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454286+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:01:38.454286+0000","last_peered":"2023-08-11T14:01:38.454286+0000","last_clean":"2023-08-11T14:01:38.454286+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:01:38.454286+0000","last_undegraded":"2023-08-11T14:01:38.454286+0000","last_fullsized":"2023-08-11T14:01:38.454286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:09.697084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454819+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:01:38.454819+0000","last_peered":"2023-08-11T14:01:38.454819+0000","last_clean":"2023-08-11T14:01:38.454819+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:01:38.454819+0000","last_undegraded":"2023-08-11T14:01:38.454819+0000","last_fullsized":"2023-08-11T14:01:38.454819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:08:07.050432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455253+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:01:38.455253+0000","last_peered":"2023-08-11T14:01:38.455253+0000","last_clean":"2023-08-11T14:01:38.455253+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:01:38.455253+0000","last_undegraded":"2023-08-11T14:01:38.455253+0000","last_fullsized":"2023-08-11T14:01:38.455253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:01:53.038590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455190+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:01:38.455190+0000","last_peered":"2023-08-11T14:01:38.455190+0000","last_clean":"2023-08-11T14:01:38.455190+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:01:38.455190+0000","last_undegraded":"2023-08-11T14:01:38.455190+0000","last_fullsized":"2023-08-11T14:01:38.455190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:01:59.437947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454868+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:01:38.454868+0000","last_peered":"2023-08-11T14:01:38.454868+0000","last_clean":"2023-08-11T14:01:38.454868+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:01:38.454868+0000","last_undegraded":"2023-08-11T14:01:38.454868+0000","last_fullsized":"2023-08-11T14:01:38.454868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:30.935593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347343+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:02.347343+0000","last_peered":"2023-08-11T14:02:02.347343+0000","last_clean":"2023-08-11T14:02:02.347343+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:02.347343+0000","last_undegraded":"2023-08-11T14:02:02.347343+0000","last_fullsized":"2023-08-11T14:02:02.347343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:36.721470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453988+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:01:38.453988+0000","last_peered":"2023-08-11T14:01:38.453988+0000","last_clean":"2023-08-11T14:01:38.453988+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:01:38.453988+0000","last_undegraded":"2023-08-11T14:01:38.453988+0000","last_fullsized":"2023-08-11T14:01:38.453988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:53.361416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49985,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":47,"num_read_kb":84,"num_write":211,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":111512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.754}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1121039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.38},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.498},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.63}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72599999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.437}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.88},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.751},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.96999999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.94399999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364834,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1057687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.498}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.622},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.753},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.75},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65800000000000003}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.872}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":471614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.876},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.439}]},{"osd":6,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":466208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.86899999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.745}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":470986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.439}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79900000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1063084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:04.307 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-08-11T14:02:04.307 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T14:01:38.454237+0000', 'last_change': '2023-08-11T14:02:03.242225+0000', 'last_active': '2023-08-11T14:02:03.242225+0000', 'last_peered': '2023-08-11T14:01:38.454237+0000', 'last_clean': '2023-08-11T14:01:38.454237+0000', 'last_became_active': '2023-08-11T14:01:33.311835+0000', 'last_became_peered': '2023-08-11T14:01:33.311835+0000', 'last_unstale': '2023-08-11T14:01:38.454237+0000', 'last_undegraded': '2023-08-11T14:01:38.454237+0000', 'last_fullsized': '2023-08-11T14:01:38.454237+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_clean_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 5, 10], 'acting': [9, 5, 10], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-08-11T14:02:04.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-08-11T14:02:04.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T14:01:33.290559+0000', 'last_change': '2023-08-11T14:02:03.242268+0000', 'last_active': '2023-08-11T14:01:32.273793+0000', 'last_peered': '2023-08-11T14:01:32.273793+0000', 'last_clean': '2023-08-11T14:01:32.273793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-11T14:01:33.290559+0000', 'last_undegraded': '2023-08-11T14:01:33.290559+0000', 'last_fullsized': '2023-08-11T14:01:33.290559+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_clean_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 8, 1], 'acting': [9, 8, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-08-11T14:02:04.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-11T14:02:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 -- 172.21.15.3:0/2524536912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 msgr2=0x7f36d8100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 --2- 172.21.15.3:0/2524536912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8100d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f36d40099c0 tx=0x7f36d4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 -- 172.21.15.3:0/2524536912 shutdown_connections 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 --2- 172.21.15.3:0/2524536912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 --2- 172.21.15.3:0/2524536912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8100d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 --2- 172.21.15.3:0/2524536912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 0x7f36d8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.538+0000 7f36df9ff700 1 -- 172.21.15.3:0/2524536912 >> 172.21.15.3:0/2524536912 conn(0x7f36d80fbe50 msgr2=0x7f36d80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.539+0000 7f36df9ff700 1 -- 172.21.15.3:0/2524536912 shutdown_connections 2023-08-11T14:02:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.539+0000 7f36df9ff700 1 -- 172.21.15.3:0/2524536912 wait complete. 2023-08-11T14:02:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 Processor -- start 2023-08-11T14:02:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 -- start start 2023-08-11T14:02:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 0x7f36d8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d81a03c0 con 0x7f36d8100920 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d81a0500 con 0x7f36d81024b0 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36df9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d81a0690 con 0x7f36d8101b10 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36dd79b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36dd79b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41312/0 (socket says 172.21.15.3:41312) 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36dd79b700 1 -- 172.21.15.3:0/3260305991 learned_addr learned my addr 172.21.15.3:0/3260305991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.540+0000 7f36dcf9a700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 0x7f36d8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36ddf9c700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 -- 172.21.15.3:0/3260305991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 msgr2=0x7f36d8198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 0x7f36d8198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 -- 172.21.15.3:0/3260305991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 msgr2=0x7f36d819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 -- 172.21.15.3:0/3260305991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40095f0 con 0x7f36d8100920 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36ddf9c700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36dd79b700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401d070 con 0x7f36d8100920 2023-08-11T14:02:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d40054f0 con 0x7f36d8100920 2023-08-11T14:02:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d81a0910 con 0x7f36d8100920 2023-08-11T14:02:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.541+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4004bd0 con 0x7f36d8100920 2023-08-11T14:02:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.542+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d81a0d90 con 0x7f36d8100920 2023-08-11T14:02:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.543+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d810c180 con 0x7f36d8100920 2023-08-11T14:02:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.545+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f36d400fb80 con 0x7f36d8100920 2023-08-11T14:02:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.546+0000 7f36ce7fc700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 0x7f36c40a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.546+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f36d40cc910 con 0x7f36d8100920 2023-08-11T14:02:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.546+0000 7f36dcf9a700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 0x7f36c40a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.547+0000 7f36dcf9a700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 0x7f36c40a2560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009c60 tx=0x7f36c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:04.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.549+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f36d4096270 con 0x7f36d8100920 2023-08-11T14:02:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:04 smithi103 ceph-mon[130434]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:04 smithi103 ceph-mon[130434]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T14:02:04.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:04 smithi003 ceph-mon[97195]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:04.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:04 smithi003 ceph-mon[97195]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T14:02:04.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:04 smithi187 ceph-mon[130973]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:04.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:04 smithi187 ceph-mon[130973]: osdmap e89: 12 total, 12 up, 12 in 2023-08-11T14:02:04.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.843+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36d810c370 con 0x7f36d8100920 2023-08-11T14:02:04.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.843+0000 7f36ce7fc700 1 -- 172.21.15.3:0/3260305991 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3204 (secure 0 0 0) 0x7f36d4099890 con 0x7f36d8100920 2023-08-11T14:02:04.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:04.845 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":364,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1691762444,"num_in_osds":12,"osd_in_since":1691762427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":95,"data_bytes":640476,"bytes_used":345837568,"bytes_avail":1151594590208,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"read_bytes_sec":10749,"write_bytes_sec":12028,"read_op_per_sec":5,"write_op_per_sec":20},"fsmap":{"epoch":27,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14502},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14514},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24454},{"filesystem_id":1,"rank":1,"name":"k","status":"up:standby-replay","gid":24464},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24499},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24500},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24511},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24527},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24532},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24569}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.3:8443/"}},"servicemap":{"epoch":20,"modified":"2023-08-11T14:02:02.166400+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-11T14:02:04.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 msgr2=0x7f36c40a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 0x7f36c40a2560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009c60 tx=0x7f36c8009260 comp rx=0 tx=0).stop 2023-08-11T14:02:04.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 msgr2=0x7f36d8198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:04.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009990 tx=0x7f36d400f690 comp rx=0 tx=0).stop 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 shutdown_connections 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c40a00a0 0x7f36c40a2560 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36d81024b0 0x7f36d819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36d8100920 0x7f36d8198840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 --2- 172.21.15.3:0/3260305991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36d8101b10 0x7f36d8198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 >> 172.21.15.3:0/3260305991 conn(0x7f36d80fbe50 msgr2=0x7f36d80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 shutdown_connections 2023-08-11T14:02:04.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:04.846+0000 7f36df9ff700 1 -- 172.21.15.3:0/3260305991 wait complete. 2023-08-11T14:02:04.866 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-11T14:02:04.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:05.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 -- 172.21.15.3:0/3775229697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da41024b0 msgr2=0x7f3da4108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 --2- 172.21.15.3:0/3775229697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da41024b0 0x7f3da4108d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00b0d0 tx=0x7f3d9c00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 -- 172.21.15.3:0/3775229697 shutdown_connections 2023-08-11T14:02:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 --2- 172.21.15.3:0/3775229697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da41024b0 0x7f3da4108d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 --2- 172.21.15.3:0/3775229697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4100920 0x7f3da4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 --2- 172.21.15.3:0/3775229697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da4101b10 0x7f3da4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 -- 172.21.15.3:0/3775229697 >> 172.21.15.3:0/3775229697 conn(0x7f3da40fbe50 msgr2=0x7f3da40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 -- 172.21.15.3:0/3775229697 shutdown_connections 2023-08-11T14:02:05.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.070+0000 7f3da97ff700 1 -- 172.21.15.3:0/3775229697 wait complete. 2023-08-11T14:02:05.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.072+0000 7f3da97ff700 1 Processor -- start 2023-08-11T14:02:05.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.072+0000 7f3da97ff700 1 -- start start 2023-08-11T14:02:05.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.072+0000 7f3da97ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:05.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.072+0000 7f3da97ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4101b10 0x7f3da4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.072+0000 7f3da97ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da41024b0 0x7f3da419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da97ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da41a0450 con 0x7f3da4100920 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da97ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da41a0590 con 0x7f3da41024b0 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da97ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da41a0720 con 0x7f3da4101b10 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41338/0 (socket says 172.21.15.3:41338) 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 -- 172.21.15.3:0/10593051 learned_addr learned my addr 172.21.15.3:0/10593051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:05.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da27fc700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4101b10 0x7f3da4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da37fe700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da41024b0 0x7f3da419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 -- 172.21.15.3:0/10593051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4101b10 msgr2=0x7f3da4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4101b10 0x7f3da4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 -- 172.21.15.3:0/10593051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da41024b0 msgr2=0x7f3da419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da41024b0 0x7f3da419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 -- 172.21.15.3:0/10593051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c009d00 con 0x7f3da4100920 2023-08-11T14:02:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.073+0000 7f3da2ffd700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e970 tx=0x7f3d9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.074+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94009790 con 0x7f3da4100920 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.074+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d9400c7e0 con 0x7f3da4100920 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.074+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da41a0a00 con 0x7f3da4100920 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.074+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94003c70 con 0x7f3da4100920 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.074+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da41a0f60 con 0x7f3da4100920 2023-08-11T14:02:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.075+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da410c190 con 0x7f3da4100920 2023-08-11T14:02:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.077+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f3d9400cc50 con 0x7f3da4100920 2023-08-11T14:02:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.078+0000 7f3d8bfff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 0x7f3d8c0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:05.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.078+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3d94014070 con 0x7f3da4100920 2023-08-11T14:02:05.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.079+0000 7f3da27fc700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 0x7f3d8c0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:05.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.079+0000 7f3da27fc700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 0x7f3d8c0a2560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009c60 tx=0x7f3d98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:05.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.081+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3d94095280 con 0x7f3da4100920 2023-08-11T14:02:05.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.289+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3da4108a70 con 0x7f3d8c0a00a0 2023-08-11T14:02:05.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.302+0000 7f3d8bfff700 1 -- 172.21.15.3:0/10593051 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422588 (secure 0 0 0) 0x7f3da4108a70 con 0x7f3d8c0a00a0 2023-08-11T14:02:05.303 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 msgr2=0x7f3d8c0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:05.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 0x7f3d8c0a2560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009c60 tx=0x7f3d98009260 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 msgr2=0x7f3da41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e970 tx=0x7f3d9400ec80 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 shutdown_connections 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d8c0a00a0 0x7f3d8c0a2560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3da41024b0 0x7f3da419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3da4100920 0x7f3da41988d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 --2- 172.21.15.3:0/10593051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3da4101b10 0x7f3da4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:05.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 >> 172.21.15.3:0/10593051 conn(0x7f3da40fbe50 msgr2=0x7f3da4107350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 shutdown_connections 2023-08-11T14:02:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:05.308+0000 7f3da97ff700 1 -- 172.21.15.3:0/10593051 wait complete. 2023-08-11T14:02:05.311 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:05.329 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-08-11T14:02:04.166679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640476,"num_objects":95,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":173,"num_read_kb":193,"num_write":393,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":323,"ondisk_log_size":323,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337732,"kb_used_data":13520,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604092,"statfs":{"total":1151940427776,"available":1151594590208,"internally_reserved":0,"allocated":13844480,"data_stored":7543484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45609,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":47,"num_read_kb":84,"num_write":164,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002114"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454489+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:01:38.454489+0000","last_peered":"2023-08-11T14:01:38.454489+0000","last_clean":"2023-08-11T14:01:38.454489+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:01:38.454489+0000","last_undegraded":"2023-08-11T14:01:38.454489+0000","last_fullsized":"2023-08-11T14:01:38.454489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:27:15.565502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453396+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:01:38.453396+0000","last_peered":"2023-08-11T14:01:38.453396+0000","last_clean":"2023-08-11T14:01:38.453396+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:01:38.453396+0000","last_undegraded":"2023-08-11T14:01:38.453396+0000","last_fullsized":"2023-08-11T14:01:38.453396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:05.487390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453350+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:01:38.453350+0000","last_peered":"2023-08-11T14:01:38.453350+0000","last_clean":"2023-08-11T14:01:38.453350+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:01:38.453350+0000","last_undegraded":"2023-08-11T14:01:38.453350+0000","last_fullsized":"2023-08-11T14:01:38.453350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:32:07.002359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453773+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:01:38.453773+0000","last_peered":"2023-08-11T14:01:38.453773+0000","last_clean":"2023-08-11T14:01:38.453773+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:01:38.453773+0000","last_undegraded":"2023-08-11T14:01:38.453773+0000","last_fullsized":"2023-08-11T14:01:38.453773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:24.933688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357698+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:01:38.357698+0000","last_peered":"2023-08-11T14:01:38.357698+0000","last_clean":"2023-08-11T14:01:38.357698+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:01:38.357698+0000","last_undegraded":"2023-08-11T14:01:38.357698+0000","last_fullsized":"2023-08-11T14:01:38.357698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:30:43.465070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453427+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:01:38.453427+0000","last_peered":"2023-08-11T14:01:38.453427+0000","last_clean":"2023-08-11T14:01:38.453427+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:01:38.453427+0000","last_undegraded":"2023-08-11T14:01:38.453427+0000","last_fullsized":"2023-08-11T14:01:38.453427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:45:09.879636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:38.454237+0000","last_change":"2023-08-11T14:02:03.242225+0000","last_active":"2023-08-11T14:02:03.242225+0000","last_peered":"2023-08-11T14:01:38.454237+0000","last_clean":"2023-08-11T14:01:38.454237+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454237+0000","last_undegraded":"2023-08-11T14:01:38.454237+0000","last_fullsized":"2023-08-11T14:01:38.454237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347768+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:02.347768+0000","last_peered":"2023-08-11T14:02:02.347768+0000","last_clean":"2023-08-11T14:02:02.347768+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:02.347768+0000","last_undegraded":"2023-08-11T14:02:02.347768+0000","last_fullsized":"2023-08-11T14:02:02.347768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:30:58.508053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454608+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:01:38.454608+0000","last_peered":"2023-08-11T14:01:38.454608+0000","last_clean":"2023-08-11T14:01:38.454608+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:01:38.454608+0000","last_undegraded":"2023-08-11T14:01:38.454608+0000","last_fullsized":"2023-08-11T14:01:38.454608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:14:36.691099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453612+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:01:38.453612+0000","last_peered":"2023-08-11T14:01:38.453612+0000","last_clean":"2023-08-11T14:01:38.453612+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:01:38.453612+0000","last_undegraded":"2023-08-11T14:01:38.453612+0000","last_fullsized":"2023-08-11T14:01:38.453612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:41.734924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453057+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:01:38.453057+0000","last_peered":"2023-08-11T14:01:38.453057+0000","last_clean":"2023-08-11T14:01:38.453057+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:01:38.453057+0000","last_undegraded":"2023-08-11T14:01:38.453057+0000","last_fullsized":"2023-08-11T14:01:38.453057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:54:14.411087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453329+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:01:38.453329+0000","last_peered":"2023-08-11T14:01:38.453329+0000","last_clean":"2023-08-11T14:01:38.453329+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:01:38.453329+0000","last_undegraded":"2023-08-11T14:01:38.453329+0000","last_fullsized":"2023-08-11T14:01:38.453329+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:45.999575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346972+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:02.346972+0000","last_peered":"2023-08-11T14:02:02.346972+0000","last_clean":"2023-08-11T14:02:02.346972+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:02.346972+0000","last_undegraded":"2023-08-11T14:02:02.346972+0000","last_fullsized":"2023-08-11T14:02:02.346972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:31.791055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454690+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:01:38.454690+0000","last_peered":"2023-08-11T14:01:38.454690+0000","last_clean":"2023-08-11T14:01:38.454690+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:01:38.454690+0000","last_undegraded":"2023-08-11T14:01:38.454690+0000","last_fullsized":"2023-08-11T14:01:38.454690+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:06:02.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345350+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:02.345350+0000","last_peered":"2023-08-11T14:02:02.345350+0000","last_clean":"2023-08-11T14:02:02.345350+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:02.345350+0000","last_undegraded":"2023-08-11T14:02:02.345350+0000","last_fullsized":"2023-08-11T14:02:02.345350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:08:36.944921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454406+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:01:38.454406+0000","last_peered":"2023-08-11T14:01:38.454406+0000","last_clean":"2023-08-11T14:01:38.454406+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:01:38.454406+0000","last_undegraded":"2023-08-11T14:01:38.454406+0000","last_fullsized":"2023-08-11T14:01:38.454406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:02:50.171886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455090+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:01:38.455090+0000","last_peered":"2023-08-11T14:01:38.455090+0000","last_clean":"2023-08-11T14:01:38.455090+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:01:38.455090+0000","last_undegraded":"2023-08-11T14:01:38.455090+0000","last_fullsized":"2023-08-11T14:01:38.455090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:39.906536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339916+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:02.339916+0000","last_peered":"2023-08-11T14:02:02.339916+0000","last_clean":"2023-08-11T14:02:02.339916+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:02.339916+0000","last_undegraded":"2023-08-11T14:02:02.339916+0000","last_fullsized":"2023-08-11T14:02:02.339916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:13:11.710971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454673+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:01:38.454673+0000","last_peered":"2023-08-11T14:01:38.454673+0000","last_clean":"2023-08-11T14:01:38.454673+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:01:38.454673+0000","last_undegraded":"2023-08-11T14:01:38.454673+0000","last_fullsized":"2023-08-11T14:01:38.454673+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:09:31.987246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453033+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:01:38.453033+0000","last_peered":"2023-08-11T14:01:38.453033+0000","last_clean":"2023-08-11T14:01:38.453033+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:01:38.453033+0000","last_undegraded":"2023-08-11T14:01:38.453033+0000","last_fullsized":"2023-08-11T14:01:38.453033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:24:09.010833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454851+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:01:38.454851+0000","last_peered":"2023-08-11T14:01:38.454851+0000","last_clean":"2023-08-11T14:01:38.454851+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:01:38.454851+0000","last_undegraded":"2023-08-11T14:01:38.454851+0000","last_fullsized":"2023-08-11T14:01:38.454851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:09.972413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453569+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:01:38.453569+0000","last_peered":"2023-08-11T14:01:38.453569+0000","last_clean":"2023-08-11T14:01:38.453569+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:01:38.453569+0000","last_undegraded":"2023-08-11T14:01:38.453569+0000","last_fullsized":"2023-08-11T14:01:38.453569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:31.879999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453610+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:01:38.453610+0000","last_peered":"2023-08-11T14:01:38.453610+0000","last_clean":"2023-08-11T14:01:38.453610+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:01:38.453610+0000","last_undegraded":"2023-08-11T14:01:38.453610+0000","last_fullsized":"2023-08-11T14:01:38.453610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:00.077438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322139+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:00.322139+0000","last_peered":"2023-08-11T14:02:00.322139+0000","last_clean":"2023-08-11T14:02:00.322139+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:00.322139+0000","last_undegraded":"2023-08-11T14:02:00.322139+0000","last_fullsized":"2023-08-11T14:02:00.322139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:53:53.482576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347387+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:02.347387+0000","last_peered":"2023-08-11T14:02:02.347387+0000","last_clean":"2023-08-11T14:02:02.347387+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:02.347387+0000","last_undegraded":"2023-08-11T14:02:02.347387+0000","last_fullsized":"2023-08-11T14:02:02.347387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:13.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.452997+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:01:38.452997+0000","last_peered":"2023-08-11T14:01:38.452997+0000","last_clean":"2023-08-11T14:01:38.452997+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:01:38.452997+0000","last_undegraded":"2023-08-11T14:01:38.452997+0000","last_fullsized":"2023-08-11T14:01:38.452997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:46.488285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320442+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:00.320442+0000","last_peered":"2023-08-11T14:02:00.320442+0000","last_clean":"2023-08-11T14:02:00.320442+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:00.320442+0000","last_undegraded":"2023-08-11T14:02:00.320442+0000","last_fullsized":"2023-08-11T14:02:00.320442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:37:57.485057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455704+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:01:38.455704+0000","last_peered":"2023-08-11T14:01:38.455704+0000","last_clean":"2023-08-11T14:01:38.455704+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:01:38.455704+0000","last_undegraded":"2023-08-11T14:01:38.455704+0000","last_fullsized":"2023-08-11T14:01:38.455704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:25.911259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454143+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:01:38.454143+0000","last_peered":"2023-08-11T14:01:38.454143+0000","last_clean":"2023-08-11T14:01:38.454143+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:01:38.454143+0000","last_undegraded":"2023-08-11T14:01:38.454143+0000","last_fullsized":"2023-08-11T14:01:38.454143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:38:00.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321984+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:01.321984+0000","last_peered":"2023-08-11T14:02:01.321984+0000","last_clean":"2023-08-11T14:02:01.321984+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:01.321984+0000","last_undegraded":"2023-08-11T14:02:01.321984+0000","last_fullsized":"2023-08-11T14:02:01.321984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:54:10.771366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357613+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:01:38.357613+0000","last_peered":"2023-08-11T14:01:38.357613+0000","last_clean":"2023-08-11T14:01:38.357613+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:01:38.357613+0000","last_undegraded":"2023-08-11T14:01:38.357613+0000","last_fullsized":"2023-08-11T14:01:38.357613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:46:18.787688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454174+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:01:38.454174+0000","last_peered":"2023-08-11T14:01:38.454174+0000","last_clean":"2023-08-11T14:01:38.454174+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:01:38.454174+0000","last_undegraded":"2023-08-11T14:01:38.454174+0000","last_fullsized":"2023-08-11T14:01:38.454174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:42:21.945068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455026+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:01:38.455026+0000","last_peered":"2023-08-11T14:01:38.455026+0000","last_clean":"2023-08-11T14:01:38.455026+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:01:38.455026+0000","last_undegraded":"2023-08-11T14:01:38.455026+0000","last_fullsized":"2023-08-11T14:01:38.455026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:26:04.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454393+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:01:38.454393+0000","last_peered":"2023-08-11T14:01:38.454393+0000","last_clean":"2023-08-11T14:01:38.454393+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:01:38.454393+0000","last_undegraded":"2023-08-11T14:01:38.454393+0000","last_fullsized":"2023-08-11T14:01:38.454393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:09.108932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454198+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:01:38.454198+0000","last_peered":"2023-08-11T14:01:38.454198+0000","last_clean":"2023-08-11T14:01:38.454198+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:01:38.454198+0000","last_undegraded":"2023-08-11T14:01:38.454198+0000","last_fullsized":"2023-08-11T14:01:38.454198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:02.471143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453242+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:01:38.453242+0000","last_peered":"2023-08-11T14:01:38.453242+0000","last_clean":"2023-08-11T14:01:38.453242+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:01:38.453242+0000","last_undegraded":"2023-08-11T14:01:38.453242+0000","last_fullsized":"2023-08-11T14:01:38.453242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:21:10.060538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321844+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:00.321844+0000","last_peered":"2023-08-11T14:02:00.321844+0000","last_clean":"2023-08-11T14:02:00.321844+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:00.321844+0000","last_undegraded":"2023-08-11T14:02:00.321844+0000","last_fullsized":"2023-08-11T14:02:00.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:29.646533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339573+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:02.339573+0000","last_peered":"2023-08-11T14:02:02.339573+0000","last_clean":"2023-08-11T14:02:02.339573+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:02.339573+0000","last_undegraded":"2023-08-11T14:02:02.339573+0000","last_fullsized":"2023-08-11T14:02:02.339573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:12.537664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454637+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:01:38.454637+0000","last_peered":"2023-08-11T14:01:38.454637+0000","last_clean":"2023-08-11T14:01:38.454637+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:01:38.454637+0000","last_undegraded":"2023-08-11T14:01:38.454637+0000","last_fullsized":"2023-08-11T14:01:38.454637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:14:38.485351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332630+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:03.332630+0000","last_peered":"2023-08-11T14:02:03.332630+0000","last_clean":"2023-08-11T14:02:03.332630+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:03.332630+0000","last_undegraded":"2023-08-11T14:02:03.332630+0000","last_fullsized":"2023-08-11T14:02:03.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347670+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:02.347670+0000","last_peered":"2023-08-11T14:02:02.347670+0000","last_clean":"2023-08-11T14:02:02.347670+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:02.347670+0000","last_undegraded":"2023-08-11T14:02:02.347670+0000","last_fullsized":"2023-08-11T14:02:02.347670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:29.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340313+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:02.340313+0000","last_peered":"2023-08-11T14:02:02.340313+0000","last_clean":"2023-08-11T14:02:02.340313+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:02.340313+0000","last_undegraded":"2023-08-11T14:02:02.340313+0000","last_fullsized":"2023-08-11T14:02:02.340313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:37:35.917226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.293732+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:01:57.293732+0000","last_peered":"2023-08-11T14:01:57.293732+0000","last_clean":"2023-08-11T14:01:57.293732+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:01:57.293732+0000","last_undegraded":"2023-08-11T14:01:57.293732+0000","last_fullsized":"2023-08-11T14:01:57.293732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:00.643436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453798+0000","last_change":"2023-08-11T14:01:33.308455+0000","last_active":"2023-08-11T14:01:38.453798+0000","last_peered":"2023-08-11T14:01:38.453798+0000","last_clean":"2023-08-11T14:01:38.453798+0000","last_became_active":"2023-08-11T14:01:33.308084+0000","last_became_peered":"2023-08-11T14:01:33.308084+0000","last_unstale":"2023-08-11T14:01:38.453798+0000","last_undegraded":"2023-08-11T14:01:38.453798+0000","last_fullsized":"2023-08-11T14:01:38.453798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:00:33.705786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453779+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:01:38.453779+0000","last_peered":"2023-08-11T14:01:38.453779+0000","last_clean":"2023-08-11T14:01:38.453779+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:01:38.453779+0000","last_undegraded":"2023-08-11T14:01:38.453779+0000","last_fullsized":"2023-08-11T14:01:38.453779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:41.727745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312606+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:01:59.312606+0000","last_peered":"2023-08-11T14:01:59.312606+0000","last_clean":"2023-08-11T14:01:59.312606+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:01:59.312606+0000","last_undegraded":"2023-08-11T14:01:59.312606+0000","last_fullsized":"2023-08-11T14:01:59.312606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:33:25.490224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347579+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:02.347579+0000","last_peered":"2023-08-11T14:02:02.347579+0000","last_clean":"2023-08-11T14:02:02.347579+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:02.347579+0000","last_undegraded":"2023-08-11T14:02:02.347579+0000","last_fullsized":"2023-08-11T14:02:02.347579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:04.886749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340016+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:02.340016+0000","last_peered":"2023-08-11T14:02:02.340016+0000","last_clean":"2023-08-11T14:02:02.340016+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:02.340016+0000","last_undegraded":"2023-08-11T14:02:02.340016+0000","last_fullsized":"2023-08-11T14:02:02.340016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:07.686519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'25","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333081+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:03.333081+0000","last_peered":"2023-08-11T14:02:03.333081+0000","last_clean":"2023-08-11T14:02:03.333081+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:03.333081+0000","last_undegraded":"2023-08-11T14:02:03.333081+0000","last_fullsized":"2023-08-11T14:02:03.333081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":48,"num_write":27,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357914+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:01:38.357914+0000","last_peered":"2023-08-11T14:01:38.357914+0000","last_clean":"2023-08-11T14:01:38.357914+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:01:38.357914+0000","last_undegraded":"2023-08-11T14:01:38.357914+0000","last_fullsized":"2023-08-11T14:01:38.357914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:21:16.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455907+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:01:38.455907+0000","last_peered":"2023-08-11T14:01:38.455907+0000","last_clean":"2023-08-11T14:01:38.455907+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:01:38.455907+0000","last_undegraded":"2023-08-11T14:01:38.455907+0000","last_fullsized":"2023-08-11T14:01:38.455907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:16:56.419641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305981+0000","last_change":"2023-08-11T14:01:32.306792+0000","last_active":"2023-08-11T14:01:58.305981+0000","last_peered":"2023-08-11T14:01:58.305981+0000","last_clean":"2023-08-11T14:01:58.305981+0000","last_became_active":"2023-08-11T14:01:32.306447+0000","last_became_peered":"2023-08-11T14:01:32.306447+0000","last_unstale":"2023-08-11T14:01:58.305981+0000","last_undegraded":"2023-08-11T14:01:58.305981+0000","last_fullsized":"2023-08-11T14:01:58.305981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:09.901925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453894+0000","last_change":"2023-08-11T14:01:37.375702+0000","last_active":"2023-08-11T14:01:38.453894+0000","last_peered":"2023-08-11T14:01:38.453894+0000","last_clean":"2023-08-11T14:01:38.453894+0000","last_became_active":"2023-08-11T14:01:37.375199+0000","last_became_peered":"2023-08-11T14:01:37.375199+0000","last_unstale":"2023-08-11T14:01:38.453894+0000","last_undegraded":"2023-08-11T14:01:38.453894+0000","last_fullsized":"2023-08-11T14:01:38.453894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:55:14.150791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.321941+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:00.321941+0000","last_peered":"2023-08-11T14:02:00.321941+0000","last_clean":"2023-08-11T14:02:00.321941+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:00.321941+0000","last_undegraded":"2023-08-11T14:02:00.321941+0000","last_fullsized":"2023-08-11T14:02:00.321941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:02:07.982583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453519+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:01:38.453519+0000","last_peered":"2023-08-11T14:01:38.453519+0000","last_clean":"2023-08-11T14:01:38.453519+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:01:38.453519+0000","last_undegraded":"2023-08-11T14:01:38.453519+0000","last_fullsized":"2023-08-11T14:01:38.453519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:50:57.895242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454218+0000","last_change":"2023-08-11T14:01:37.377532+0000","last_active":"2023-08-11T14:01:38.454218+0000","last_peered":"2023-08-11T14:01:38.454218+0000","last_clean":"2023-08-11T14:01:38.454218+0000","last_became_active":"2023-08-11T14:01:37.377142+0000","last_became_peered":"2023-08-11T14:01:37.377142+0000","last_unstale":"2023-08-11T14:01:38.454218+0000","last_undegraded":"2023-08-11T14:01:38.454218+0000","last_fullsized":"2023-08-11T14:01:38.454218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:13.506622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.321844+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:01.321844+0000","last_peered":"2023-08-11T14:02:01.321844+0000","last_clean":"2023-08-11T14:02:01.321844+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:01.321844+0000","last_undegraded":"2023-08-11T14:02:01.321844+0000","last_fullsized":"2023-08-11T14:02:01.321844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:28:27.821349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453776+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:01:38.453776+0000","last_peered":"2023-08-11T14:01:38.453776+0000","last_clean":"2023-08-11T14:01:38.453776+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:01:38.453776+0000","last_undegraded":"2023-08-11T14:01:38.453776+0000","last_fullsized":"2023-08-11T14:01:38.453776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:13:35.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.318041+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:00.318041+0000","last_peered":"2023-08-11T14:02:00.318041+0000","last_clean":"2023-08-11T14:02:00.318041+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:00.318041+0000","last_undegraded":"2023-08-11T14:02:00.318041+0000","last_fullsized":"2023-08-11T14:02:00.318041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:33:54.507816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:33.290559+0000","last_change":"2023-08-11T14:02:03.242268+0000","last_active":"2023-08-11T14:01:32.273793+0000","last_peered":"2023-08-11T14:01:32.273793+0000","last_clean":"2023-08-11T14:01:32.273793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-11T14:01:33.290559+0000","last_undegraded":"2023-08-11T14:01:33.290559+0000","last_fullsized":"2023-08-11T14:01:33.290559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454385+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:01:38.454385+0000","last_peered":"2023-08-11T14:01:38.454385+0000","last_clean":"2023-08-11T14:01:38.454385+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:01:38.454385+0000","last_undegraded":"2023-08-11T14:01:38.454385+0000","last_fullsized":"2023-08-11T14:01:38.454385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:21.288285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.230701+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:01.230701+0000","last_peered":"2023-08-11T14:02:01.230701+0000","last_clean":"2023-08-11T14:02:01.230701+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:01.230701+0000","last_undegraded":"2023-08-11T14:02:01.230701+0000","last_fullsized":"2023-08-11T14:02:01.230701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:14.365596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453956+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:01:38.453956+0000","last_peered":"2023-08-11T14:01:38.453956+0000","last_clean":"2023-08-11T14:01:38.453956+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:01:38.453956+0000","last_undegraded":"2023-08-11T14:01:38.453956+0000","last_fullsized":"2023-08-11T14:01:38.453956+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:22.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357583+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:01:38.357583+0000","last_peered":"2023-08-11T14:01:38.357583+0000","last_clean":"2023-08-11T14:01:38.357583+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:01:38.357583+0000","last_undegraded":"2023-08-11T14:01:38.357583+0000","last_fullsized":"2023-08-11T14:01:38.357583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:04:46.172105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.338227+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:02.338227+0000","last_peered":"2023-08-11T14:02:02.338227+0000","last_clean":"2023-08-11T14:02:02.338227+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:02.338227+0000","last_undegraded":"2023-08-11T14:02:02.338227+0000","last_fullsized":"2023-08-11T14:02:02.338227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:37.579833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453781+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:01:38.453781+0000","last_peered":"2023-08-11T14:01:38.453781+0000","last_clean":"2023-08-11T14:01:38.453781+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:01:38.453781+0000","last_undegraded":"2023-08-11T14:01:38.453781+0000","last_fullsized":"2023-08-11T14:01:38.453781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:37.699260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455326+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:01:38.455326+0000","last_peered":"2023-08-11T14:01:38.455326+0000","last_clean":"2023-08-11T14:01:38.455326+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:01:38.455326+0000","last_undegraded":"2023-08-11T14:01:38.455326+0000","last_fullsized":"2023-08-11T14:01:38.455326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:19:57.290710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331730+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:03.331730+0000","last_peered":"2023-08-11T14:02:03.331730+0000","last_clean":"2023-08-11T14:02:03.331730+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:03.331730+0000","last_undegraded":"2023-08-11T14:02:03.331730+0000","last_fullsized":"2023-08-11T14:02:03.331730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454372+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:01:38.454372+0000","last_peered":"2023-08-11T14:01:38.454372+0000","last_clean":"2023-08-11T14:01:38.454372+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:01:38.454372+0000","last_undegraded":"2023-08-11T14:01:38.454372+0000","last_fullsized":"2023-08-11T14:01:38.454372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:32:11.305694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454548+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:01:38.454548+0000","last_peered":"2023-08-11T14:01:38.454548+0000","last_clean":"2023-08-11T14:01:38.454548+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:01:38.454548+0000","last_undegraded":"2023-08-11T14:01:38.454548+0000","last_fullsized":"2023-08-11T14:01:38.454548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:18:38.933163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:01.319779+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:01.319779+0000","last_peered":"2023-08-11T14:02:01.319779+0000","last_clean":"2023-08-11T14:02:01.319779+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:01.319779+0000","last_undegraded":"2023-08-11T14:02:01.319779+0000","last_fullsized":"2023-08-11T14:02:01.319779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:56:44.916198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453444+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:01:38.453444+0000","last_peered":"2023-08-11T14:01:38.453444+0000","last_clean":"2023-08-11T14:01:38.453444+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:01:38.453444+0000","last_undegraded":"2023-08-11T14:01:38.453444+0000","last_fullsized":"2023-08-11T14:01:38.453444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:24:10.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455405+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:01:38.455405+0000","last_peered":"2023-08-11T14:01:38.455405+0000","last_clean":"2023-08-11T14:01:38.455405+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:01:38.455405+0000","last_undegraded":"2023-08-11T14:01:38.455405+0000","last_fullsized":"2023-08-11T14:01:38.455405+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:31:57.406400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.305859+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:01:58.305859+0000","last_peered":"2023-08-11T14:01:58.305859+0000","last_clean":"2023-08-11T14:01:58.305859+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:01:58.305859+0000","last_undegraded":"2023-08-11T14:01:58.305859+0000","last_fullsized":"2023-08-11T14:01:58.305859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:21:11.143846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454625+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:01:38.454625+0000","last_peered":"2023-08-11T14:01:38.454625+0000","last_clean":"2023-08-11T14:01:38.454625+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:01:38.454625+0000","last_undegraded":"2023-08-11T14:01:38.454625+0000","last_fullsized":"2023-08-11T14:01:38.454625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:40:26.622868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320780+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:00.320780+0000","last_peered":"2023-08-11T14:02:00.320780+0000","last_clean":"2023-08-11T14:02:00.320780+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:00.320780+0000","last_undegraded":"2023-08-11T14:02:00.320780+0000","last_fullsized":"2023-08-11T14:02:00.320780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:17:38.449276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454047+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:01:38.454047+0000","last_peered":"2023-08-11T14:01:38.454047+0000","last_clean":"2023-08-11T14:01:38.454047+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:01:38.454047+0000","last_undegraded":"2023-08-11T14:01:38.454047+0000","last_fullsized":"2023-08-11T14:01:38.454047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:47:11.994511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345212+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:02.345212+0000","last_peered":"2023-08-11T14:02:02.345212+0000","last_clean":"2023-08-11T14:02:02.345212+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:02.345212+0000","last_undegraded":"2023-08-11T14:02:02.345212+0000","last_fullsized":"2023-08-11T14:02:02.345212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:23.734051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455078+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:01:38.455078+0000","last_peered":"2023-08-11T14:01:38.455078+0000","last_clean":"2023-08-11T14:01:38.455078+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:01:38.455078+0000","last_undegraded":"2023-08-11T14:01:38.455078+0000","last_fullsized":"2023-08-11T14:01:38.455078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:14:16.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454784+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:01:38.454784+0000","last_peered":"2023-08-11T14:01:38.454784+0000","last_clean":"2023-08-11T14:01:38.454784+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:01:38.454784+0000","last_undegraded":"2023-08-11T14:01:38.454784+0000","last_fullsized":"2023-08-11T14:01:38.454784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:33.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320128+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:00.320128+0000","last_peered":"2023-08-11T14:02:00.320128+0000","last_clean":"2023-08-11T14:02:00.320128+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:00.320128+0000","last_undegraded":"2023-08-11T14:02:00.320128+0000","last_fullsized":"2023-08-11T14:02:00.320128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:50.342786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453594+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:01:38.453594+0000","last_peered":"2023-08-11T14:01:38.453594+0000","last_clean":"2023-08-11T14:01:38.453594+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:01:38.453594+0000","last_undegraded":"2023-08-11T14:01:38.453594+0000","last_fullsized":"2023-08-11T14:01:38.453594+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:05.286537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453251+0000","last_change":"2023-08-11T14:01:37.586607+0000","last_active":"2023-08-11T14:01:38.453251+0000","last_peered":"2023-08-11T14:01:38.453251+0000","last_clean":"2023-08-11T14:01:38.453251+0000","last_became_active":"2023-08-11T14:01:37.586160+0000","last_became_peered":"2023-08-11T14:01:37.586160+0000","last_unstale":"2023-08-11T14:01:38.453251+0000","last_undegraded":"2023-08-11T14:01:38.453251+0000","last_fullsized":"2023-08-11T14:01:38.453251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:11.941518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347080+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:02.347080+0000","last_peered":"2023-08-11T14:02:02.347080+0000","last_clean":"2023-08-11T14:02:02.347080+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:02.347080+0000","last_undegraded":"2023-08-11T14:02:02.347080+0000","last_fullsized":"2023-08-11T14:02:02.347080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:51.895542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.357500+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:01:38.357500+0000","last_peered":"2023-08-11T14:01:38.357500+0000","last_clean":"2023-08-11T14:01:38.357500+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:01:38.357500+0000","last_undegraded":"2023-08-11T14:01:38.357500+0000","last_fullsized":"2023-08-11T14:01:38.357500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:49:09.080233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339784+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:02.339784+0000","last_peered":"2023-08-11T14:02:02.339784+0000","last_clean":"2023-08-11T14:02:02.339784+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:02.339784+0000","last_undegraded":"2023-08-11T14:02:02.339784+0000","last_fullsized":"2023-08-11T14:02:02.339784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:41.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.322084+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:00.322084+0000","last_peered":"2023-08-11T14:02:00.322084+0000","last_clean":"2023-08-11T14:02:00.322084+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:00.322084+0000","last_undegraded":"2023-08-11T14:02:00.322084+0000","last_fullsized":"2023-08-11T14:02:00.322084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:13:22.765102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453667+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:01:38.453667+0000","last_peered":"2023-08-11T14:01:38.453667+0000","last_clean":"2023-08-11T14:01:38.453667+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:01:38.453667+0000","last_undegraded":"2023-08-11T14:01:38.453667+0000","last_fullsized":"2023-08-11T14:01:38.453667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:10:22.858975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453688+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:01:38.453688+0000","last_peered":"2023-08-11T14:01:38.453688+0000","last_clean":"2023-08-11T14:01:38.453688+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:01:38.453688+0000","last_undegraded":"2023-08-11T14:01:38.453688+0000","last_fullsized":"2023-08-11T14:01:38.453688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:18.721663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:58.470170+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:01:58.470170+0000","last_peered":"2023-08-11T14:01:58.470170+0000","last_clean":"2023-08-11T14:01:58.470170+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:01:58.470170+0000","last_undegraded":"2023-08-11T14:01:58.470170+0000","last_fullsized":"2023-08-11T14:01:58.470170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:54:36.626882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453866+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:01:38.453866+0000","last_peered":"2023-08-11T14:01:38.453866+0000","last_clean":"2023-08-11T14:01:38.453866+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:01:38.453866+0000","last_undegraded":"2023-08-11T14:01:38.453866+0000","last_fullsized":"2023-08-11T14:01:38.453866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:57.682792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453150+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:01:38.453150+0000","last_peered":"2023-08-11T14:01:38.453150+0000","last_clean":"2023-08-11T14:01:38.453150+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:01:38.453150+0000","last_undegraded":"2023-08-11T14:01:38.453150+0000","last_fullsized":"2023-08-11T14:01:38.453150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:14:41.914296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:57.294819+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:01:57.294819+0000","last_peered":"2023-08-11T14:01:57.294819+0000","last_clean":"2023-08-11T14:01:57.294819+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:01:57.294819+0000","last_undegraded":"2023-08-11T14:01:57.294819+0000","last_fullsized":"2023-08-11T14:01:57.294819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:44:25.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453769+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:01:38.453769+0000","last_peered":"2023-08-11T14:01:38.453769+0000","last_clean":"2023-08-11T14:01:38.453769+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:01:38.453769+0000","last_undegraded":"2023-08-11T14:01:38.453769+0000","last_fullsized":"2023-08-11T14:01:38.453769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:24.669978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346978+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:02.346978+0000","last_peered":"2023-08-11T14:02:02.346978+0000","last_clean":"2023-08-11T14:02:02.346978+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:02.346978+0000","last_undegraded":"2023-08-11T14:02:02.346978+0000","last_fullsized":"2023-08-11T14:02:02.346978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:00:01.572653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:59.312575+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:01:59.312575+0000","last_peered":"2023-08-11T14:01:59.312575+0000","last_clean":"2023-08-11T14:01:59.312575+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:01:59.312575+0000","last_undegraded":"2023-08-11T14:01:59.312575+0000","last_fullsized":"2023-08-11T14:01:59.312575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:06:59.189153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454907+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:01:38.454907+0000","last_peered":"2023-08-11T14:01:38.454907+0000","last_clean":"2023-08-11T14:01:38.454907+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:01:38.454907+0000","last_undegraded":"2023-08-11T14:01:38.454907+0000","last_fullsized":"2023-08-11T14:01:38.454907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:01.579515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453184+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:01:38.453184+0000","last_peered":"2023-08-11T14:01:38.453184+0000","last_clean":"2023-08-11T14:01:38.453184+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:01:38.453184+0000","last_undegraded":"2023-08-11T14:01:38.453184+0000","last_fullsized":"2023-08-11T14:01:38.453184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:17:44.247485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:02:00.320548+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:00.320548+0000","last_peered":"2023-08-11T14:02:00.320548+0000","last_clean":"2023-08-11T14:02:00.320548+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:00.320548+0000","last_undegraded":"2023-08-11T14:02:00.320548+0000","last_fullsized":"2023-08-11T14:02:00.320548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:22.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453958+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:01:38.453958+0000","last_peered":"2023-08-11T14:01:38.453958+0000","last_clean":"2023-08-11T14:01:38.453958+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:01:38.453958+0000","last_undegraded":"2023-08-11T14:01:38.453958+0000","last_fullsized":"2023-08-11T14:01:38.453958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:03.238481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453996+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:01:38.453996+0000","last_peered":"2023-08-11T14:01:38.453996+0000","last_clean":"2023-08-11T14:01:38.453996+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:01:38.453996+0000","last_undegraded":"2023-08-11T14:01:38.453996+0000","last_fullsized":"2023-08-11T14:01:38.453996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:16:23.427336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454315+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:01:38.454315+0000","last_peered":"2023-08-11T14:01:38.454315+0000","last_clean":"2023-08-11T14:01:38.454315+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:01:38.454315+0000","last_undegraded":"2023-08-11T14:01:38.454315+0000","last_fullsized":"2023-08-11T14:01:38.454315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:28:23.010885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453705+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:01:38.453705+0000","last_peered":"2023-08-11T14:01:38.453705+0000","last_clean":"2023-08-11T14:01:38.453705+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:01:38.453705+0000","last_undegraded":"2023-08-11T14:01:38.453705+0000","last_fullsized":"2023-08-11T14:01:38.453705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:23:29.103331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455259+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:01:38.455259+0000","last_peered":"2023-08-11T14:01:38.455259+0000","last_clean":"2023-08-11T14:01:38.455259+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:01:38.455259+0000","last_undegraded":"2023-08-11T14:01:38.455259+0000","last_fullsized":"2023-08-11T14:01:38.455259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:25:44.638534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454215+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:01:38.454215+0000","last_peered":"2023-08-11T14:01:38.454215+0000","last_clean":"2023-08-11T14:01:38.454215+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:01:38.454215+0000","last_undegraded":"2023-08-11T14:01:38.454215+0000","last_fullsized":"2023-08-11T14:01:38.454215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:25.219837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453473+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:01:38.453473+0000","last_peered":"2023-08-11T14:01:38.453473+0000","last_clean":"2023-08-11T14:01:38.453473+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:01:38.453473+0000","last_undegraded":"2023-08-11T14:01:38.453473+0000","last_fullsized":"2023-08-11T14:01:38.453473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:09:10.549528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454286+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:01:38.454286+0000","last_peered":"2023-08-11T14:01:38.454286+0000","last_clean":"2023-08-11T14:01:38.454286+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:01:38.454286+0000","last_undegraded":"2023-08-11T14:01:38.454286+0000","last_fullsized":"2023-08-11T14:01:38.454286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:09.697084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454819+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:01:38.454819+0000","last_peered":"2023-08-11T14:01:38.454819+0000","last_clean":"2023-08-11T14:01:38.454819+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:01:38.454819+0000","last_undegraded":"2023-08-11T14:01:38.454819+0000","last_fullsized":"2023-08-11T14:01:38.454819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:08:07.050432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455253+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:01:38.455253+0000","last_peered":"2023-08-11T14:01:38.455253+0000","last_clean":"2023-08-11T14:01:38.455253+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:01:38.455253+0000","last_undegraded":"2023-08-11T14:01:38.455253+0000","last_fullsized":"2023-08-11T14:01:38.455253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:01:53.038590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.455190+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:01:38.455190+0000","last_peered":"2023-08-11T14:01:38.455190+0000","last_clean":"2023-08-11T14:01:38.455190+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:01:38.455190+0000","last_undegraded":"2023-08-11T14:01:38.455190+0000","last_fullsized":"2023-08-11T14:01:38.455190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:01:59.437947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.454868+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:01:38.454868+0000","last_peered":"2023-08-11T14:01:38.454868+0000","last_clean":"2023-08-11T14:01:38.454868+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:01:38.454868+0000","last_undegraded":"2023-08-11T14:01:38.454868+0000","last_fullsized":"2023-08-11T14:01:38.454868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:30.935593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347343+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:02.347343+0000","last_peered":"2023-08-11T14:02:02.347343+0000","last_clean":"2023-08-11T14:02:02.347343+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:02.347343+0000","last_undegraded":"2023-08-11T14:02:02.347343+0000","last_fullsized":"2023-08-11T14:02:02.347343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:36.721470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-08-11T14:01:38.453988+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:01:38.453988+0000","last_peered":"2023-08-11T14:01:38.453988+0000","last_clean":"2023-08-11T14:01:38.453988+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:01:38.453988+0000","last_undegraded":"2023-08-11T14:01:38.453988+0000","last_fullsized":"2023-08-11T14:01:38.453988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:53.361416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49985,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":47,"num_read_kb":84,"num_write":211,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":111512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.754}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.876}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0169999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93700000000000006}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1121039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.38},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.498},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.63}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72599999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.437}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.88},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":467300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.751},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.96999999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.875},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.94399999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.89200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364834,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1057687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.498}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.622},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83099999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.753},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":467978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.90000000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.75},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65800000000000003}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.872}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":471614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.876},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.439}]},{"osd":6,"last update":"Fri Aug 11 14:01:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83499999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.89300000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":466208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.39100000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.86899999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.745}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":470986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.439}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79900000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1063084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.748},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:05 smithi103 ceph-mon[130434]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:05 smithi103 ceph-mon[130434]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 26 op/s 2023-08-11T14:02:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:05 smithi103 ceph-mon[130434]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:05 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3260305991' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:05.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:05 smithi003 ceph-mon[97195]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:05 smithi003 ceph-mon[97195]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 26 op/s 2023-08-11T14:02:05.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:05 smithi003 ceph-mon[97195]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:05 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3260305991' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:05.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:05 smithi187 ceph-mon[130973]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:05 smithi187 ceph-mon[130973]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 26 op/s 2023-08-11T14:02:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:05 smithi187 ceph-mon[130973]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:05 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3260305991' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:06.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:06 smithi103 ceph-mon[130434]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:06.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:06 smithi003 ceph-mon[97195]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:06.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:06 smithi187 ceph-mon[130973]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:07 smithi103 ceph-mon[130434]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 29 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:07.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:07 smithi003 ceph-mon[97195]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 29 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:07.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:07 smithi187 ceph-mon[130973]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 29 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:08.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:08.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 -- 172.21.15.3:0/919394345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c073380 msgr2=0x7fb81c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:08.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 --2- 172.21.15.3:0/919394345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c073380 0x7fb81c073800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009990 tx=0x7fb80c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 -- 172.21.15.3:0/919394345 shutdown_connections 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 --2- 172.21.15.3:0/919394345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 0x7fb81c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 --2- 172.21.15.3:0/919394345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c074960 0x7fb81c072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 --2- 172.21.15.3:0/919394345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c073380 0x7fb81c073800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 -- 172.21.15.3:0/919394345 >> 172.21.15.3:0/919394345 conn(0x7fb81c0fbe60 msgr2=0x7fb81c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 -- 172.21.15.3:0/919394345 shutdown_connections 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.554+0000 7fb8239ff700 1 -- 172.21.15.3:0/919394345 wait complete. 2023-08-11T14:02:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 Processor -- start 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 -- start start 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 0x7fb81c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 0x7fb81c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a03c0 con 0x7fb81c074960 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a0500 con 0x7fb81c073380 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.555+0000 7fb8239ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1a0690 con 0x7fb81c1087f0 2023-08-11T14:02:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb82179b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 0x7fb81c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb82179b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52124/0 (socket says 172.21.15.3:52124) 2023-08-11T14:02:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb82179b700 1 -- 172.21.15.3:0/3906872961 learned_addr learned my addr 172.21.15.3:0/3906872961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb821f9c700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 0x7fb81c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 -- 172.21.15.3:0/3906872961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 msgr2=0x7fb81c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 0x7fb81c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 -- 172.21.15.3:0/3906872961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 msgr2=0x7fb81c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 -- 172.21.15.3:0/3906872961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009520 con 0x7fb81c074960 2023-08-11T14:02:08.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb82179b700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.556+0000 7fb820f9a700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 0x7fb81c198d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098e0 tx=0x7fb80c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.557+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c01d070 con 0x7fb81c074960 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.557+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80c021410 con 0x7fb81c074960 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.557+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c0095f0 con 0x7fb81c074960 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.557+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c005720 con 0x7fb81c074960 2023-08-11T14:02:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.557+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c1a0ba0 con 0x7fb81c074960 2023-08-11T14:02:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.558+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c106ea0 con 0x7fb81c074960 2023-08-11T14:02:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.560+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fb80c021580 con 0x7fb81c074960 2023-08-11T14:02:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.561+0000 7fb8127fc700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 0x7fb8080a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.561+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb80c0cc970 con 0x7fb81c074960 2023-08-11T14:02:08.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.561+0000 7fb82179b700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 0x7fb8080a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:08.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.562+0000 7fb82179b700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 0x7fb8080a24e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fd0 tx=0x7fb818009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:08.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.565+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb80c0962d0 con 0x7fb81c074960 2023-08-11T14:02:08.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.770+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb81c02ccc0 con 0x7fb8080a0020 2023-08-11T14:02:08.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.783+0000 7fb8127fc700 1 -- 172.21.15.3:0/3906872961 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422489 (secure 0 0 0) 0x7fb81c02ccc0 con 0x7fb8080a0020 2023-08-11T14:02:08.788 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:08.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.788+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 msgr2=0x7fb8080a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:08.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.788+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 0x7fb8080a24e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb818009fd0 tx=0x7fb818009260 comp rx=0 tx=0).stop 2023-08-11T14:02:08.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.788+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 msgr2=0x7fb81c198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.788+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 0x7fb81c198d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098e0 tx=0x7fb80c009580 comp rx=0 tx=0).stop 2023-08-11T14:02:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.788+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 shutdown_connections 2023-08-11T14:02:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb81c1087f0 0x7fb81c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb8080a0020 0x7fb8080a24e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb81c073380 0x7fb81c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 --2- 172.21.15.3:0/3906872961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb81c074960 0x7fb81c198d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 >> 172.21.15.3:0/3906872961 conn(0x7fb81c0fbe60 msgr2=0x7fb81c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 shutdown_connections 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:08.789+0000 7fb8239ff700 1 -- 172.21.15.3:0/3906872961 wait complete. 2023-08-11T14:02:08.793 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:08.808 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-08-11T14:02:08.167957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641187,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":194,"num_read_kb":196,"num_write":406,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338116,"kb_used_data":13852,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603708,"statfs":{"total":1151940427776,"available":1151594196992,"internally_reserved":0,"allocated":14184448,"data_stored":7753421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45999,"num_objects":70,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":68,"num_read_kb":87,"num_write":156,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002299"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:38.454237+0000","last_change":"2023-08-11T14:02:03.242225+0000","last_active":"2023-08-11T14:02:03.242225+0000","last_peered":"2023-08-11T14:01:38.454237+0000","last_clean":"2023-08-11T14:01:38.454237+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454237+0000","last_undegraded":"2023-08-11T14:01:38.454237+0000","last_fullsized":"2023-08-11T14:01:38.454237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347768+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:02.347768+0000","last_peered":"2023-08-11T14:02:02.347768+0000","last_clean":"2023-08-11T14:02:02.347768+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:02.347768+0000","last_undegraded":"2023-08-11T14:02:02.347768+0000","last_fullsized":"2023-08-11T14:02:02.347768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:30:58.508053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346972+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:02.346972+0000","last_peered":"2023-08-11T14:02:02.346972+0000","last_clean":"2023-08-11T14:02:02.346972+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:02.346972+0000","last_undegraded":"2023-08-11T14:02:02.346972+0000","last_fullsized":"2023-08-11T14:02:02.346972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:31.791055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345350+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:02.345350+0000","last_peered":"2023-08-11T14:02:02.345350+0000","last_clean":"2023-08-11T14:02:02.345350+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:02.345350+0000","last_undegraded":"2023-08-11T14:02:02.345350+0000","last_fullsized":"2023-08-11T14:02:02.345350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:08:36.944921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339916+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:02.339916+0000","last_peered":"2023-08-11T14:02:02.339916+0000","last_clean":"2023-08-11T14:02:02.339916+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:02.339916+0000","last_undegraded":"2023-08-11T14:02:02.339916+0000","last_fullsized":"2023-08-11T14:02:02.339916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:13:11.710971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347387+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:02.347387+0000","last_peered":"2023-08-11T14:02:02.347387+0000","last_clean":"2023-08-11T14:02:02.347387+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:02.347387+0000","last_undegraded":"2023-08-11T14:02:02.347387+0000","last_fullsized":"2023-08-11T14:02:02.347387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:13.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.329322+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:05.329322+0000","last_peered":"2023-08-11T14:02:05.329322+0000","last_clean":"2023-08-11T14:02:05.329322+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:05.329322+0000","last_undegraded":"2023-08-11T14:02:05.329322+0000","last_fullsized":"2023-08-11T14:02:05.329322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339573+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:02.339573+0000","last_peered":"2023-08-11T14:02:02.339573+0000","last_clean":"2023-08-11T14:02:02.339573+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:02.339573+0000","last_undegraded":"2023-08-11T14:02:02.339573+0000","last_fullsized":"2023-08-11T14:02:02.339573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:12.537664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332630+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:03.332630+0000","last_peered":"2023-08-11T14:02:03.332630+0000","last_clean":"2023-08-11T14:02:03.332630+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:03.332630+0000","last_undegraded":"2023-08-11T14:02:03.332630+0000","last_fullsized":"2023-08-11T14:02:03.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347670+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:02.347670+0000","last_peered":"2023-08-11T14:02:02.347670+0000","last_clean":"2023-08-11T14:02:02.347670+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:02.347670+0000","last_undegraded":"2023-08-11T14:02:02.347670+0000","last_fullsized":"2023-08-11T14:02:02.347670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:29.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340313+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:02.340313+0000","last_peered":"2023-08-11T14:02:02.340313+0000","last_clean":"2023-08-11T14:02:02.340313+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:02.340313+0000","last_undegraded":"2023-08-11T14:02:02.340313+0000","last_fullsized":"2023-08-11T14:02:02.340313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:37:35.917226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347579+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:02.347579+0000","last_peered":"2023-08-11T14:02:02.347579+0000","last_clean":"2023-08-11T14:02:02.347579+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:02.347579+0000","last_undegraded":"2023-08-11T14:02:02.347579+0000","last_fullsized":"2023-08-11T14:02:02.347579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:04.886749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340016+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:02.340016+0000","last_peered":"2023-08-11T14:02:02.340016+0000","last_clean":"2023-08-11T14:02:02.340016+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:02.340016+0000","last_undegraded":"2023-08-11T14:02:02.340016+0000","last_fullsized":"2023-08-11T14:02:02.340016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:07.686519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'25","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333081+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:03.333081+0000","last_peered":"2023-08-11T14:02:03.333081+0000","last_clean":"2023-08-11T14:02:03.333081+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:03.333081+0000","last_undegraded":"2023-08-11T14:02:03.333081+0000","last_fullsized":"2023-08-11T14:02:03.333081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":48,"num_write":27,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.330060+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:05.330060+0000","last_peered":"2023-08-11T14:02:05.330060+0000","last_clean":"2023-08-11T14:02:05.330060+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:05.330060+0000","last_undegraded":"2023-08-11T14:02:05.330060+0000","last_fullsized":"2023-08-11T14:02:05.330060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:33.290559+0000","last_change":"2023-08-11T14:02:03.242268+0000","last_active":"2023-08-11T14:01:32.273793+0000","last_peered":"2023-08-11T14:01:32.273793+0000","last_clean":"2023-08-11T14:01:32.273793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-11T14:01:33.290559+0000","last_undegraded":"2023-08-11T14:01:33.290559+0000","last_fullsized":"2023-08-11T14:01:33.290559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.338227+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:02.338227+0000","last_peered":"2023-08-11T14:02:02.338227+0000","last_clean":"2023-08-11T14:02:02.338227+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:02.338227+0000","last_undegraded":"2023-08-11T14:02:02.338227+0000","last_fullsized":"2023-08-11T14:02:02.338227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:37.579833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513937+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:03.513937+0000","last_peered":"2023-08-11T14:02:03.513937+0000","last_clean":"2023-08-11T14:02:03.513937+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:03.513937+0000","last_undegraded":"2023-08-11T14:02:03.513937+0000","last_fullsized":"2023-08-11T14:02:03.513937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331730+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:03.331730+0000","last_peered":"2023-08-11T14:02:03.331730+0000","last_clean":"2023-08-11T14:02:03.331730+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:03.331730+0000","last_undegraded":"2023-08-11T14:02:03.331730+0000","last_fullsized":"2023-08-11T14:02:03.331730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345212+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:02.345212+0000","last_peered":"2023-08-11T14:02:02.345212+0000","last_clean":"2023-08-11T14:02:02.345212+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:02.345212+0000","last_undegraded":"2023-08-11T14:02:02.345212+0000","last_fullsized":"2023-08-11T14:02:02.345212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:23.734051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347080+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:02.347080+0000","last_peered":"2023-08-11T14:02:02.347080+0000","last_clean":"2023-08-11T14:02:02.347080+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:02.347080+0000","last_undegraded":"2023-08-11T14:02:02.347080+0000","last_fullsized":"2023-08-11T14:02:02.347080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:51.895542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339784+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:02.339784+0000","last_peered":"2023-08-11T14:02:02.339784+0000","last_clean":"2023-08-11T14:02:02.339784+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:02.339784+0000","last_undegraded":"2023-08-11T14:02:02.339784+0000","last_fullsized":"2023-08-11T14:02:02.339784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:41.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346978+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:02.346978+0000","last_peered":"2023-08-11T14:02:02.346978+0000","last_clean":"2023-08-11T14:02:02.346978+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:02.346978+0000","last_undegraded":"2023-08-11T14:02:02.346978+0000","last_fullsized":"2023-08-11T14:02:02.346978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:00:01.572653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329388+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:03.329388+0000","last_peered":"2023-08-11T14:02:03.329388+0000","last_clean":"2023-08-11T14:02:03.329388+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:03.329388+0000","last_undegraded":"2023-08-11T14:02:03.329388+0000","last_fullsized":"2023-08-11T14:02:03.329388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347343+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:02.347343+0000","last_peered":"2023-08-11T14:02:02.347343+0000","last_clean":"2023-08-11T14:02:02.347343+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:02.347343+0000","last_undegraded":"2023-08-11T14:02:02.347343+0000","last_fullsized":"2023-08-11T14:02:02.347343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:36.721470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50573,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":68,"num_read_kb":87,"num_write":223,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":151719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.5}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.746}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80900000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1121039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.38},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.498},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.63}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72599999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.437}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.88},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.85799999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.878},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":1.044}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.872},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.628}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.753},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.872},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.751},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.034},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.94499999999999995}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.69799999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":1.0089999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.98899999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":530449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.443},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.497},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.877},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.008},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.048}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.873},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.747},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.96999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48899999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:08.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:09.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.053+0000 7fca1bbff700 1 -- 172.21.15.3:0/3074085141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca141024b0 msgr2=0x7fca14108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.053+0000 7fca1bbff700 1 --2- 172.21.15.3:0/3074085141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca141024b0 0x7fca14108d60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b0d0 tx=0x7fca0c00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 -- 172.21.15.3:0/3074085141 shutdown_connections 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 --2- 172.21.15.3:0/3074085141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca141024b0 0x7fca14108d60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 --2- 172.21.15.3:0/3074085141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca14100920 0x7fca14100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 --2- 172.21.15.3:0/3074085141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 0x7fca14101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 -- 172.21.15.3:0/3074085141 >> 172.21.15.3:0/3074085141 conn(0x7fca140fbe50 msgr2=0x7fca140fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 -- 172.21.15.3:0/3074085141 shutdown_connections 2023-08-11T14:02:09.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.054+0000 7fca1bbff700 1 -- 172.21.15.3:0/3074085141 wait complete. 2023-08-11T14:02:09.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 Processor -- start 2023-08-11T14:02:09.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 -- start start 2023-08-11T14:02:09.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 0x7fca14072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca141024b0 0x7fca14073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca140739f0 con 0x7fca14100920 2023-08-11T14:02:09.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca14073b60 con 0x7fca14101b10 2023-08-11T14:02:09.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1bbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca141a0740 con 0x7fca141024b0 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.055+0000 7fca1999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41382/0 (socket says 172.21.15.3:41382) 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 -- 172.21.15.3:0/2428872503 learned_addr learned my addr 172.21.15.3:0/2428872503 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1919a700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 0x7fca14072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1a19c700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca141024b0 0x7fca14073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 -- 172.21.15.3:0/2428872503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca141024b0 msgr2=0x7fca14073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca141024b0 0x7fca14073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 -- 172.21.15.3:0/2428872503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 msgr2=0x7fca14072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 0x7fca14072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 -- 172.21.15.3:0/2428872503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009d00 con 0x7fca14100920 2023-08-11T14:02:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1999b700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca1000e970 tx=0x7fca1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:09.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10009790 con 0x7fca14100920 2023-08-11T14:02:09.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1000c7e0 con 0x7fca14100920 2023-08-11T14:02:09.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.056+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca141a0a20 con 0x7fca14100920 2023-08-11T14:02:09.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.057+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10003c70 con 0x7fca14100920 2023-08-11T14:02:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.057+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca141a0f80 con 0x7fca14100920 2023-08-11T14:02:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.058+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca14107dc0 con 0x7fca14100920 2023-08-11T14:02:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.060+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fca1000cc50 con 0x7fca14100920 2023-08-11T14:02:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.062+0000 7fca0affd700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 0x7fca000a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.062+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fca10014070 con 0x7fca14100920 2023-08-11T14:02:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.062+0000 7fca1919a700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 0x7fca000a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.063+0000 7fca1919a700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 0x7fca000a2560 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca04009c60 tx=0x7fca04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.066+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fca10095150 con 0x7fca14100920 2023-08-11T14:02:09.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.271+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca1402ce20 con 0x7fca000a00a0 2023-08-11T14:02:09.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.284+0000 7fca0affd700 1 -- 172.21.15.3:0/2428872503 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422489 (secure 0 0 0) 0x7fca1402ce20 con 0x7fca000a00a0 2023-08-11T14:02:09.289 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:09.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 msgr2=0x7fca000a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 0x7fca000a2560 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fca04009c60 tx=0x7fca04009260 comp rx=0 tx=0).stop 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 msgr2=0x7fca140746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca1000e970 tx=0x7fca1000ec80 comp rx=0 tx=0).stop 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 shutdown_connections 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca141024b0 0x7fca14073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca14100920 0x7fca140746d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca000a00a0 0x7fca000a2560 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 --2- 172.21.15.3:0/2428872503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca14101b10 0x7fca14072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 >> 172.21.15.3:0/2428872503 conn(0x7fca140fbe50 msgr2=0x7fca14104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:09.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 shutdown_connections 2023-08-11T14:02:09.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.289+0000 7fca1bbff700 1 -- 172.21.15.3:0/2428872503 wait complete. 2023-08-11T14:02:09.293 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:09.309 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-08-11T14:02:08.167957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641187,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":194,"num_read_kb":196,"num_write":406,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338116,"kb_used_data":13852,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603708,"statfs":{"total":1151940427776,"available":1151594196992,"internally_reserved":0,"allocated":14184448,"data_stored":7753421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45999,"num_objects":70,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":68,"num_read_kb":87,"num_write":156,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002299"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:38.454237+0000","last_change":"2023-08-11T14:02:03.242225+0000","last_active":"2023-08-11T14:02:03.242225+0000","last_peered":"2023-08-11T14:01:38.454237+0000","last_clean":"2023-08-11T14:01:38.454237+0000","last_became_active":"2023-08-11T14:01:33.311835+0000","last_became_peered":"2023-08-11T14:01:33.311835+0000","last_unstale":"2023-08-11T14:01:38.454237+0000","last_undegraded":"2023-08-11T14:01:38.454237+0000","last_fullsized":"2023-08-11T14:01:38.454237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347768+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:02.347768+0000","last_peered":"2023-08-11T14:02:02.347768+0000","last_clean":"2023-08-11T14:02:02.347768+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:02.347768+0000","last_undegraded":"2023-08-11T14:02:02.347768+0000","last_fullsized":"2023-08-11T14:02:02.347768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:30:58.508053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346972+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:02.346972+0000","last_peered":"2023-08-11T14:02:02.346972+0000","last_clean":"2023-08-11T14:02:02.346972+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:02.346972+0000","last_undegraded":"2023-08-11T14:02:02.346972+0000","last_fullsized":"2023-08-11T14:02:02.346972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:16:31.791055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345350+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:02.345350+0000","last_peered":"2023-08-11T14:02:02.345350+0000","last_clean":"2023-08-11T14:02:02.345350+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:02.345350+0000","last_undegraded":"2023-08-11T14:02:02.345350+0000","last_fullsized":"2023-08-11T14:02:02.345350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:08:36.944921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339916+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:02.339916+0000","last_peered":"2023-08-11T14:02:02.339916+0000","last_clean":"2023-08-11T14:02:02.339916+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:02.339916+0000","last_undegraded":"2023-08-11T14:02:02.339916+0000","last_fullsized":"2023-08-11T14:02:02.339916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:13:11.710971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347387+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:02.347387+0000","last_peered":"2023-08-11T14:02:02.347387+0000","last_clean":"2023-08-11T14:02:02.347387+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:02.347387+0000","last_undegraded":"2023-08-11T14:02:02.347387+0000","last_fullsized":"2023-08-11T14:02:02.347387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:10:13.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.329322+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:05.329322+0000","last_peered":"2023-08-11T14:02:05.329322+0000","last_clean":"2023-08-11T14:02:05.329322+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:05.329322+0000","last_undegraded":"2023-08-11T14:02:05.329322+0000","last_fullsized":"2023-08-11T14:02:05.329322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339573+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:02.339573+0000","last_peered":"2023-08-11T14:02:02.339573+0000","last_clean":"2023-08-11T14:02:02.339573+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:02.339573+0000","last_undegraded":"2023-08-11T14:02:02.339573+0000","last_fullsized":"2023-08-11T14:02:02.339573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:42:12.537664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332630+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:03.332630+0000","last_peered":"2023-08-11T14:02:03.332630+0000","last_clean":"2023-08-11T14:02:03.332630+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:03.332630+0000","last_undegraded":"2023-08-11T14:02:03.332630+0000","last_fullsized":"2023-08-11T14:02:03.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347670+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:02.347670+0000","last_peered":"2023-08-11T14:02:02.347670+0000","last_clean":"2023-08-11T14:02:02.347670+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:02.347670+0000","last_undegraded":"2023-08-11T14:02:02.347670+0000","last_fullsized":"2023-08-11T14:02:02.347670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:49:29.812104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340313+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:02.340313+0000","last_peered":"2023-08-11T14:02:02.340313+0000","last_clean":"2023-08-11T14:02:02.340313+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:02.340313+0000","last_undegraded":"2023-08-11T14:02:02.340313+0000","last_fullsized":"2023-08-11T14:02:02.340313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:37:35.917226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347579+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:02.347579+0000","last_peered":"2023-08-11T14:02:02.347579+0000","last_clean":"2023-08-11T14:02:02.347579+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:02.347579+0000","last_undegraded":"2023-08-11T14:02:02.347579+0000","last_fullsized":"2023-08-11T14:02:02.347579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:04.886749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.340016+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:02.340016+0000","last_peered":"2023-08-11T14:02:02.340016+0000","last_clean":"2023-08-11T14:02:02.340016+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:02.340016+0000","last_undegraded":"2023-08-11T14:02:02.340016+0000","last_fullsized":"2023-08-11T14:02:02.340016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:48:07.686519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'25","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333081+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:03.333081+0000","last_peered":"2023-08-11T14:02:03.333081+0000","last_clean":"2023-08-11T14:02:03.333081+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:03.333081+0000","last_undegraded":"2023-08-11T14:02:03.333081+0000","last_fullsized":"2023-08-11T14:02:03.333081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43761,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":48,"num_write":27,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.330060+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:05.330060+0000","last_peered":"2023-08-11T14:02:05.330060+0000","last_clean":"2023-08-11T14:02:05.330060+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:05.330060+0000","last_undegraded":"2023-08-11T14:02:05.330060+0000","last_fullsized":"2023-08-11T14:02:05.330060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":2,"reported_epoch":88,"state":"peering","last_fresh":"2023-08-11T14:01:33.290559+0000","last_change":"2023-08-11T14:02:03.242268+0000","last_active":"2023-08-11T14:01:32.273793+0000","last_peered":"2023-08-11T14:01:32.273793+0000","last_clean":"2023-08-11T14:01:32.273793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-11T14:01:33.290559+0000","last_undegraded":"2023-08-11T14:01:33.290559+0000","last_fullsized":"2023-08-11T14:01:33.290559+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.338227+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:02.338227+0000","last_peered":"2023-08-11T14:02:02.338227+0000","last_clean":"2023-08-11T14:02:02.338227+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:02.338227+0000","last_undegraded":"2023-08-11T14:02:02.338227+0000","last_fullsized":"2023-08-11T14:02:02.338227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:37.579833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513937+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:03.513937+0000","last_peered":"2023-08-11T14:02:03.513937+0000","last_clean":"2023-08-11T14:02:03.513937+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:03.513937+0000","last_undegraded":"2023-08-11T14:02:03.513937+0000","last_fullsized":"2023-08-11T14:02:03.513937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331730+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:03.331730+0000","last_peered":"2023-08-11T14:02:03.331730+0000","last_clean":"2023-08-11T14:02:03.331730+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:03.331730+0000","last_undegraded":"2023-08-11T14:02:03.331730+0000","last_fullsized":"2023-08-11T14:02:03.331730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.345212+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:02.345212+0000","last_peered":"2023-08-11T14:02:02.345212+0000","last_clean":"2023-08-11T14:02:02.345212+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:02.345212+0000","last_undegraded":"2023-08-11T14:02:02.345212+0000","last_fullsized":"2023-08-11T14:02:02.345212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:23.734051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347080+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:02.347080+0000","last_peered":"2023-08-11T14:02:02.347080+0000","last_clean":"2023-08-11T14:02:02.347080+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:02.347080+0000","last_undegraded":"2023-08-11T14:02:02.347080+0000","last_fullsized":"2023-08-11T14:02:02.347080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:51.895542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.339784+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:02.339784+0000","last_peered":"2023-08-11T14:02:02.339784+0000","last_clean":"2023-08-11T14:02:02.339784+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:02.339784+0000","last_undegraded":"2023-08-11T14:02:02.339784+0000","last_fullsized":"2023-08-11T14:02:02.339784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:17:41.171008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.346978+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:02.346978+0000","last_peered":"2023-08-11T14:02:02.346978+0000","last_clean":"2023-08-11T14:02:02.346978+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:02.346978+0000","last_undegraded":"2023-08-11T14:02:02.346978+0000","last_fullsized":"2023-08-11T14:02:02.346978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:00:01.572653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329388+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:03.329388+0000","last_peered":"2023-08-11T14:02:03.329388+0000","last_clean":"2023-08-11T14:02:03.329388+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:03.329388+0000","last_undegraded":"2023-08-11T14:02:03.329388+0000","last_fullsized":"2023-08-11T14:02:03.329388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-08-11T14:02:02.347343+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:02.347343+0000","last_peered":"2023-08-11T14:02:02.347343+0000","last_clean":"2023-08-11T14:02:02.347343+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:02.347343+0000","last_undegraded":"2023-08-11T14:02:02.347343+0000","last_fullsized":"2023-08-11T14:02:02.347343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:36.721470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50573,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":68,"num_read_kb":87,"num_write":223,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":151719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.5}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.746}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80900000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678037,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1121039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.38},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.498},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.67200000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.63}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72599999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":484582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.437}]},{"osd":4,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.71199999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.90200000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.88},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.85799999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.878},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":1.044}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.872},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.628}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.87},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80400000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.56699999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.753},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.872},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.751},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.034},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.94499999999999995}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.69799999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":1.0089999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.98899999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":530449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.443},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.497},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.877},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.008},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.048}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.873},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.747},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.96999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48899999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:09.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-08-11T14:02:09.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T14:01:38.454237+0000', 'last_change': '2023-08-11T14:02:03.242225+0000', 'last_active': '2023-08-11T14:02:03.242225+0000', 'last_peered': '2023-08-11T14:01:38.454237+0000', 'last_clean': '2023-08-11T14:01:38.454237+0000', 'last_became_active': '2023-08-11T14:01:33.311835+0000', 'last_became_peered': '2023-08-11T14:01:33.311835+0000', 'last_unstale': '2023-08-11T14:01:38.454237+0000', 'last_undegraded': '2023-08-11T14:01:38.454237+0000', 'last_fullsized': '2023-08-11T14:01:38.454237+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_clean_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 5, 10], 'acting': [9, 5, 10], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-08-11T14:02:09.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-08-11T14:02:09.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2023-08-11T14:01:33.290559+0000', 'last_change': '2023-08-11T14:02:03.242268+0000', 'last_active': '2023-08-11T14:01:32.273793+0000', 'last_peered': '2023-08-11T14:01:32.273793+0000', 'last_clean': '2023-08-11T14:01:32.273793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-11T14:01:33.290559+0000', 'last_undegraded': '2023-08-11T14:01:33.290559+0000', 'last_fullsized': '2023-08-11T14:01:33.290559+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'last_clean_scrub_stamp': '2023-08-11T14:01:32.273793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9, 8, 1], 'acting': [9, 8, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-08-11T14:02:09.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-11T14:02:09.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 -- 172.21.15.3:0/1699393176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0940101b10 msgr2=0x7f0940101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 --2- 172.21.15.3:0/1699393176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0940101b10 0x7f0940101f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09300099c0 tx=0x7f0930009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 -- 172.21.15.3:0/1699393176 shutdown_connections 2023-08-11T14:02:09.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 --2- 172.21.15.3:0/1699393176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09401024b0 0x7f0940108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 --2- 172.21.15.3:0/1699393176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940100920 0x7f0940100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 --2- 172.21.15.3:0/1699393176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0940101b10 0x7f0940101f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 -- 172.21.15.3:0/1699393176 >> 172.21.15.3:0/1699393176 conn(0x7f09400fbe50 msgr2=0x7f09400fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.553+0000 7f0947dff700 1 -- 172.21.15.3:0/1699393176 shutdown_connections 2023-08-11T14:02:09.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.554+0000 7f0947dff700 1 -- 172.21.15.3:0/1699393176 wait complete. 2023-08-11T14:02:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 Processor -- start 2023-08-11T14:02:09.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 -- start start 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940101b10 0x7f0940198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 0x7f094019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401a0390 con 0x7f09401024b0 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401a04d0 con 0x7f0940100920 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0947dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401a0660 con 0x7f0940101b10 2023-08-11T14:02:09.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0945b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f094639c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 0x7f094019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0945b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52168/0 (socket says 172.21.15.3:52168) 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f0945b9b700 1 -- 172.21.15.3:0/1618964040 learned_addr learned my addr 172.21.15.3:0/1618964040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.555+0000 7f094539a700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940101b10 0x7f0940198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 -- 172.21.15.3:0/1618964040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940101b10 msgr2=0x7f0940198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940101b10 0x7f0940198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 -- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 msgr2=0x7f094019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 0x7f094019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 -- 172.21.15.3:0/1618964040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300095f0 con 0x7f0940100920 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f094639c700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 0x7f094019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0945b9b700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e970 tx=0x7f093c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c009790 con 0x7f0940100920 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09401a0940 con 0x7f0940100920 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f093c00c7e0 con 0x7f0940100920 2023-08-11T14:02:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.556+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c003c70 con 0x7f0940100920 2023-08-11T14:02:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.557+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09401a0ea0 con 0x7f0940100920 2023-08-11T14:02:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.557+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094010c180 con 0x7f0940100920 2023-08-11T14:02:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.560+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f093c00cc50 con 0x7f0940100920 2023-08-11T14:02:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.561+0000 7f0936ffd700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 0x7f092c0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.561+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f093c014070 con 0x7f0940100920 2023-08-11T14:02:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.561+0000 7f094539a700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 0x7f092c0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:09.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.562+0000 7f094539a700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 0x7f092c0a2560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f093000b580 tx=0x7f0930005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:09.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.563+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f093c095260 con 0x7f0940100920 2023-08-11T14:02:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:09 smithi103 ceph-mon[130434]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 27 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:09.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:09 smithi003 ceph-mon[97195]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 27 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:09.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:09 smithi187 ceph-mon[130973]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 27 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:09.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.858+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f09401a0ad0 con 0x7f0940100920 2023-08-11T14:02:09.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.859+0000 7f0936ffd700 1 -- 172.21.15.3:0/1618964040 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3368 (secure 0 0 0) 0x7f09401a0ad0 con 0x7f0940100920 2023-08-11T14:02:09.860 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:09.860 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":369,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1691762444,"num_in_osds":12,"osd_in_since":1691762427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":102,"data_bytes":641187,"bytes_used":346230784,"bytes_avail":1151594196992,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":8,"recovering_keys_per_sec":1,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"read_bytes_sec":11132,"write_bytes_sec":11900,"read_op_per_sec":8,"write_op_per_sec":19},"fsmap":{"epoch":27,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14502},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14514},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24454},{"filesystem_id":1,"rank":1,"name":"k","status":"up:standby-replay","gid":24464},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24499},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24500},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24511},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24527},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24532},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24569}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.3:8443/"}},"servicemap":{"epoch":21,"modified":"2023-08-11T14:02:06.167599+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-08-11T14:02:09.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 msgr2=0x7f092c0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 0x7f092c0a2560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f093000b580 tx=0x7f0930005fb0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 msgr2=0x7f0940198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:09.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e970 tx=0x7f093c00ec80 comp rx=0 tx=0).stop 2023-08-11T14:02:09.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 shutdown_connections 2023-08-11T14:02:09.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09401024b0 0x7f094019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f092c0a00a0 0x7f092c0a2560 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0940100920 0x7f0940198810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 --2- 172.21.15.3:0/1618964040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0940101b10 0x7f0940198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:09.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 >> 172.21.15.3:0/1618964040 conn(0x7f09400fbe50 msgr2=0x7f09400fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:09.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 shutdown_connections 2023-08-11T14:02:09.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:09.862+0000 7f0947dff700 1 -- 172.21.15.3:0/1618964040 wait complete. 2023-08-11T14:02:09.881 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2023-08-11T14:02:09.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:10.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 -- 172.21.15.3:0/3169347208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb801024b0 msgr2=0x7feb80108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 --2- 172.21.15.3:0/3169347208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb801024b0 0x7feb80108d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb78009f90 tx=0x7feb7800b330 comp rx=0 tx=0).stop 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 -- 172.21.15.3:0/3169347208 shutdown_connections 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 --2- 172.21.15.3:0/3169347208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb801024b0 0x7feb80108d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 --2- 172.21.15.3:0/3169347208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80100920 0x7feb80100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 --2- 172.21.15.3:0/3169347208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb80101b10 0x7feb80101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 -- 172.21.15.3:0/3169347208 >> 172.21.15.3:0/3169347208 conn(0x7feb800fbe50 msgr2=0x7feb800fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:10.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 -- 172.21.15.3:0/3169347208 shutdown_connections 2023-08-11T14:02:10.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.085+0000 7feb84dff700 1 -- 172.21.15.3:0/3169347208 wait complete. 2023-08-11T14:02:10.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 Processor -- start 2023-08-11T14:02:10.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 -- start start 2023-08-11T14:02:10.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:10.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101b10 0x7feb80198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:10.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 0x7feb8019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801a0490 con 0x7feb801024b0 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801a05d0 con 0x7feb80101b10 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.086+0000 7feb84dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb801a0760 con 0x7feb80100920 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101b10 0x7feb80198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35968/0 (socket says 172.21.15.3:35968) 2023-08-11T14:02:10.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 -- 172.21.15.3:0/437928638 learned_addr learned my addr 172.21.15.3:0/437928638 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7ed9d700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 0x7feb8019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 -- 172.21.15.3:0/437928638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101b10 msgr2=0x7feb80198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:10.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101b10 0x7feb80198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 -- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 msgr2=0x7feb8019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 0x7feb8019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 -- 172.21.15.3:0/437928638 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb78009c40 con 0x7feb80100920 2023-08-11T14:02:10.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7ed9d700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 0x7feb8019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:10.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.087+0000 7feb7e59c700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e970 tx=0x7feb7000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:10.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.088+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70009790 con 0x7feb80100920 2023-08-11T14:02:10.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.088+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb7000c7c0 con 0x7feb80100920 2023-08-11T14:02:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.088+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb801a0a40 con 0x7feb80100920 2023-08-11T14:02:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.088+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70003c20 con 0x7feb80100920 2023-08-11T14:02:10.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.088+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb801a0f50 con 0x7feb80100920 2023-08-11T14:02:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.089+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8010c190 con 0x7feb80100920 2023-08-11T14:02:10.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.091+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7feb7000c930 con 0x7feb80100920 2023-08-11T14:02:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.092+0000 7feb6f7fe700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 0x7feb680a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.092+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7feb70014070 con 0x7feb80100920 2023-08-11T14:02:10.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.093+0000 7feb7dd9b700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 0x7feb680a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:10.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.093+0000 7feb7dd9b700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 0x7feb680a24e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:10.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.094+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7feb700952c0 con 0x7feb80100920 2023-08-11T14:02:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.301+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb80108a60 con 0x7feb680a0020 2023-08-11T14:02:10.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.314+0000 7feb6f7fe700 1 -- 172.21.15.3:0/437928638 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422275 (secure 0 0 0) 0x7feb80108a60 con 0x7feb680a0020 2023-08-11T14:02:10.319 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 msgr2=0x7feb680a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 0x7feb680a24e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb74005fd0 tx=0x7feb74009310 comp rx=0 tx=0).stop 2023-08-11T14:02:10.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 msgr2=0x7feb80198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e970 tx=0x7feb7000ec80 comp rx=0 tx=0).stop 2023-08-11T14:02:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 shutdown_connections 2023-08-11T14:02:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb801024b0 0x7feb8019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb80100920 0x7feb80198910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7feb680a0020 0x7feb680a24e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 --2- 172.21.15.3:0/437928638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101b10 0x7feb80198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:10.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.319+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 >> 172.21.15.3:0/437928638 conn(0x7feb800fbe50 msgr2=0x7feb80107340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:10.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.320+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 shutdown_connections 2023-08-11T14:02:10.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:10.320+0000 7feb84dff700 1 -- 172.21.15.3:0/437928638 wait complete. 2023-08-11T14:02:10.325 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-08-11T14:02:10.168527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649403,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":225,"num_read_kb":230,"num_write":414,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":424,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338152,"kb_used_data":13900,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603672,"statfs":{"total":1151940427776,"available":1151594160128,"internally_reserved":0,"allocated":14233600,"data_stored":7788885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50924,"num_objects":29,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":82,"num_read_kb":105,"num_write":80,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002369"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.339952+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:02:03.339952+0000","last_peered":"2023-08-11T14:02:03.339952+0000","last_clean":"2023-08-11T14:02:03.339952+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:02:03.339952+0000","last_undegraded":"2023-08-11T14:02:03.339952+0000","last_fullsized":"2023-08-11T14:02:03.339952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:07:02.851636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514145+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:03.514145+0000","last_peered":"2023-08-11T14:02:03.514145+0000","last_clean":"2023-08-11T14:02:03.514145+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:03.514145+0000","last_undegraded":"2023-08-11T14:02:03.514145+0000","last_fullsized":"2023-08-11T14:02:03.514145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:30.192305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514351+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:03.514351+0000","last_peered":"2023-08-11T14:02:03.514351+0000","last_clean":"2023-08-11T14:02:03.514351+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:03.514351+0000","last_undegraded":"2023-08-11T14:02:03.514351+0000","last_fullsized":"2023-08-11T14:02:03.514351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:53:22.056513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330963+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:03.330963+0000","last_peered":"2023-08-11T14:02:03.330963+0000","last_clean":"2023-08-11T14:02:03.330963+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:03.330963+0000","last_undegraded":"2023-08-11T14:02:03.330963+0000","last_fullsized":"2023-08-11T14:02:03.330963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:51.334309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:06.330693+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:06.330693+0000","last_peered":"2023-08-11T14:02:06.330693+0000","last_clean":"2023-08-11T14:02:06.330693+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:06.330693+0000","last_undegraded":"2023-08-11T14:02:06.330693+0000","last_fullsized":"2023-08-11T14:02:06.330693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:49.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513358+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:03.513358+0000","last_peered":"2023-08-11T14:02:03.513358+0000","last_clean":"2023-08-11T14:02:03.513358+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:03.513358+0000","last_undegraded":"2023-08-11T14:02:03.513358+0000","last_fullsized":"2023-08-11T14:02:03.513358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:08.181790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.329322+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:05.329322+0000","last_peered":"2023-08-11T14:02:05.329322+0000","last_clean":"2023-08-11T14:02:05.329322+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:05.329322+0000","last_undegraded":"2023-08-11T14:02:05.329322+0000","last_fullsized":"2023-08-11T14:02:05.329322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330576+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:03.330576+0000","last_peered":"2023-08-11T14:02:03.330576+0000","last_clean":"2023-08-11T14:02:03.330576+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:03.330576+0000","last_undegraded":"2023-08-11T14:02:03.330576+0000","last_fullsized":"2023-08-11T14:02:03.330576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:21.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.333138+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:08.333138+0000","last_peered":"2023-08-11T14:02:08.333138+0000","last_clean":"2023-08-11T14:02:08.333138+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:08.333138+0000","last_undegraded":"2023-08-11T14:02:08.333138+0000","last_fullsized":"2023-08-11T14:02:08.333138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514217+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:03.514217+0000","last_peered":"2023-08-11T14:02:03.514217+0000","last_clean":"2023-08-11T14:02:03.514217+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:03.514217+0000","last_undegraded":"2023-08-11T14:02:03.514217+0000","last_fullsized":"2023-08-11T14:02:03.514217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:49:01.488839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330205+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:03.330205+0000","last_peered":"2023-08-11T14:02:03.330205+0000","last_clean":"2023-08-11T14:02:03.330205+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:03.330205+0000","last_undegraded":"2023-08-11T14:02:03.330205+0000","last_fullsized":"2023-08-11T14:02:03.330205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:32.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513781+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:03.513781+0000","last_peered":"2023-08-11T14:02:03.513781+0000","last_clean":"2023-08-11T14:02:03.513781+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:03.513781+0000","last_undegraded":"2023-08-11T14:02:03.513781+0000","last_fullsized":"2023-08-11T14:02:03.513781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:08.653787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330092+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:03.330092+0000","last_peered":"2023-08-11T14:02:03.330092+0000","last_clean":"2023-08-11T14:02:03.330092+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:03.330092+0000","last_undegraded":"2023-08-11T14:02:03.330092+0000","last_fullsized":"2023-08-11T14:02:03.330092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:16.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'30","reported_seq":74,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.345527+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:08.345527+0000","last_peered":"2023-08-11T14:02:08.345527+0000","last_clean":"2023-08-11T14:02:08.345527+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:08.345527+0000","last_undegraded":"2023-08-11T14:02:08.345527+0000","last_fullsized":"2023-08-11T14:02:08.345527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51977,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":62,"num_write":32,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.330060+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:05.330060+0000","last_peered":"2023-08-11T14:02:05.330060+0000","last_clean":"2023-08-11T14:02:05.330060+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:05.330060+0000","last_undegraded":"2023-08-11T14:02:05.330060+0000","last_fullsized":"2023-08-11T14:02:05.330060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515481+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:02:03.515481+0000","last_peered":"2023-08-11T14:02:03.515481+0000","last_clean":"2023-08-11T14:02:03.515481+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:02:03.515481+0000","last_undegraded":"2023-08-11T14:02:03.515481+0000","last_fullsized":"2023-08-11T14:02:03.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:07:44.577236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330437+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:03.330437+0000","last_peered":"2023-08-11T14:02:03.330437+0000","last_clean":"2023-08-11T14:02:03.330437+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:03.330437+0000","last_undegraded":"2023-08-11T14:02:03.330437+0000","last_fullsized":"2023-08-11T14:02:03.330437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:52.907857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513937+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:03.513937+0000","last_peered":"2023-08-11T14:02:03.513937+0000","last_clean":"2023-08-11T14:02:03.513937+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:03.513937+0000","last_undegraded":"2023-08-11T14:02:03.513937+0000","last_fullsized":"2023-08-11T14:02:03.513937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:07.332219+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:07.332219+0000","last_peered":"2023-08-11T14:02:07.332219+0000","last_clean":"2023-08-11T14:02:07.332219+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:07.332219+0000","last_undegraded":"2023-08-11T14:02:07.332219+0000","last_fullsized":"2023-08-11T14:02:07.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330920+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:03.330920+0000","last_peered":"2023-08-11T14:02:03.330920+0000","last_clean":"2023-08-11T14:02:03.330920+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:03.330920+0000","last_undegraded":"2023-08-11T14:02:03.330920+0000","last_fullsized":"2023-08-11T14:02:03.330920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:08:56.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514463+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:03.514463+0000","last_peered":"2023-08-11T14:02:03.514463+0000","last_clean":"2023-08-11T14:02:03.514463+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:03.514463+0000","last_undegraded":"2023-08-11T14:02:03.514463+0000","last_fullsized":"2023-08-11T14:02:03.514463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:19.003199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330601+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:03.330601+0000","last_peered":"2023-08-11T14:02:03.330601+0000","last_clean":"2023-08-11T14:02:03.330601+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:03.330601+0000","last_undegraded":"2023-08-11T14:02:03.330601+0000","last_fullsized":"2023-08-11T14:02:03.330601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:32:17.015669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513303+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:03.513303+0000","last_peered":"2023-08-11T14:02:03.513303+0000","last_clean":"2023-08-11T14:02:03.513303+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:03.513303+0000","last_undegraded":"2023-08-11T14:02:03.513303+0000","last_fullsized":"2023-08-11T14:02:03.513303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:16.532299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329388+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:03.329388+0000","last_peered":"2023-08-11T14:02:03.329388+0000","last_clean":"2023-08-11T14:02:03.329388+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:03.329388+0000","last_undegraded":"2023-08-11T14:02:03.329388+0000","last_fullsized":"2023-08-11T14:02:03.329388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514220+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:03.514220+0000","last_peered":"2023-08-11T14:02:03.514220+0000","last_clean":"2023-08-11T14:02:03.514220+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:03.514220+0000","last_undegraded":"2023-08-11T14:02:03.514220+0000","last_fullsized":"2023-08-11T14:02:03.514220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:06.901263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58789,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":99,"num_read_kb":121,"num_write":231,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":168151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.5}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.746}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80900000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1129255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.194},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.0309999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.147}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.01}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.0880000000000001}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.1100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.99299999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":1.234},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.129}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":494098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1,"15min":1},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97599999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.73099999999999998,"15min":0.75700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93799999999999994,"5min":1.123,"15min":1.123},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.72899999999999998,"15min":0.755},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.878,"15min":0.90700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.075,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.93400000000000005,"15min":0.96499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0289999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.93100000000000005,"15min":0.96399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.90600000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1,"15min":1.038},"min":{"1min":0.54600000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1679999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":1.006,"15min":1.042},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1240000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1759999999999999,"5min":1.484,"15min":1.484},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.103,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.92200000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.504},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.85799999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.878},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.99399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":1.044}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.872},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.628}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:01:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.133}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":1.117}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":1.167}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.151}]},{"osd":9,"last update":"Fri Aug 11 14:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2130000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.1000000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.175},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.212}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":538240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.503}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.37}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.873}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.90200000000000002},"min":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95599999999999996,"5min":3.194,"15min":3.194},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2370000000000001,"5min":1.393,"15min":1.393},"last":0.70599999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:01:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.47999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.872},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.751},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.034},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.94499999999999995}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.69799999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":1.0089999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.98899999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.79700000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":530449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.443},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.497},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.75}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.877},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.008},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.048}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.873},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.747},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.90800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:01:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78500000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.96999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48899999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:10 smithi103 ceph-mon[130434]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:10 smithi103 ceph-mon[130434]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:10 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1618964040' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:10 smithi003 ceph-mon[97195]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:10 smithi003 ceph-mon[97195]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1618964040' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:10.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:10 smithi187 ceph-mon[130973]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:10 smithi187 ceph-mon[130973]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:10.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:10 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1618964040' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-08-11T14:02:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:11 smithi103 ceph-mon[130434]: pgmap v293: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:11 smithi103 ceph-mon[130434]: from='client.24892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:11.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:11 smithi003 ceph-mon[97195]: pgmap v293: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:11.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:11 smithi003 ceph-mon[97195]: from='client.24892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:11.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:11 smithi187 ceph-mon[130973]: pgmap v293: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 20 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:11.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:11 smithi187 ceph-mon[130973]: from='client.24892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:13.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:02:13.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:13 smithi003 ceph-mon[97195]: pgmap v294: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:13.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:13 smithi187 ceph-mon[130973]: pgmap v294: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1095540779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc100930 msgr2=0x7fd0dc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1095540779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc100930 0x7fd0dc100d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009990 tx=0x7fd0cc009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1095540779 shutdown_connections 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1095540779 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc1024c0 0x7fd0dc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1095540779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc100930 0x7fd0dc100d50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1095540779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 0x7fd0dc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.564+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1095540779 >> 172.21.15.3:0/1095540779 conn(0x7fd0dc0fbe60 msgr2=0x7fd0dc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.565+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1095540779 shutdown_connections 2023-08-11T14:02:13.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.565+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1095540779 wait complete. 2023-08-11T14:02:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 Processor -- start 2023-08-11T14:02:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 -- start start 2023-08-11T14:02:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 0x7fd0dc072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc1024c0 0x7fd0dc073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0739f0 con 0x7fd0dc100930 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc073b60 con 0x7fd0dc101b20 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0e27ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1a0680 con 0x7fd0dc1024c0 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40000/0 (socket says 172.21.15.3:40000) 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.566+0000 7fd0dbfff700 1 -- 172.21.15.3:0/1085124948 learned_addr learned my addr 172.21.15.3:0/1085124948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0db7fe700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 0x7fd0dc072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0e0d9c700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc1024c0 0x7fd0dc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 -- 172.21.15.3:0/1085124948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc1024c0 msgr2=0x7fd0dc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc1024c0 0x7fd0dc073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 -- 172.21.15.3:0/1085124948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 msgr2=0x7fd0dc072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 0x7fd0dc072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 -- 172.21.15.3:0/1085124948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0095f0 con 0x7fd0dc100930 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0dbfff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005720 tx=0x7fd0cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc01d070 con 0x7fd0dc100930 2023-08-11T14:02:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.568+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0cc021410 con 0x7fd0dc100930 2023-08-11T14:02:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.567+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc1a0900 con 0x7fd0dc100930 2023-08-11T14:02:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.568+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc017650 con 0x7fd0dc100930 2023-08-11T14:02:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.568+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc1a0d20 con 0x7fd0dc100930 2023-08-11T14:02:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.569+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc10c190 con 0x7fd0dc100930 2023-08-11T14:02:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.571+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fd0cc021580 con 0x7fd0dc100930 2023-08-11T14:02:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.572+0000 7fd0d97fa700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 0x7fd0c40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.572+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd0cc0cc8c0 con 0x7fd0dc100930 2023-08-11T14:02:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.573+0000 7fd0db7fe700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 0x7fd0c40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.573+0000 7fd0db7fe700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 0x7fd0c40a24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009c60 tx=0x7fd0d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.576+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd0cc096220 con 0x7fd0dc100930 2023-08-11T14:02:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:13 smithi103 ceph-mon[130434]: pgmap v294: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 16 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2023-08-11T14:02:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.782+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0dc060d60 con 0x7fd0c40a0020 2023-08-11T14:02:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.795+0000 7fd0d97fa700 1 -- 172.21.15.3:0/1085124948 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422290 (secure 0 0 0) 0x7fd0dc060d60 con 0x7fd0c40a0020 2023-08-11T14:02:13.800 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 msgr2=0x7fd0c40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 0x7fd0c40a24e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009c60 tx=0x7fd0d0009260 comp rx=0 tx=0).stop 2023-08-11T14:02:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 msgr2=0x7fd0dc0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005720 tx=0x7fd0cc009580 comp rx=0 tx=0).stop 2023-08-11T14:02:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 shutdown_connections 2023-08-11T14:02:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc1024c0 0x7fd0dc073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0dc100930 0x7fd0dc0746d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd0c40a0020 0x7fd0c40a24e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 --2- 172.21.15.3:0/1085124948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0dc101b20 0x7fd0dc072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 >> 172.21.15.3:0/1085124948 conn(0x7fd0dc0fbe60 msgr2=0x7fd0dc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 shutdown_connections 2023-08-11T14:02:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:13.800+0000 7fd0e27ff700 1 -- 172.21.15.3:0/1085124948 wait complete. 2023-08-11T14:02:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:13.819 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-08-11T14:02:12.169050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649403,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":240,"num_read_kb":230,"num_write":414,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338144,"kb_used_data":13908,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603680,"statfs":{"total":1151940427776,"available":1151594168320,"internally_reserved":0,"allocated":14241792,"data_stored":7797101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49048,"num_objects":21,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":89,"num_read_kb":98,"num_write":69,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.841867"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.339952+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:02:03.339952+0000","last_peered":"2023-08-11T14:02:03.339952+0000","last_clean":"2023-08-11T14:02:03.339952+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:02:03.339952+0000","last_undegraded":"2023-08-11T14:02:03.339952+0000","last_fullsized":"2023-08-11T14:02:03.339952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:07:02.851636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514145+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:03.514145+0000","last_peered":"2023-08-11T14:02:03.514145+0000","last_clean":"2023-08-11T14:02:03.514145+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:03.514145+0000","last_undegraded":"2023-08-11T14:02:03.514145+0000","last_fullsized":"2023-08-11T14:02:03.514145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:30.192305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514351+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:03.514351+0000","last_peered":"2023-08-11T14:02:03.514351+0000","last_clean":"2023-08-11T14:02:03.514351+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:03.514351+0000","last_undegraded":"2023-08-11T14:02:03.514351+0000","last_fullsized":"2023-08-11T14:02:03.514351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:53:22.056513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330963+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:03.330963+0000","last_peered":"2023-08-11T14:02:03.330963+0000","last_clean":"2023-08-11T14:02:03.330963+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:03.330963+0000","last_undegraded":"2023-08-11T14:02:03.330963+0000","last_fullsized":"2023-08-11T14:02:03.330963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:51.334309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:06.330693+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:06.330693+0000","last_peered":"2023-08-11T14:02:06.330693+0000","last_clean":"2023-08-11T14:02:06.330693+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:06.330693+0000","last_undegraded":"2023-08-11T14:02:06.330693+0000","last_fullsized":"2023-08-11T14:02:06.330693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:49.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513358+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:03.513358+0000","last_peered":"2023-08-11T14:02:03.513358+0000","last_clean":"2023-08-11T14:02:03.513358+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:03.513358+0000","last_undegraded":"2023-08-11T14:02:03.513358+0000","last_fullsized":"2023-08-11T14:02:03.513358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:08.181790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:05.329322+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:05.329322+0000","last_peered":"2023-08-11T14:02:05.329322+0000","last_clean":"2023-08-11T14:02:05.329322+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:05.329322+0000","last_undegraded":"2023-08-11T14:02:05.329322+0000","last_fullsized":"2023-08-11T14:02:05.329322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330576+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:03.330576+0000","last_peered":"2023-08-11T14:02:03.330576+0000","last_clean":"2023-08-11T14:02:03.330576+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:03.330576+0000","last_undegraded":"2023-08-11T14:02:03.330576+0000","last_fullsized":"2023-08-11T14:02:03.330576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:21.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.333138+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:08.333138+0000","last_peered":"2023-08-11T14:02:08.333138+0000","last_clean":"2023-08-11T14:02:08.333138+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:08.333138+0000","last_undegraded":"2023-08-11T14:02:08.333138+0000","last_fullsized":"2023-08-11T14:02:08.333138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514217+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:03.514217+0000","last_peered":"2023-08-11T14:02:03.514217+0000","last_clean":"2023-08-11T14:02:03.514217+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:03.514217+0000","last_undegraded":"2023-08-11T14:02:03.514217+0000","last_fullsized":"2023-08-11T14:02:03.514217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:49:01.488839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330205+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:03.330205+0000","last_peered":"2023-08-11T14:02:03.330205+0000","last_clean":"2023-08-11T14:02:03.330205+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:03.330205+0000","last_undegraded":"2023-08-11T14:02:03.330205+0000","last_fullsized":"2023-08-11T14:02:03.330205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:32.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513781+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:03.513781+0000","last_peered":"2023-08-11T14:02:03.513781+0000","last_clean":"2023-08-11T14:02:03.513781+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:03.513781+0000","last_undegraded":"2023-08-11T14:02:03.513781+0000","last_fullsized":"2023-08-11T14:02:03.513781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:08.653787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330092+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:03.330092+0000","last_peered":"2023-08-11T14:02:03.330092+0000","last_clean":"2023-08-11T14:02:03.330092+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:03.330092+0000","last_undegraded":"2023-08-11T14:02:03.330092+0000","last_fullsized":"2023-08-11T14:02:03.330092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:16.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'30","reported_seq":74,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.345527+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:08.345527+0000","last_peered":"2023-08-11T14:02:08.345527+0000","last_clean":"2023-08-11T14:02:08.345527+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:08.345527+0000","last_undegraded":"2023-08-11T14:02:08.345527+0000","last_fullsized":"2023-08-11T14:02:08.345527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51977,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":62,"num_write":32,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:10.337317+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:10.337317+0000","last_peered":"2023-08-11T14:02:10.337317+0000","last_clean":"2023-08-11T14:02:10.337317+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:10.337317+0000","last_undegraded":"2023-08-11T14:02:10.337317+0000","last_fullsized":"2023-08-11T14:02:10.337317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515481+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:02:03.515481+0000","last_peered":"2023-08-11T14:02:03.515481+0000","last_clean":"2023-08-11T14:02:03.515481+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:02:03.515481+0000","last_undegraded":"2023-08-11T14:02:03.515481+0000","last_fullsized":"2023-08-11T14:02:03.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:07:44.577236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330437+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:03.330437+0000","last_peered":"2023-08-11T14:02:03.330437+0000","last_clean":"2023-08-11T14:02:03.330437+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:03.330437+0000","last_undegraded":"2023-08-11T14:02:03.330437+0000","last_fullsized":"2023-08-11T14:02:03.330437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:52.907857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.334142+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:08.334142+0000","last_peered":"2023-08-11T14:02:08.334142+0000","last_clean":"2023-08-11T14:02:08.334142+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:08.334142+0000","last_undegraded":"2023-08-11T14:02:08.334142+0000","last_fullsized":"2023-08-11T14:02:08.334142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:07.332219+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:07.332219+0000","last_peered":"2023-08-11T14:02:07.332219+0000","last_clean":"2023-08-11T14:02:07.332219+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:07.332219+0000","last_undegraded":"2023-08-11T14:02:07.332219+0000","last_fullsized":"2023-08-11T14:02:07.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330920+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:03.330920+0000","last_peered":"2023-08-11T14:02:03.330920+0000","last_clean":"2023-08-11T14:02:03.330920+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:03.330920+0000","last_undegraded":"2023-08-11T14:02:03.330920+0000","last_fullsized":"2023-08-11T14:02:03.330920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:08:56.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514463+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:03.514463+0000","last_peered":"2023-08-11T14:02:03.514463+0000","last_clean":"2023-08-11T14:02:03.514463+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:03.514463+0000","last_undegraded":"2023-08-11T14:02:03.514463+0000","last_fullsized":"2023-08-11T14:02:03.514463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:19.003199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330601+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:03.330601+0000","last_peered":"2023-08-11T14:02:03.330601+0000","last_clean":"2023-08-11T14:02:03.330601+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:03.330601+0000","last_undegraded":"2023-08-11T14:02:03.330601+0000","last_fullsized":"2023-08-11T14:02:03.330601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:32:17.015669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513303+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:03.513303+0000","last_peered":"2023-08-11T14:02:03.513303+0000","last_clean":"2023-08-11T14:02:03.513303+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:03.513303+0000","last_undegraded":"2023-08-11T14:02:03.513303+0000","last_fullsized":"2023-08-11T14:02:03.513303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:16.532299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:08.333976+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:08.333976+0000","last_peered":"2023-08-11T14:02:08.333976+0000","last_clean":"2023-08-11T14:02:08.333976+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:08.333976+0000","last_undegraded":"2023-08-11T14:02:08.333976+0000","last_fullsized":"2023-08-11T14:02:08.333976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514220+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:03.514220+0000","last_peered":"2023-08-11T14:02:03.514220+0000","last_clean":"2023-08-11T14:02:03.514220+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:03.514220+0000","last_undegraded":"2023-08-11T14:02:03.514220+0000","last_fullsized":"2023-08-11T14:02:03.514220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:06.901263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58789,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":114,"num_read_kb":121,"num_write":231,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":176367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.92700000000000005}]},{"osd":7,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.048},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.77500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.0720000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:01:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.97399999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1691648,"data_stored":1129255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.194},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.0309999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.147}]},{"osd":3,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.01}]},{"osd":6,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.0880000000000001}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.1100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.99299999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:01:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":1.234},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.129}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":494098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1,"15min":1},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97599999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.73099999999999998,"15min":0.75700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93799999999999994,"5min":1.123,"15min":1.123},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.72899999999999998,"15min":0.755},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.878,"15min":0.90700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.075,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.93400000000000005,"15min":0.96499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0289999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.93100000000000005,"15min":0.96399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.90600000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1,"15min":1.038},"min":{"1min":0.54600000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1679999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":1.006,"15min":1.042},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1240000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1759999999999999,"5min":1.484,"15min":1.484},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.103,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.92200000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103202,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64700000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.86599999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85999999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77200000000000002,"5min":1.022,"15min":1.022},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64900000000000002,"15min":0.65400000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91000000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74399999999999999,"5min":1.101,"15min":1.101},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88700000000000001,"5min":1.042,"15min":1.042},"last":0.91700000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.503,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83999999999999997,"5min":1.083,"15min":1.083},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83399999999999996,"5min":1.29,"15min":1.29},"last":0.751}]},{"osd":9,"last update":"Fri Aug 11 14:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.98099999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.87,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.33100000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92800000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.67700000000000005,"15min":0.66300000000000003},"min":{"1min":0.54300000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.58799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92300000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.877}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92000000000000004,"5min":1.143,"15min":1.143},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92800000000000005,"5min":1.173,"15min":1.173},"last":0.84999999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:01:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.86299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71199999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.133}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":1.117}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.86899999999999999,"15min":0.86699999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":1.167}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.875,"15min":0.89100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0620000000000001,"5min":1.663,"15min":1.663},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.151}]},{"osd":9,"last update":"Fri Aug 11 14:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2130000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.1000000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.175},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.212}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":538240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.054,"15min":1.054},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.503}]},{"osd":2,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.438,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78200000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80400000000000005,"5min":0.876,"15min":0.876},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.316,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.37}]},{"osd":5,"last update":"Fri Aug 11 14:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.873}]},{"osd":7,"last update":"Fri Aug 11 14:01:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.59099999999999997,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.61199999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1140000000000001,"5min":1.302,"15min":1.302},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.55600000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.90200000000000002},"min":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95599999999999996,"5min":3.194,"15min":3.194},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2370000000000001,"5min":1.393,"15min":1.393},"last":0.70599999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.57999999999999996},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.434,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78600000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.60999999999999999},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95599999999999996,"5min":1.069,"15min":1.069},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Aug 11 13:59:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.502},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.46800000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.45100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.006,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.75},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.79600000000000004,"15min":0.78900000000000003},"min":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":1.02}]},{"osd":8,"last update":"Fri Aug 11 14:01:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.069,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.054},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0840000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":1.081}]},{"osd":10,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.98199999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0029999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":538665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53400000000000003,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.877,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.35899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54400000000000004,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.80200000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.442,"15min":0.40100000000000002},"min":{"1min":0.32800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.432},{"interface":"front","average":{"1min":0.5,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.496}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.46400000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83299999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.78500000000000003}]},{"osd":8,"last update":"Fri Aug 11 14:01:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.45000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.754}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46500000000000002,"15min":0.41599999999999998},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79600000000000004,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.91900000000000004,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.375}]},{"osd":1,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.60599999999999998},"min":{"1min":0.47499999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89200000000000002,"5min":2.548,"15min":2.548},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:01:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.65800000000000003,"15min":0.63200000000000001},"min":{"1min":0.48399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90300000000000002,"5min":1.095,"15min":1.095},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:01:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.65200000000000002,"15min":0.61399999999999999},"min":{"1min":0.56799999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.625},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.628},"min":{"1min":0.54300000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71299999999999997,"15min":0.68999999999999995},"min":{"1min":0.48799999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.753},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.81299999999999994,"15min":0.80700000000000005},"min":{"1min":0.58999999999999997,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.74399999999999999,"15min":0.73299999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.76900000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:02:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.97699999999999998,"15min":1.0009999999999999},"min":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0409999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0649999999999999,"5min":1.891,"15min":1.891},"last":0.65900000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:01:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.38800000000000001,"15min":0.36399999999999999},"min":{"1min":0.30399999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.39000000000000001,"15min":0.374},"min":{"1min":0.27300000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.54300000000000004},"min":{"1min":0.40899999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61599999999999999,"15min":0.56699999999999995},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80800000000000005,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.46999999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.68899999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.754},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.439,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63300000000000001}]},{"osd":8,"last update":"Fri Aug 11 14:01:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.745}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.753,"15min":0.747},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.70999999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.38600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47599999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82799999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.878,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.48899999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:01:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:02:13.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-11T14:02:13.830 INFO:tasks.cephadm:Setting up client nodes... 2023-08-11T14:02:13.830 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-08-11T14:02:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:15.390 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:15 smithi003 ceph-mon[97195]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:15.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:15 smithi003 ceph-mon[97195]: pgmap v295: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 15 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:15 smithi103 ceph-mon[130434]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:15 smithi103 ceph-mon[130434]: pgmap v295: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 15 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:15 smithi187 ceph-mon[130973]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:02:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:15 smithi187 ceph-mon[130973]: pgmap v295: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 KiB/s wr, 15 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 -- 172.21.15.3:0/3398254831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 msgr2=0x7f9478072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 --2- 172.21.15.3:0/3398254831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f9478072db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9468009990 tx=0x7f9468009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 -- 172.21.15.3:0/3398254831 shutdown_connections 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 --2- 172.21.15.3:0/3398254831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f947810cfc0 0x7f947810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 --2- 172.21.15.3:0/3398254831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f9478072db0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 --2- 172.21.15.3:0/3398254831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9478073380 0x7f9478073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 -- 172.21.15.3:0/3398254831 >> 172.21.15.3:0/3398254831 conn(0x7f94780fbfa0 msgr2=0x7f94780fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 -- 172.21.15.3:0/3398254831 shutdown_connections 2023-08-11T14:02:16.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.022+0000 7f947e1c1700 1 -- 172.21.15.3:0/3398254831 wait complete. 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.023+0000 7f947e1c1700 1 Processor -- start 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.023+0000 7f947e1c1700 1 -- start start 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f947819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947810cfc0 0x7f94781a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:16.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a4830 con 0x7f9478074950 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a4970 con 0x7f947810cfc0 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f947e1c1700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781a4b30 con 0x7f9478073380 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f947819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36990/0 (socket says 172.21.15.3:36990) 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 -- 172.21.15.3:0/2602250299 learned_addr learned my addr 172.21.15.3:0/2602250299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:16.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f9477fff700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947810cfc0 0x7f94781a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 -- 172.21.15.3:0/2602250299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947810cfc0 msgr2=0x7f94781a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947810cfc0 0x7f94781a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 -- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 msgr2=0x7f947819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f947819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.024+0000 7f94777fe700 1 -- 172.21.15.3:0/2602250299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94680095f0 con 0x7f9478073380 2023-08-11T14:02:16.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f94777fe700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f946800b680 tx=0x7f9468004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:16.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946801d070 con 0x7f9478073380 2023-08-11T14:02:16.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94680054f0 con 0x7f9478073380 2023-08-11T14:02:16.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004ca0 con 0x7f9478073380 2023-08-11T14:02:16.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94781a4db0 con 0x7f9478073380 2023-08-11T14:02:16.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.025+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94781a5160 con 0x7f9478073380 2023-08-11T14:02:16.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.026+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9478102b10 con 0x7f9478073380 2023-08-11T14:02:16.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.028+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f946800fc70 con 0x7f9478073380 2023-08-11T14:02:16.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.030+0000 7f9474ff9700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 0x7f94600a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:16.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.030+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f94680ccb20 con 0x7f9478073380 2023-08-11T14:02:16.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.030+0000 7f9476ffd700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 0x7f94600a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:16.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.030+0000 7f9476ffd700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 0x7f94600a2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9478074680 tx=0x7f946c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:16.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.032+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9468096510 con 0x7f9478073380 2023-08-11T14:02:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.322+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f9478112ac0 con 0x7f9478073380 2023-08-11T14:02:16.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.330+0000 7f9474ff9700 1 -- 172.21.15.3:0/2602250299 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9478112ac0 con 0x7f9478073380 2023-08-11T14:02:16.331 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-08-11T14:02:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBoP9ZkAUJbExAASOM1oBjvVC2tvSkxtQhS5w== 2023-08-11T14:02:16.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 msgr2=0x7f94600a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 0x7f94600a2520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9478074680 tx=0x7f946c007400 comp rx=0 tx=0).stop 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 msgr2=0x7f947819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f946800b680 tx=0x7f9468004730 comp rx=0 tx=0).stop 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 shutdown_connections 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f94600a0060 0x7f94600a2520 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947810cfc0 0x7f94781a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9478073380 0x7f947819cce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 --2- 172.21.15.3:0/2602250299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9478074950 0x7f947819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 >> 172.21.15.3:0/2602250299 conn(0x7f94780fbfa0 msgr2=0x7f94780fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 shutdown_connections 2023-08-11T14:02:16.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:16.332+0000 7f947e1c1700 1 -- 172.21.15.3:0/2602250299 wait complete. 2023-08-11T14:02:16.793 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:02:16.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-08-11T14:02:16.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-11T14:02:16.897 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-11T14:02:16.899 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-08-11T14:02:16.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph mgr dump --format=json 2023-08-11T14:02:17.669 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:17 smithi103 ceph-mon[130434]: pgmap v296: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:17 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2602250299' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:17 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:17 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T14:02:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:17 smithi003 ceph-mon[97195]: pgmap v296: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:17.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:17 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2602250299' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:17 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:17 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T14:02:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:17 smithi187 ceph-mon[130973]: pgmap v296: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2023-08-11T14:02:17.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:17 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2602250299' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:17 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-08-11T14:02:17.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:17 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-08-11T14:02:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.121+0000 7f29dce33700 1 -- 172.21.15.3:0/2280979972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8103780 msgr2=0x7f29d81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.121+0000 7f29dce33700 1 --2- 172.21.15.3:0/2280979972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8103780 0x7f29d81077d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009990 tx=0x7f29cc009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 -- 172.21.15.3:0/2280979972 shutdown_connections 2023-08-11T14:02:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 --2- 172.21.15.3:0/2280979972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8107eb0 0x7f29d810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 --2- 172.21.15.3:0/2280979972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 0x7f29d81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 --2- 172.21.15.3:0/2280979972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8103780 0x7f29d81077d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 -- 172.21.15.3:0/2280979972 >> 172.21.15.3:0/2280979972 conn(0x7f29d80fe620 msgr2=0x7f29d8100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 -- 172.21.15.3:0/2280979972 shutdown_connections 2023-08-11T14:02:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.122+0000 7f29dce33700 1 -- 172.21.15.3:0/2280979972 wait complete. 2023-08-11T14:02:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 Processor -- start 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 -- start start 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 0x7f29d8198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8103780 0x7f29d8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d819dcd0 con 0x7f29d8107eb0 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d819de40 con 0x7f29d8103780 2023-08-11T14:02:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.123+0000 7f29dce33700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d819e000 con 0x7f29d8102dd0 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40032/0 (socket says 172.21.15.3:40032) 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 -- 172.21.15.3:0/2330026984 learned_addr learned my addr 172.21.15.3:0/2330026984 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d659c700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 0x7f29d8198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d5d9b700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8103780 0x7f29d8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 -- 172.21.15.3:0/2330026984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 msgr2=0x7f29d8198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 0x7f29d8198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 -- 172.21.15.3:0/2330026984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8103780 msgr2=0x7f29d8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8103780 0x7f29d8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 -- 172.21.15.3:0/2330026984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8009520 con 0x7f29d8107eb0 2023-08-11T14:02:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.124+0000 7f29d6d9d700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ef30 tx=0x7f29d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.125+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0018070 con 0x7f29d8107eb0 2023-08-11T14:02:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.125+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d000f040 con 0x7f29d8107eb0 2023-08-11T14:02:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.125+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc0095f0 con 0x7f29d8107eb0 2023-08-11T14:02:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.125+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000cd00 con 0x7f29d8107eb0 2023-08-11T14:02:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.125+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d81a5970 con 0x7f29d8107eb0 2023-08-11T14:02:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.126+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d804e620 con 0x7f29d8107eb0 2023-08-11T14:02:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.128+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f29d0003a00 con 0x7f29d8107eb0 2023-08-11T14:02:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.130+0000 7f29c77fe700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 0x7f29c00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.130+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f29d00cb210 con 0x7f29d8107eb0 2023-08-11T14:02:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.131+0000 7f29d659c700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 0x7f29c00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.131+0000 7f29d659c700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 0x7f29c00a24a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009df0 tx=0x7f29c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:19.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.135+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f29d0094b70 con 0x7f29d8107eb0 2023-08-11T14:02:19.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.407+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f29d8112be0 con 0x7f29d8107eb0 2023-08-11T14:02:19.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.413+0000 7f29c77fe700 1 -- 172.21.15.3:0/2330026984 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258091 (secure 0 0 0) 0x7f29d0098190 con 0x7f29d8107eb0 2023-08-11T14:02:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 msgr2=0x7f29c00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 0x7f29c00a24a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009df0 tx=0x7f29c8009260 comp rx=0 tx=0).stop 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 msgr2=0x7f29d819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ef30 tx=0x7f29d000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 shutdown_connections 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29d8107eb0 0x7f29d819d6c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d8102dd0 0x7f29d8198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f29c009ffe0 0x7f29c00a24a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 --2- 172.21.15.3:0/2330026984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29d8103780 0x7f29d8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 >> 172.21.15.3:0/2330026984 conn(0x7f29d80fe620 msgr2=0x7f29d80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 shutdown_connections 2023-08-11T14:02:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:19.418+0000 7f29dce33700 1 -- 172.21.15.3:0/2330026984 wait complete. 2023-08-11T14:02:19.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:19 smithi003 ceph-mon[97195]: pgmap v297: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-08-11T14:02:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:19 smithi103 ceph-mon[130434]: pgmap v297: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-08-11T14:02:19.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:19 smithi187 ceph-mon[130973]: pgmap v297: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-08-11T14:02:19.965 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2363526909},{"type":"v1","addr":"172.21.15.3:6801","nonce":2363526909}]},"active_addr":"172.21.15.3:6801/2363526909","active_change":"2023-08-11T13:55:02.079157+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":485728070}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1234370696}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3364078261}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2459012490}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1258271004}]}]} 2023-08-11T14:02:19.972 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-08-11T14:02:19.973 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-08-11T14:02:19.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T14:02:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:20 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2330026984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T14:02:20.703 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:20 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2330026984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T14:02:20.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:20 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2330026984' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-08-11T14:02:21.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:21 smithi003 ceph-mon[97195]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-11T14:02:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:21 smithi103 ceph-mon[130434]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-11T14:02:21.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:21 smithi187 ceph-mon[130973]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-08-11T14:02:22.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 -- 172.21.15.3:0/3440899229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 msgr2=0x7f9f9810cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/3440899229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9810cc40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009990 tx=0x7f9f80009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 -- 172.21.15.3:0/3440899229 shutdown_connections 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/3440899229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 0x7f9f9810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/3440899229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 0x7f9f98073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/3440899229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9810cc40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 -- 172.21.15.3:0/3440899229 >> 172.21.15.3:0/3440899229 conn(0x7f9f980fc410 msgr2=0x7f9f980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:22.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 -- 172.21.15.3:0/3440899229 shutdown_connections 2023-08-11T14:02:22.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.231+0000 7f9f9e24c700 1 -- 172.21.15.3:0/3440899229 wait complete. 2023-08-11T14:02:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.232+0000 7f9f9e24c700 1 Processor -- start 2023-08-11T14:02:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.232+0000 7f9f9e24c700 1 -- start start 2023-08-11T14:02:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.232+0000 7f9f9e24c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 0x7f9f9819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f9e24c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:22.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f9e24c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 0x7f9f98198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f9e24c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f981990e0 con 0x7f9f98073670 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f9e24c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98199250 con 0x7f9f98072d50 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f9e24c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98199410 con 0x7f9f9810d180 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50686/0 (socket says 172.21.15.3:50686) 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 -- 172.21.15.3:0/649335609 learned_addr learned my addr 172.21.15.3:0/649335609 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:22.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f8f7fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 0x7f9f9819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f97fff700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 0x7f9f98198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 -- 172.21.15.3:0/649335609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 msgr2=0x7f9f98198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 0x7f9f98198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 -- 172.21.15.3:0/649335609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 msgr2=0x7f9f9819e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:22.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 0x7f9f9819e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.233+0000 7f9f977fe700 1 -- 172.21.15.3:0/649335609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800095f0 con 0x7f9f98073670 2023-08-11T14:02:22.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f977fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8001d070 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f80021410 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f98199690 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80017650 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.234+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f981a59e0 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.235+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9804e620 con 0x7f9f98073670 2023-08-11T14:02:22.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.237+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f9f80021580 con 0x7f9f98073670 2023-08-11T14:02:22.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.238+0000 7f9f957fa700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 0x7f9f840a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:22.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.238+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9f800cc9d0 con 0x7f9f98073670 2023-08-11T14:02:22.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.239+0000 7f9f8f7fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 0x7f9f840a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:22.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.239+0000 7f9f8f7fe700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 0x7f9f840a24a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009c60 tx=0x7f9f88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:22.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.242+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9f80096330 con 0x7f9f98073670 2023-08-11T14:02:22.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.454+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f98112b60 con 0x7f9f98073670 2023-08-11T14:02:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.455+0000 7f9f957fa700 1 -- 172.21.15.3:0/649335609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24357 (secure 0 0 0) 0x7f9f80099950 con 0x7f9f98073670 2023-08-11T14:02:22.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:22.458 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:02:03.324509+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:02:22.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 msgr2=0x7f9f840a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:22.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 0x7f9f840a24a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009c60 tx=0x7f9f88009260 comp rx=0 tx=0).stop 2023-08-11T14:02:22.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 msgr2=0x7f9f9819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80004790 comp rx=0 tx=0).stop 2023-08-11T14:02:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 shutdown_connections 2023-08-11T14:02:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f9810d180 0x7f9f98198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f8409ffe0 0x7f9f840a24a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f98072d50 0x7f9f9819e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 --2- 172.21.15.3:0/649335609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f98073670 0x7f9f9819ef30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 >> 172.21.15.3:0/649335609 conn(0x7f9f980fc410 msgr2=0x7f9f9810f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 shutdown_connections 2023-08-11T14:02:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:22.458+0000 7f9f9e24c700 1 -- 172.21.15.3:0/649335609 wait complete. 2023-08-11T14:02:22.906 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-08-11T14:02:22.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd dump --format=json 2023-08-11T14:02:23.651 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:23 smithi103 ceph-mon[130434]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:02:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:23 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/649335609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:23 smithi003 ceph-mon[97195]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:02:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:23 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/649335609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:23.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:23 smithi187 ceph-mon[130973]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:02:23.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:23 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/649335609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/1114190193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c101300 msgr2=0x7fbf9c10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:25.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/1114190193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c101300 0x7fbf9c10d810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b0a0 tx=0x7fbf9400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/1114190193 shutdown_connections 2023-08-11T14:02:25.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/1114190193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c101300 0x7fbf9c10d810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/1114190193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c106940 0x7fbf9c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/1114190193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c100940 0x7fbf9c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/1114190193 >> 172.21.15.3:0/1114190193 conn(0x7fbf9c0fc410 msgr2=0x7fbf9c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:25.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/1114190193 shutdown_connections 2023-08-11T14:02:25.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.204+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/1114190193 wait complete. 2023-08-11T14:02:25.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.205+0000 7fbfa1f6f700 1 Processor -- start 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.205+0000 7fbfa1f6f700 1 -- start start 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 0x7fbf9c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 0x7fbf9c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19ddb0 con 0x7fbf9c100940 2023-08-11T14:02:25.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19df20 con 0x7fbf9c106940 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbfa1f6f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19e0e0 con 0x7fbf9c101300 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50562/0 (socket says 172.21.15.3:50562) 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 -- 172.21.15.3:0/4225954383 learned_addr learned my addr 172.21.15.3:0/4225954383 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9b7fe700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 0x7fbf9c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9bfff700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 0x7fbf9c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 -- 172.21.15.3:0/4225954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 msgr2=0x7fbf9c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:25.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 0x7fbf9c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 -- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 msgr2=0x7fbf9c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 0x7fbf9c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9affd700 1 -- 172.21.15.3:0/4225954383 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009d00 con 0x7fbf9c101300 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9bfff700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 0x7fbf9c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.206+0000 7fbf9b7fe700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 0x7fbf9c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbf9affd700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000d950 tx=0x7fbf9000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:25.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90009940 con 0x7fbf9c101300 2023-08-11T14:02:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf90010460 con 0x7fbf9c101300 2023-08-11T14:02:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c19e3c0 con 0x7fbf9c101300 2023-08-11T14:02:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000f620 con 0x7fbf9c101300 2023-08-11T14:02:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.207+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c1a5c20 con 0x7fbf9c101300 2023-08-11T14:02:25.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.208+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c04e620 con 0x7fbf9c101300 2023-08-11T14:02:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.210+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fbf900108d0 con 0x7fbf9c101300 2023-08-11T14:02:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.212+0000 7fbf98ff9700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 0x7fbf840a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.212+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbf900cb5a0 con 0x7fbf9c101300 2023-08-11T14:02:25.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.212+0000 7fbf9b7fe700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 0x7fbf840a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:25.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.212+0000 7fbf9b7fe700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 0x7fbf840a24a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005950 tx=0x7fbf8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:25.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.214+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbf90094e70 con 0x7fbf9c101300 2023-08-11T14:02:25.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.425+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf9c1109c0 con 0x7fbf9c101300 2023-08-11T14:02:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.427+0000 7fbf98ff9700 1 -- 172.21.15.3:0/4225954383 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24357 (secure 0 0 0) 0x7fbf90098490 con 0x7fbf9c101300 2023-08-11T14:02:25.430 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:25.430 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:02:03.324509+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:02:25.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.429+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 msgr2=0x7fbf840a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:25.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.429+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 0x7fbf840a24a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005950 tx=0x7fbf8c00b410 comp rx=0 tx=0).stop 2023-08-11T14:02:25.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.429+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 msgr2=0x7fbf9c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:25.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.429+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000d950 tx=0x7fbf9000dc60 comp rx=0 tx=0).stop 2023-08-11T14:02:25.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 shutdown_connections 2023-08-11T14:02:25.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf8409ffe0 0x7fbf840a24a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf9c106940 0x7fbf9c19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf9c100940 0x7fbf9c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 --2- 172.21.15.3:0/4225954383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf9c101300 0x7fbf9c199330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:25.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 >> 172.21.15.3:0/4225954383 conn(0x7fbf9c0fc410 msgr2=0x7fbf9c10be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:25.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 shutdown_connections 2023-08-11T14:02:25.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:25.430+0000 7fbfa1f6f700 1 -- 172.21.15.3:0/4225954383 wait complete. 2023-08-11T14:02:25.576 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:25 smithi003 ceph-mon[97195]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:02:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:25 smithi103 ceph-mon[130434]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:02:25.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:25 smithi187 ceph-mon[130973]: pgmap v300: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:02:25.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-08-11T14:02:25.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-08-11T14:02:25.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-08-11T14:02:25.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-08-11T14:02:25.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-08-11T14:02:25.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-08-11T14:02:25.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-08-11T14:02:25.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-08-11T14:02:25.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-08-11T14:02:25.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-08-11T14:02:25.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-08-11T14:02:25.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-08-11T14:02:26.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:26 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4225954383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:26 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4225954383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:26.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:26 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4225954383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:02:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:27 smithi103 ceph-mon[130434]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:27.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:27 smithi003 ceph-mon[97195]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:27.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:27 smithi187 ceph-mon[130973]: pgmap v301: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:27.940 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.039 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.136 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.367 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.367 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.756 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.858 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.899 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:28.942 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:29 smithi103 ceph-mon[130434]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:29.753 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:29 smithi003 ceph-mon[97195]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:29 smithi187 ceph-mon[130973]: pgmap v302: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2023-08-11T14:02:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:31.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:31 smithi003 ceph-mon[97195]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-08-11T14:02:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:31 smithi103 ceph-mon[130434]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-08-11T14:02:31.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:31 smithi187 ceph-mon[130973]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2023-08-11T14:02:32.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.097+0000 7f6c6b534700 1 -- 172.21.15.3:0/1889715418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c64100220 msgr2=0x7f6c6410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.097+0000 7f6c6b534700 1 --2- 172.21.15.3:0/1889715418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c64100220 0x7f6c6410c6c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b0a0 tx=0x7f6c5c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 -- 172.21.15.3:0/1889715418 shutdown_connections 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 --2- 172.21.15.3:0/1889715418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c64100220 0x7f6c6410c6c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 --2- 172.21.15.3:0/1889715418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 0x7f6c64104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 --2- 172.21.15.3:0/1889715418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c640ff860 0x7f6c640ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 -- 172.21.15.3:0/1889715418 >> 172.21.15.3:0/1889715418 conn(0x7f6c64076920 msgr2=0x7f6c64076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 -- 172.21.15.3:0/1889715418 shutdown_connections 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.098+0000 7f6c6b534700 1 -- 172.21.15.3:0/1889715418 wait complete. 2023-08-11T14:02:32.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 Processor -- start 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 -- start start 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 0x7f6c641013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c64100220 0x7f6c64101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 0x7f6c64101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641a5070 con 0x7f6c640ff860 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641a51e0 con 0x7f6c64104730 2023-08-11T14:02:32.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.099+0000 7f6c6b534700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641a53a0 con 0x7f6c64100220 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 0x7f6c641013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 0x7f6c641013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48828/0 (socket says 172.21.15.3:48828) 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 -- 172.21.15.3:0/3602421267 learned_addr learned my addr 172.21.15.3:0/3602421267 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c68acf700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c64100220 0x7f6c64101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c69ad1700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 0x7f6c64101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 -- 172.21.15.3:0/3602421267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c64100220 msgr2=0x7f6c64101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c64100220 0x7f6c64101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 -- 172.21.15.3:0/3602421267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 msgr2=0x7f6c64101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 0x7f6c64101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009d00 con 0x7f6c640ff860 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c69ad1700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c64104730 0x7f6c64101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.100+0000 7f6c692d0700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 0x7f6c641013e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d950 tx=0x7f6c6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.101+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60009940 con 0x7f6c640ff860 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.101+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c641a5680 con 0x7f6c640ff860 2023-08-11T14:02:32.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.101+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c60010460 con 0x7f6c640ff860 2023-08-11T14:02:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.101+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000f620 con 0x7f6c640ff860 2023-08-11T14:02:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.101+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c641a5b80 con 0x7f6c640ff860 2023-08-11T14:02:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.102+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c6410fa60 con 0x7f6c640ff860 2023-08-11T14:02:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.104+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f6c600108d0 con 0x7f6c640ff860 2023-08-11T14:02:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.105+0000 7f6c5a7fc700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c5009ffe0 0x7f6c500a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.105+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6c600cba30 con 0x7f6c640ff860 2023-08-11T14:02:32.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c68acf700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c5009ffe0 0x7f6c500a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c5a7fc700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] conn(0x7f6c500aaf80 0x7f6c500ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c500adaf0 con 0x7f6c500aaf80 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c600cbda0 con 0x7f6c640ff860 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c69ad1700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] conn(0x7f6c500aaf80 0x7f6c500ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c68acf700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c5009ffe0 0x7f6c500a24a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c64102bd0 tx=0x7f6c54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:32.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.106+0000 7f6c69ad1700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] conn(0x7f6c500aaf80 0x7f6c500ad420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:32.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.107+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== osd.0 v2:172.21.15.3:6802/2824703498 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6c500adaf0 con 0x7f6c500aaf80 2023-08-11T14:02:32.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.125+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 --> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c6404e620 con 0x7f6c500aaf80 2023-08-11T14:02:32.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.125+0000 7f6c5a7fc700 1 -- 172.21.15.3:0/3602421267 <== osd.0 v2:172.21.15.3:6802/2824703498 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c6404e620 con 0x7f6c500aaf80 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] conn(0x7f6c500aaf80 msgr2=0x7f6c500ad420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] conn(0x7f6c500aaf80 0x7f6c500ad420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c5009ffe0 msgr2=0x7f6c500a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c5009ffe0 0x7f6c500a24a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6c64102bd0 tx=0x7f6c54009260 comp rx=0 tx=0).stop 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 msgr2=0x7f6c641013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:32.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 --2- 172.21.15.3:0/3602421267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c640ff860 0x7f6c641013e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000d950 tx=0x7f6c6000dc60 comp rx=0 tx=0).stop 2023-08-11T14:02:32.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c69ad1700 1 -- 172.21.15.3:0/3602421267 reap_dead start 2023-08-11T14:02:32.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 shutdown_connections 2023-08-11T14:02:32.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 >> 172.21.15.3:0/3602421267 conn(0x7f6c64076920 msgr2=0x7f6c64068440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:32.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.126+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 shutdown_connections 2023-08-11T14:02:32.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:32.127+0000 7f6c6b534700 1 -- 172.21.15.3:0/3602421267 wait complete. 2023-08-11T14:02:33.415 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:33 smithi003 ceph-mon[97195]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:33 smithi103 ceph-mon[130434]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:33.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:33 smithi187 ceph-mon[130973]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 -- 172.21.15.3:0/118888261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301067a0 msgr2=0x7f3a30106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/118888261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301067a0 0x7f3a30106b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009990 tx=0x7f3a20009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 -- 172.21.15.3:0/118888261 shutdown_connections 2023-08-11T14:02:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/118888261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a30101160 0x7f3a3010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/118888261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301067a0 0x7f3a30106b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/118888261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 0x7f3a30100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 -- 172.21.15.3:0/118888261 >> 172.21.15.3:0/118888261 conn(0x7f3a300fc2b0 msgr2=0x7f3a300fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 -- 172.21.15.3:0/118888261 shutdown_connections 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.168+0000 7f3a34a8d700 1 -- 172.21.15.3:0/118888261 wait complete. 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.170+0000 7f3a34a8d700 1 Processor -- start 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.170+0000 7f3a34a8d700 1 -- start start 2023-08-11T14:02:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 0x7f3a3019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a30101160 0x7f3a3019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 0x7f3a30196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30196dc0 con 0x7f3a301067a0 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30196f30 con 0x7f3a30101160 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a34a8d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301970f0 con 0x7f3a301007a0 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 0x7f3a30196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 0x7f3a30196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48858/0 (socket says 172.21.15.3:48858) 2023-08-11T14:02:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a2ffff700 1 -- 172.21.15.3:0/2629551363 learned_addr learned my addr 172.21.15.3:0/2629551363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a2f7fe700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 0x7f3a3019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.171+0000 7f3a2effd700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a30101160 0x7f3a3019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 -- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 msgr2=0x7f3a3019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 0x7f3a3019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 -- 172.21.15.3:0/2629551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a30101160 msgr2=0x7f3a3019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a30101160 0x7f3a3019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24009520 con 0x7f3a301067a0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2f7fe700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a301007a0 0x7f3a3019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2ffff700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 0x7f3a30196750 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800ef30 tx=0x7f3a2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28018070 con 0x7f3a301067a0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a2800f040 con 0x7f3a301067a0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a200095f0 con 0x7f3a301067a0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800cd00 con 0x7f3a301067a0 2023-08-11T14:02:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.172+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a301a34f0 con 0x7f3a301067a0 2023-08-11T14:02:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.173+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a30110a90 con 0x7f3a301067a0 2023-08-11T14:02:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.175+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f3a28003a00 con 0x7f3a301067a0 2023-08-11T14:02:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2cff9700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3a1809ffe0 0x7f3a180a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3a280cbb60 con 0x7f3a301067a0 2023-08-11T14:02:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2f7fe700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3a1809ffe0 0x7f3a180a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2cff9700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] conn(0x7f3a180aaf80 0x7f3a180ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a180adaf0 con 0x7f3a180aaf80 2023-08-11T14:02:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.177+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a280cbed0 con 0x7f3a301067a0 2023-08-11T14:02:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.178+0000 7f3a2f7fe700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3a1809ffe0 0x7f3a180a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000b2e0 tx=0x7f3a20005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.178+0000 7f3a2effd700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] conn(0x7f3a180aaf80 0x7f3a180ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.178+0000 7f3a2effd700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] conn(0x7f3a180aaf80 0x7f3a180ad420 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.179+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== osd.11 v2:172.21.15.187:6824/3190542621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3a180adaf0 con 0x7f3a180aaf80 2023-08-11T14:02:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.197+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 --> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a3004ee70 con 0x7f3a180aaf80 2023-08-11T14:02:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.197+0000 7f3a2cff9700 1 -- 172.21.15.3:0/2629551363 <== osd.11 v2:172.21.15.187:6824/3190542621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a3004ee70 con 0x7f3a180aaf80 2023-08-11T14:02:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] conn(0x7f3a180aaf80 msgr2=0x7f3a180ad420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] conn(0x7f3a180aaf80 0x7f3a180ad420 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3a1809ffe0 msgr2=0x7f3a180a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3a1809ffe0 0x7f3a180a24a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a2000b2e0 tx=0x7f3a20005fd0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 msgr2=0x7f3a30196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.198+0000 7f3a34a8d700 1 --2- 172.21.15.3:0/2629551363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301067a0 0x7f3a30196750 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800ef30 tx=0x7f3a2800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.199+0000 7f3a2ffff700 1 -- 172.21.15.3:0/2629551363 reap_dead start 2023-08-11T14:02:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.199+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 shutdown_connections 2023-08-11T14:02:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.199+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 >> 172.21.15.3:0/2629551363 conn(0x7f3a300fc2b0 msgr2=0x7f3a301042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.200+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 shutdown_connections 2023-08-11T14:02:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.200+0000 7f3a34a8d700 1 -- 172.21.15.3:0/2629551363 wait complete. 2023-08-11T14:02:35.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:35 smithi003 ceph-mon[97195]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:35.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:35 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:02:35.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 -- 172.21.15.3:0/2306910962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 msgr2=0x7f022410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 --2- 172.21.15.3:0/2306910962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f022410d810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f021c00b0a0 tx=0x7f021c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 -- 172.21.15.3:0/2306910962 shutdown_connections 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 --2- 172.21.15.3:0/2306910962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f022410d810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 --2- 172.21.15.3:0/2306910962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0224106940 0x7f0224106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 --2- 172.21.15.3:0/2306910962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224100940 0x7f0224100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 -- 172.21.15.3:0/2306910962 >> 172.21.15.3:0/2306910962 conn(0x7f02240fc410 msgr2=0x7f02240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 -- 172.21.15.3:0/2306910962 shutdown_connections 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.051+0000 7f022bf95700 1 -- 172.21.15.3:0/2306910962 wait complete. 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.052+0000 7f022bf95700 1 Processor -- start 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.052+0000 7f022bf95700 1 -- start start 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0224100940 0x7f0224072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f0224073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224106940 0x7f02240736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a94c0 con 0x7f0224101300 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a9600 con 0x7f0224100940 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022bf95700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02241a9740 con 0x7f0224106940 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f0224073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f0224073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48842/0 (socket says 172.21.15.3:48842) 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 -- 172.21.15.3:0/2970365509 learned_addr learned my addr 172.21.15.3:0/2970365509 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:35.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022a532700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224106940 0x7f02240736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229d31700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0224100940 0x7f0224072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 -- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224106940 msgr2=0x7f02240736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224106940 0x7f02240736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 -- 172.21.15.3:0/2970365509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0224100940 msgr2=0x7f0224072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0224100940 0x7f0224072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f0229530700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c009d00 con 0x7f0224101300 2023-08-11T14:02:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.053+0000 7f022a532700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224106940 0x7f02240736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f0229530700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f0224073160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9d0 tx=0x7f021400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214009790 con 0x7f0224101300 2023-08-11T14:02:35.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02241a9a20 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021400ee70 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214010ea0 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.054+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02241a9f70 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.055+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0224110c00 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.057+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f021400cbd0 con 0x7f0224101300 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f021affd700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f021009ffe0 0x7f02100a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0214014070 con 0x7f0224101300 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f0229d31700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f021009ffe0 0x7f02100a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f021affd700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] conn(0x7f02100aaf80 0x7f02100ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02100adaf0 con 0x7f02100aaf80 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.059+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02140cdce0 con 0x7f0224101300 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.060+0000 7f0229d31700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f021009ffe0 0x7f02100a24a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0220009be0 tx=0x7f0220009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.060+0000 7f022a532700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] conn(0x7f02100aaf80 0x7f02100ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.060+0000 7f022a532700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] conn(0x7f02100aaf80 0x7f02100ad420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:35.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.061+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== osd.2 v2:172.21.15.187:6800/2156234450 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f02100adaf0 con 0x7f02100aaf80 2023-08-11T14:02:35.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.078+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 --> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f022404e620 con 0x7f02100aaf80 2023-08-11T14:02:35.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.079+0000 7f021affd700 1 -- 172.21.15.3:0/2970365509 <== osd.2 v2:172.21.15.187:6800/2156234450 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f022404e620 con 0x7f02100aaf80 2023-08-11T14:02:35.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] conn(0x7f02100aaf80 msgr2=0x7f02100ad420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] conn(0x7f02100aaf80 0x7f02100ad420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f021009ffe0 msgr2=0x7f02100a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f021009ffe0 0x7f02100a24a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0220009be0 tx=0x7f0220009260 comp rx=0 tx=0).stop 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 msgr2=0x7f0224073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 --2- 172.21.15.3:0/2970365509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224101300 0x7f0224073160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9d0 tx=0x7f021400ece0 comp rx=0 tx=0).stop 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022a532700 1 -- 172.21.15.3:0/2970365509 reap_dead start 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 shutdown_connections 2023-08-11T14:02:35.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 >> 172.21.15.3:0/2970365509 conn(0x7f02240fc410 msgr2=0x7f02240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:35.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 shutdown_connections 2023-08-11T14:02:35.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:35.080+0000 7f022bf95700 1 -- 172.21.15.3:0/2970365509 wait complete. 2023-08-11T14:02:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:35 smithi103 ceph-mon[130434]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:35 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:02:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:35 smithi187 ceph-mon[130973]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:35 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:02:36.041 INFO:teuthology.orchestra.run.smithi003.stdout:34359738443 2023-08-11T14:02:36.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-08-11T14:02:36.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 -- 172.21.15.3:0/3020884547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 msgr2=0x7f9e1410cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 --2- 172.21.15.3:0/3020884547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 0x7f9e1410cc40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009990 tx=0x7f9e08009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 -- 172.21.15.3:0/3020884547 shutdown_connections 2023-08-11T14:02:36.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 --2- 172.21.15.3:0/3020884547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e1410d180 0x7f9e1410f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 --2- 172.21.15.3:0/3020884547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e14072d50 0x7f9e14073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 --2- 172.21.15.3:0/3020884547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 0x7f9e1410cc40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 -- 172.21.15.3:0/3020884547 >> 172.21.15.3:0/3020884547 conn(0x7f9e140fc410 msgr2=0x7f9e140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 -- 172.21.15.3:0/3020884547 shutdown_connections 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.526+0000 7f9e18e50700 1 -- 172.21.15.3:0/3020884547 wait complete. 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.527+0000 7f9e18e50700 1 Processor -- start 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.527+0000 7f9e18e50700 1 -- start start 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.527+0000 7f9e18e50700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e14072d50 0x7f9e14198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.527+0000 7f9e18e50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 0x7f9e141991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e18e50700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e1410d180 0x7f9e1419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e18e50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1419cbb0 con 0x7f9e14073670 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e18e50700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1419cd20 con 0x7f9e1410d180 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e18e50700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1419cee0 con 0x7f9e14072d50 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e1259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e14072d50 0x7f9e14198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e1410d180 0x7f9e1419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e1259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e14072d50 0x7f9e14198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39274/0 (socket says 172.21.15.3:39274) 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e1259c700 1 -- 172.21.15.3:0/345089333 learned_addr learned my addr 172.21.15.3:0/345089333 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e11d9b700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 0x7f9e141991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 -- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e14072d50 msgr2=0x7f9e14198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e14072d50 0x7f9e14198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 -- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 msgr2=0x7f9e141991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e14073670 0x7f9e141991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.528+0000 7f9e12d9d700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080095f0 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e12d9d700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e1410d180 0x7f9e1419c5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00efc0 tx=0x7f9e0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c018070 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1419d160 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e0c00f040 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e141a5b10 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.529+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00cd20 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.530+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e14112df0 con 0x7f9e1410d180 2023-08-11T14:02:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.532+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f9e0c003a00 con 0x7f9e1410d180 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e037fe700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9dfc0a0060 0x7f9dfc0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9e0c0cbb30 con 0x7f9e1410d180 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e1259c700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9dfc0a0060 0x7f9dfc0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e037fe700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] conn(0x7f9dfc0ab000 0x7f9dfc0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dfc0adb70 con 0x7f9dfc0ab000 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e0c0cbee0 con 0x7f9e1410d180 2023-08-11T14:02:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.534+0000 7f9e11d9b700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] conn(0x7f9dfc0ab000 0x7f9dfc0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.535+0000 7f9e1259c700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9dfc0a0060 0x7f9dfc0a2520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e04007c90 tx=0x7f9e04007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.535+0000 7f9e11d9b700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] conn(0x7f9dfc0ab000 0x7f9dfc0ad4a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.536+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== osd.8 v2:172.21.15.187:6816/4219051644 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9dfc0adb70 con 0x7f9dfc0ab000 2023-08-11T14:02:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.556+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 --> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e1404e620 con 0x7f9dfc0ab000 2023-08-11T14:02:36.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.557+0000 7f9e037fe700 1 -- 172.21.15.3:0/345089333 <== osd.8 v2:172.21.15.187:6816/4219051644 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9e1404e620 con 0x7f9dfc0ab000 2023-08-11T14:02:36.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] conn(0x7f9dfc0ab000 msgr2=0x7f9dfc0ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] conn(0x7f9dfc0ab000 0x7f9dfc0ad4a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9dfc0a0060 msgr2=0x7f9dfc0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9dfc0a0060 0x7f9dfc0a2520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e04007c90 tx=0x7f9e04007480 comp rx=0 tx=0).stop 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e1410d180 msgr2=0x7f9e1419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 --2- 172.21.15.3:0/345089333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e1410d180 0x7f9e1419c5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00efc0 tx=0x7f9e0c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e12d9d700 1 -- 172.21.15.3:0/345089333 reap_dead start 2023-08-11T14:02:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 shutdown_connections 2023-08-11T14:02:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.558+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 >> 172.21.15.3:0/345089333 conn(0x7f9e140fc410 msgr2=0x7f9e1410f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.559+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 shutdown_connections 2023-08-11T14:02:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.559+0000 7f9e18e50700 1 -- 172.21.15.3:0/345089333 wait complete. 2023-08-11T14:02:36.876 INFO:teuthology.orchestra.run.smithi003.stdout:77309411392 2023-08-11T14:02:36.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-08-11T14:02:36.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 -- 172.21.15.3:0/2885114163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 msgr2=0x7f261810ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 --2- 172.21.15.3:0/2885114163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 0x7f261810ce70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f260c009990 tx=0x7f260c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 -- 172.21.15.3:0/2885114163 shutdown_connections 2023-08-11T14:02:36.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 --2- 172.21.15.3:0/2885114163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f261810d3b0 0x7f261810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 --2- 172.21.15.3:0/2885114163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2618100340 0x7f2618100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 --2- 172.21.15.3:0/2885114163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 0x7f261810ce70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:36.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 -- 172.21.15.3:0/2885114163 >> 172.21.15.3:0/2885114163 conn(0x7f2618076920 msgr2=0x7f2618076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:37.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 -- 172.21.15.3:0/2885114163 shutdown_connections 2023-08-11T14:02:37.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.963+0000 7f261c83c700 1 -- 172.21.15.3:0/2885114163 wait complete. 2023-08-11T14:02:37.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.965+0000 7f261c83c700 1 Processor -- start 2023-08-11T14:02:37.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.965+0000 7f261c83c700 1 -- start start 2023-08-11T14:02:37.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 0x7f261819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 0x7f261819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 0x7f2618198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618199140 con 0x7f2618100c60 2023-08-11T14:02:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181992b0 con 0x7f261810d3b0 2023-08-11T14:02:37.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261c83c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618199470 con 0x7f2618100340 2023-08-11T14:02:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 0x7f261819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 0x7f2618198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f2615d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 0x7f261819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 0x7f261819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39280/0 (socket says 172.21.15.3:39280) 2023-08-11T14:02:37.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f2616d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 0x7f2618198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45484/0 (socket says 172.21.15.3:45484) 2023-08-11T14:02:37.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.966+0000 7f261659c700 1 -- 172.21.15.3:0/840546729 learned_addr learned my addr 172.21.15.3:0/840546729 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:37.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 -- 172.21.15.3:0/840546729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 msgr2=0x7f261819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 0x7f261819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 -- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 msgr2=0x7f261819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2618100c60 0x7f261819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0095f0 con 0x7f261810d3b0 2023-08-11T14:02:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f261659c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2618100340 0x7f261819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f2616d9d700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 0x7f2618198ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2610007ee0 tx=0x7f261000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610004db0 con 0x7f261810d3b0 2023-08-11T14:02:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f261000f040 con 0x7f261810d3b0 2023-08-11T14:02:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2618199750 con 0x7f261810d3b0 2023-08-11T14:02:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610005850 con 0x7f261810d3b0 2023-08-11T14:02:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.967+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26181a5a40 con 0x7f261810d3b0 2023-08-11T14:02:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.969+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2618112dc0 con 0x7f261810d3b0 2023-08-11T14:02:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.970+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f261000db20 con 0x7f261810d3b0 2023-08-11T14:02:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f26077fe700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26000a0060 0x7f26000a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f26100cae00 con 0x7f261810d3b0 2023-08-11T14:02:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f261659c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26000a0060 0x7f26000a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f26077fe700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] conn(0x7f26000ab000 0x7f26000ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26000adb70 con 0x7f26000ab000 2023-08-11T14:02:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.972+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26100d0490 con 0x7f261810d3b0 2023-08-11T14:02:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.973+0000 7f2615d9b700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] conn(0x7f26000ab000 0x7f26000ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.973+0000 7f261659c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26000a0060 0x7f26000a2520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f26080104e0 tx=0x7f2608007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.973+0000 7f2615d9b700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] conn(0x7f26000ab000 0x7f26000ad4a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.974+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== osd.4 v2:172.21.15.103:6808/811611617 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f26000adb70 con 0x7f26000ab000 2023-08-11T14:02:37.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.991+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 --> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f261810f1a0 con 0x7f26000ab000 2023-08-11T14:02:37.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.992+0000 7f26077fe700 1 -- 172.21.15.3:0/840546729 <== osd.4 v2:172.21.15.103:6808/811611617 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f261810f1a0 con 0x7f26000ab000 2023-08-11T14:02:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.994+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] conn(0x7f26000ab000 msgr2=0x7f26000ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.994+0000 7f261c83c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] conn(0x7f26000ab000 0x7f26000ad4a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26000a0060 msgr2=0x7f26000a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f26000a0060 0x7f26000a2520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f26080104e0 tx=0x7f2608007480 comp rx=0 tx=0).stop 2023-08-11T14:02:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 msgr2=0x7f2618198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 --2- 172.21.15.3:0/840546729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f261810d3b0 0x7f2618198ad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2610007ee0 tx=0x7f261000d3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f2616d9d700 1 -- 172.21.15.3:0/840546729 reap_dead start 2023-08-11T14:02:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 shutdown_connections 2023-08-11T14:02:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 >> 172.21.15.3:0/840546729 conn(0x7f2618076920 msgr2=0x7f261810e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 shutdown_connections 2023-08-11T14:02:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:36.995+0000 7f261c83c700 1 -- 172.21.15.3:0/840546729 wait complete. 2023-08-11T14:02:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3303848499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa40ff690 msgr2=0x7f0fa40ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3303848499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa40ff690 0x7f0fa40ffa70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009990 tx=0x7f0f94009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3303848499 shutdown_connections 2023-08-11T14:02:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3303848499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa410d240 0x7f0fa410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3303848499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa40ff690 0x7f0fa40ffa70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3303848499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa4100040 0x7f0fa410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.067+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3303848499 >> 172.21.15.3:0/3303848499 conn(0x7f0fa40fb390 msgr2=0x7f0fa40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.068+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3303848499 shutdown_connections 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.068+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3303848499 wait complete. 2023-08-11T14:02:37.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.069+0000 7f0fa8dfa700 1 Processor -- start 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.069+0000 7f0fa8dfa700 1 -- start start 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.069+0000 7f0fa8dfa700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 0x7f0fa419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.069+0000 7f0fa8dfa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 0x7f0fa419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.069+0000 7f0fa8dfa700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa410d240 0x7f0fa4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa8dfa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4199150 con 0x7f0fa4100040 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa8dfa700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa41992c0 con 0x7f0fa410d240 2023-08-11T14:02:37.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa8dfa700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4199480 con 0x7f0fa40ff690 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 0x7f0fa419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 0x7f0fa419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 0x7f0fa419ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39308/0 (socket says 172.21.15.3:39308) 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa259c700 1 -- 172.21.15.3:0/3276264051 learned_addr learned my addr 172.21.15.3:0/3276264051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 0x7f0fa419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48908/0 (socket says 172.21.15.3:48908) 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa2d9d700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa410d240 0x7f0fa4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 -- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 msgr2=0x7f0fa419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 0x7f0fa419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 -- 172.21.15.3:0/3276264051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa410d240 msgr2=0x7f0fa4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fa410d240 0x7f0fa4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa1d9b700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940095f0 con 0x7f0fa4100040 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.070+0000 7f0fa259c700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fa40ff690 0x7f0fa419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0fa1d9b700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 0x7f0fa419efa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800c9c0 con 0x7f0fa4100040 2023-08-11T14:02:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa4199760 con 0x7f0fa4100040 2023-08-11T14:02:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f98010430 con 0x7f0fa4100040 2023-08-11T14:02:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98018680 con 0x7f0fa4100040 2023-08-11T14:02:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.071+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa41a5a50 con 0x7f0fa4100040 2023-08-11T14:02:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.072+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fa40685b0 con 0x7f0fa4100040 2023-08-11T14:02:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.074+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f0f98010860 con 0x7f0fa4100040 2023-08-11T14:02:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0f937fe700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0f8c09ffe0 0x7f0f8c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0f98014070 con 0x7f0fa4100040 2023-08-11T14:02:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0fa259c700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0f8c09ffe0 0x7f0f8c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0f937fe700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] conn(0x7f0f8c0aaf80 0x7f0f8c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f8c0adaf0 con 0x7f0f8c0aaf80 2023-08-11T14:02:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.077+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f980cbe90 con 0x7f0fa4100040 2023-08-11T14:02:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.078+0000 7f0fa2d9d700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] conn(0x7f0f8c0aaf80 0x7f0f8c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.078+0000 7f0fa259c700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0f8c09ffe0 0x7f0f8c0a24a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f94006010 tx=0x7f0f9400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.078+0000 7f0fa2d9d700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] conn(0x7f0f8c0aaf80 0x7f0f8c0ad420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.079+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== osd.5 v2:172.21.15.187:6808/916876636 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0f8c0adaf0 con 0x7f0f8c0aaf80 2023-08-11T14:02:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.115+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 --> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fa4060fa0 con 0x7f0f8c0aaf80 2023-08-11T14:02:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.115+0000 7f0f937fe700 1 -- 172.21.15.3:0/3276264051 <== osd.5 v2:172.21.15.187:6808/916876636 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0fa4060fa0 con 0x7f0f8c0aaf80 2023-08-11T14:02:37.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] conn(0x7f0f8c0aaf80 msgr2=0x7f0f8c0ad420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] conn(0x7f0f8c0aaf80 0x7f0f8c0ad420 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0f8c09ffe0 msgr2=0x7f0f8c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0f8c09ffe0 0x7f0f8c0a24a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f94006010 tx=0x7f0f9400b260 comp rx=0 tx=0).stop 2023-08-11T14:02:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 msgr2=0x7f0fa419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 --2- 172.21.15.3:0/3276264051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa4100040 0x7f0fa419efa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800ed00 comp rx=0 tx=0).stop 2023-08-11T14:02:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa2d9d700 1 -- 172.21.15.3:0/3276264051 reap_dead start 2023-08-11T14:02:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 shutdown_connections 2023-08-11T14:02:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.116+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 >> 172.21.15.3:0/3276264051 conn(0x7f0fa40fb390 msgr2=0x7f0fa410f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.117+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 shutdown_connections 2023-08-11T14:02:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.117+0000 7f0fa8dfa700 1 -- 172.21.15.3:0/3276264051 wait complete. 2023-08-11T14:02:37.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:37 smithi003 ceph-mon[97195]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:37 smithi103 ceph-mon[130434]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:37.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:37 smithi187 ceph-mon[130973]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:37.862 INFO:teuthology.orchestra.run.smithi003.stdout:335007449112 2023-08-11T14:02:37.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-08-11T14:02:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.000+0000 7f2937896700 1 -- 172.21.15.3:0/3724322430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 msgr2=0x7f2930103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.000+0000 7f2937896700 1 --2- 172.21.15.3:0/3724322430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f2930103370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f292c01a9a0 tx=0x7f292c01acb0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 -- 172.21.15.3:0/3724322430 shutdown_connections 2023-08-11T14:02:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 --2- 172.21.15.3:0/3724322430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930068440 0x7f29300688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 --2- 172.21.15.3:0/3724322430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f2930103370 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 --2- 172.21.15.3:0/3724322430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29301038b0 0x7f2930103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 -- 172.21.15.3:0/3724322430 >> 172.21.15.3:0/3724322430 conn(0x7f2930076920 msgr2=0x7f2930076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 -- 172.21.15.3:0/3724322430 shutdown_connections 2023-08-11T14:02:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.002+0000 7f2937896700 1 -- 172.21.15.3:0/3724322430 wait complete. 2023-08-11T14:02:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.004+0000 7f2937896700 1 Processor -- start 2023-08-11T14:02:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.004+0000 7f2937896700 1 -- start start 2023-08-11T14:02:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 0x7f293010f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301038b0 0x7f293010fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f29301a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301a0c50 con 0x7f29301058e0 2023-08-11T14:02:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301a0d90 con 0x7f29301038b0 2023-08-11T14:02:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2937896700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301a0f20 con 0x7f2930068440 2023-08-11T14:02:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2935632700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 0x7f293010f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2935632700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 0x7f293010f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39322/0 (socket says 172.21.15.3:39322) 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2934e31700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301038b0 0x7f293010fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.005+0000 7f2935632700 1 -- 172.21.15.3:0/3031222819 learned_addr learned my addr 172.21.15.3:0/3031222819 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935e33700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f29301a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 -- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301038b0 msgr2=0x7f293010fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301038b0 0x7f293010fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 -- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 msgr2=0x7f29301a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f29301a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c01a600 con 0x7f2930068440 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935e33700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29301058e0 0x7f29301a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2934e31700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301038b0 0x7f293010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:38.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.006+0000 7f2935632700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 0x7f293010f9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f292c01afd0 tx=0x7f292c003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.008+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c02d070 con 0x7f2930068440 2023-08-11T14:02:38.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.008+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f292c005e10 con 0x7f2930068440 2023-08-11T14:02:38.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.008+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c031dd0 con 0x7f2930068440 2023-08-11T14:02:38.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.008+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29301a11a0 con 0x7f2930068440 2023-08-11T14:02:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.008+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29301a1560 con 0x7f2930068440 2023-08-11T14:02:38.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.010+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f293004e620 con 0x7f2930068440 2023-08-11T14:02:38.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.014+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f292c004010 con 0x7f2930068440 2023-08-11T14:02:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.016+0000 7f29227fc700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f291c0a00e0 0x7f291c0a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f292c0dd2d0 con 0x7f2930068440 2023-08-11T14:02:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f2934e31700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f291c0a00e0 0x7f291c0a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f29227fc700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] conn(0x7f291c0ab080 0x7f291c0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f291c0adbf0 con 0x7f291c0ab080 2023-08-11T14:02:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f292c0dd680 con 0x7f2930068440 2023-08-11T14:02:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.017+0000 7f2935e33700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] conn(0x7f291c0ab080 0x7f291c0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.018+0000 7f2934e31700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f291c0a00e0 0x7f291c0a25a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2930104e70 tx=0x7f292400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.018+0000 7f2935e33700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] conn(0x7f291c0ab080 0x7f291c0ad520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.019+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== osd.1 v2:172.21.15.103:6800/1127248620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f291c0adbf0 con 0x7f291c0ab080 2023-08-11T14:02:38.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 -- 172.21.15.3:0/3026989562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 msgr2=0x7f1aa410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/3026989562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 0x7f1aa410f7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00b0a0 tx=0x7f1a9c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 -- 172.21.15.3:0/3026989562 shutdown_connections 2023-08-11T14:02:38.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/3026989562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 0x7f1aa410f7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/3026989562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/3026989562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa4100c60 0x7f1aa410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.956+0000 7f1aa94c8700 1 -- 172.21.15.3:0/3026989562 >> 172.21.15.3:0/3026989562 conn(0x7f1aa4076920 msgr2=0x7f1aa4076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.957+0000 7f1aa94c8700 1 -- 172.21.15.3:0/3026989562 shutdown_connections 2023-08-11T14:02:38.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.957+0000 7f1aa94c8700 1 -- 172.21.15.3:0/3026989562 wait complete. 2023-08-11T14:02:38.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.958+0000 7f1aa94c8700 1 Processor -- start 2023-08-11T14:02:38.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.958+0000 7f1aa94c8700 1 -- start start 2023-08-11T14:02:38.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 0x7f1aa41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa419d720 0x7f1aa419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa419e110 con 0x7f1aa4100340 2023-08-11T14:02:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa419e280 con 0x7f1aa410d3b0 2023-08-11T14:02:38.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa94c8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa419e440 con 0x7f1aa419d720 2023-08-11T14:02:38.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48928/0 (socket says 172.21.15.3:48928) 2023-08-11T14:02:38.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.959+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2167233082 learned_addr learned my addr 172.21.15.3:0/2167233082 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:38.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa27fc700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 0x7f1aa41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa37fe700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa419d720 0x7f1aa419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2167233082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa419d720 msgr2=0x7f1aa419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa2ffd700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1aa419d720 0x7f1aa419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2167233082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 msgr2=0x7f1aa41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa2ffd700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa410d3b0 0x7f1aa41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.960+0000 7f1aa2ffd700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009d00 con 0x7f1aa4100340 2023-08-11T14:02:38.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.961+0000 7f1aa2ffd700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4198d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.961+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94009790 con 0x7f1aa4100340 2023-08-11T14:02:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.961+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9400cbd0 con 0x7f1aa4100340 2023-08-11T14:02:38.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.961+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94018680 con 0x7f1aa4100340 2023-08-11T14:02:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.961+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa41a59d0 con 0x7f1aa4100340 2023-08-11T14:02:38.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.962+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa41a5e80 con 0x7f1aa4100340 2023-08-11T14:02:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.963+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1aa4112de0 con 0x7f1aa4100340 2023-08-11T14:02:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.965+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f1a9400cd40 con 0x7f1aa4100340 2023-08-11T14:02:38.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.967+0000 7f1a8bfff700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a8c0a0130 0x7f1a8c0a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.967+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1a94014070 con 0x7f1aa4100340 2023-08-11T14:02:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.967+0000 7f1a8bfff700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] conn(0x7f1a8c0ab0d0 0x7f1a8c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.967+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a8c0adc40 con 0x7f1a8c0ab0d0 2023-08-11T14:02:38.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.967+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a940cdd70 con 0x7f1aa4100340 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.968+0000 7f1aa37fe700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] conn(0x7f1a8c0ab0d0 0x7f1a8c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.968+0000 7f1aa37fe700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] conn(0x7f1a8c0ab0d0 0x7f1a8c0ad570 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.969+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== osd.3 v2:172.21.15.3:6810/2193340429 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1a8c0adc40 con 0x7f1a8c0ab0d0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.969+0000 7f1aa27fc700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a8c0a0130 0x7f1a8c0a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.970+0000 7f1aa27fc700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a8c0a0130 0x7f1a8c0a25f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1aa419a390 tx=0x7f1a98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.991+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 --> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1aa404e620 con 0x7f1a8c0ab0d0 2023-08-11T14:02:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.991+0000 7f1a8bfff700 1 -- 172.21.15.3:0/2167233082 <== osd.3 v2:172.21.15.3:6810/2193340429 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1aa404e620 con 0x7f1a8c0ab0d0 2023-08-11T14:02:38.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] conn(0x7f1a8c0ab0d0 msgr2=0x7f1a8c0ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] conn(0x7f1a8c0ab0d0 0x7f1a8c0ad570 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a8c0a0130 msgr2=0x7f1a8c0a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a8c0a0130 0x7f1a8c0a25f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1aa419a390 tx=0x7f1a98008040 comp rx=0 tx=0).stop 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 msgr2=0x7f1aa4198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.992+0000 7f1aa94c8700 1 --2- 172.21.15.3:0/2167233082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1aa4100340 0x7f1aa4198d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ed60 comp rx=0 tx=0).stop 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.993+0000 7f1aa37fe700 1 -- 172.21.15.3:0/2167233082 reap_dead start 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.993+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 shutdown_connections 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.993+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 >> 172.21.15.3:0/2167233082 conn(0x7f1aa4076920 msgr2=0x7f1aa410e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.993+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 shutdown_connections 2023-08-11T14:02:38.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:37.993+0000 7f1aa94c8700 1 -- 172.21.15.3:0/2167233082 wait complete. 2023-08-11T14:02:38.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.054+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 --> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29301087e0 con 0x7f291c0ab080 2023-08-11T14:02:38.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.055+0000 7f29227fc700 1 -- 172.21.15.3:0/3031222819 <== osd.1 v2:172.21.15.103:6800/1127248620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f292800a510 con 0x7f291c0ab080 2023-08-11T14:02:38.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.055+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] conn(0x7f291c0ab080 msgr2=0x7f291c0ad520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.055+0000 7f2937896700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] conn(0x7f291c0ab080 0x7f291c0ad520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f291c0a00e0 msgr2=0x7f291c0a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f291c0a00e0 0x7f291c0a25a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2930104e70 tx=0x7f292400d040 comp rx=0 tx=0).stop 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 msgr2=0x7f293010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 --2- 172.21.15.3:0/3031222819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2930068440 0x7f293010f9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f292c01afd0 tx=0x7f292c003bb0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2935e33700 1 -- 172.21.15.3:0/3031222819 reap_dead start 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 shutdown_connections 2023-08-11T14:02:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 >> 172.21.15.3:0/3031222819 conn(0x7f2930076920 msgr2=0x7f29300fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 shutdown_connections 2023-08-11T14:02:38.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.056+0000 7f2937896700 1 -- 172.21.15.3:0/3031222819 wait complete. 2023-08-11T14:02:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.136+0000 7f0a36a63700 1 -- 172.21.15.3:0/3342119610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a3010f180 msgr2=0x7f0a3010cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.136+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3342119610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a3010f180 0x7f0a3010cc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a200099c0 tx=0x7f0a20009cd0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.137+0000 7f0a36a63700 1 -- 172.21.15.3:0/3342119610 shutdown_connections 2023-08-11T14:02:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.137+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3342119610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a3011a100 0x7f0a3011c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.137+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3342119610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a3010f180 0x7f0a3010cc60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.137+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3342119610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a3010d1a0 0x7f0a3010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.137+0000 7f0a36a63700 1 -- 172.21.15.3:0/3342119610 >> 172.21.15.3:0/3342119610 conn(0x7f0a3006cb20 msgr2=0x7f0a3006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.138+0000 7f0a36a63700 1 -- 172.21.15.3:0/3342119610 shutdown_connections 2023-08-11T14:02:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.142+0000 7f0a36a63700 1 -- 172.21.15.3:0/3342119610 wait complete. 2023-08-11T14:02:38.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.144+0000 7f0a36a63700 1 Processor -- start 2023-08-11T14:02:38.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 -- start start 2023-08-11T14:02:38.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a3010f180 0x7f0a301ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a3011a100 0x7f0a301abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 0x7f0a301a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301a63d0 con 0x7f0a3011a100 2023-08-11T14:02:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301a6510 con 0x7f0a3010f180 2023-08-11T14:02:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.145+0000 7f0a36a63700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301a66d0 con 0x7f0a301a5950 2023-08-11T14:02:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.146+0000 7f0a35000700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 0x7f0a301a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.146+0000 7f0a35000700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 0x7f0a301a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39334/0 (socket says 172.21.15.3:39334) 2023-08-11T14:02:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.146+0000 7f0a35000700 1 -- 172.21.15.3:0/3138703759 learned_addr learned my addr 172.21.15.3:0/3138703759 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:38.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a2f7fe700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a3011a100 0x7f0a301abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a35000700 1 -- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a3010f180 msgr2=0x7f0a301ab8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a35000700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a3010f180 0x7f0a301ab8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a35000700 1 -- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a3011a100 msgr2=0x7f0a301abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a35000700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a3011a100 0x7f0a301abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.147+0000 7f0a35000700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200095f0 con 0x7f0a301a5950 2023-08-11T14:02:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.148+0000 7f0a35000700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 0x7f0a301a5e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b970 tx=0x7f0a2800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.148+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800c930 con 0x7f0a301a5950 2023-08-11T14:02:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.148+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a28014440 con 0x7f0a301a5950 2023-08-11T14:02:38.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.148+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a280075b0 con 0x7f0a301a5950 2023-08-11T14:02:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.148+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a301b2750 con 0x7f0a301a5950 2023-08-11T14:02:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.149+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a301b2c80 con 0x7f0a301a5950 2023-08-11T14:02:38.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.150+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a3004e620 con 0x7f0a301a5950 2023-08-11T14:02:38.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.154+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f0a280145b0 con 0x7f0a301a5950 2023-08-11T14:02:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.155+0000 7f0a2d7fa700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a180a0130 0x7f0a180a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.156+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0a280cb3d0 con 0x7f0a301a5950 2023-08-11T14:02:38.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.156+0000 7f0a2d7fa700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] conn(0x7f0a180ab0d0 0x7f0a180ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.156+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a180adc40 con 0x7f0a180ab0d0 2023-08-11T14:02:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.156+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a280cb7e0 con 0x7f0a301a5950 2023-08-11T14:02:38.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.156+0000 7f0a2ffff700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a180a0130 0x7f0a180a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.158+0000 7f0a2ffff700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a180a0130 0x7f0a180a25f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a20003800 tx=0x7f0a20000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.159+0000 7f0a2f7fe700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] conn(0x7f0a180ab0d0 0x7f0a180ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:38.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.159+0000 7f0a2f7fe700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] conn(0x7f0a180ab0d0 0x7f0a180ad570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:38.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.161+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== osd.10 v2:172.21.15.103:6824/1382085877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0a180adc40 con 0x7f0a180ab0d0 2023-08-11T14:02:38.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.182+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 --> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a30070d60 con 0x7f0a180ab0d0 2023-08-11T14:02:38.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.182+0000 7f0a2d7fa700 1 -- 172.21.15.3:0/3138703759 <== osd.10 v2:172.21.15.103:6824/1382085877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0a30070d60 con 0x7f0a180ab0d0 2023-08-11T14:02:38.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] conn(0x7f0a180ab0d0 msgr2=0x7f0a180ad570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] conn(0x7f0a180ab0d0 0x7f0a180ad570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a180a0130 msgr2=0x7f0a180a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a180a0130 0x7f0a180a25f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a20003800 tx=0x7f0a20000bc0 comp rx=0 tx=0).stop 2023-08-11T14:02:38.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 msgr2=0x7f0a301a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:38.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 --2- 172.21.15.3:0/3138703759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a301a5950 0x7f0a301a5e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b970 tx=0x7f0a2800bc80 comp rx=0 tx=0).stop 2023-08-11T14:02:38.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a35000700 1 -- 172.21.15.3:0/3138703759 reap_dead start 2023-08-11T14:02:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 shutdown_connections 2023-08-11T14:02:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 >> 172.21.15.3:0/3138703759 conn(0x7f0a3006cb20 msgr2=0x7f0a3006f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 shutdown_connections 2023-08-11T14:02:38.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.183+0000 7f0a36a63700 1 -- 172.21.15.3:0/3138703759 wait complete. 2023-08-11T14:02:38.796 INFO:teuthology.orchestra.run.smithi003.stdout:249108103207 2023-08-11T14:02:38.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-08-11T14:02:39.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:38 smithi003 ceph-mon[97195]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:39.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 -- 172.21.15.3:0/758738827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe644101350 msgr2=0x7fe6441017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 --2- 172.21.15.3:0/758738827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe644101350 0x7fe6441017d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009990 tx=0x7fe62c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 -- 172.21.15.3:0/758738827 shutdown_connections 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 --2- 172.21.15.3:0/758738827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe644101d10 0x7fe64410c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 --2- 172.21.15.3:0/758738827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6440ffc10 0x7fe6440ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 --2- 172.21.15.3:0/758738827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe644101350 0x7fe6441017d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 -- 172.21.15.3:0/758738827 >> 172.21.15.3:0/758738827 conn(0x7fe644076910 msgr2=0x7fe644076d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 -- 172.21.15.3:0/758738827 shutdown_connections 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.799+0000 7fe6496ad700 1 -- 172.21.15.3:0/758738827 wait complete. 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 Processor -- start 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 -- start start 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 0x7fe64419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644101350 0x7fe64419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:39.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe644101d10 0x7fe6441969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644197010 con 0x7fe644101d10 2023-08-11T14:02:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644197180 con 0x7fe644101350 2023-08-11T14:02:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.801+0000 7fe6496ad700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644197340 con 0x7fe6440ffc10 2023-08-11T14:02:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe6437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe644101d10 0x7fe6441969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:39.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 0x7fe64419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 0x7fe64419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39342/0 (socket says 172.21.15.3:39342) 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 -- 172.21.15.3:0/555644360 learned_addr learned my addr 172.21.15.3:0/555644360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe63bfff700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644101350 0x7fe64419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 -- 172.21.15.3:0/555644360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644101350 msgr2=0x7fe64419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe644101350 0x7fe64419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 -- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe644101d10 msgr2=0x7fe6441969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe644101d10 0x7fe6441969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0095f0 con 0x7fe6440ffc10 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.802+0000 7fe642ffd700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 0x7fe64419c920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9a0 tx=0x7fe63400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.803+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634009790 con 0x7fe6440ffc10 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.803+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe634010430 con 0x7fe6440ffc10 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.803+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe644197620 con 0x7fe6440ffc10 2023-08-11T14:02:39.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.803+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634018680 con 0x7fe6440ffc10 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.803+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6441a3910 con 0x7fe6440ffc10 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.804+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe64410fb60 con 0x7fe6440ffc10 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.807+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fe6340105a0 con 0x7fe6440ffc10 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe640ff9700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6300a0060 0x7fe6300a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe634014070 con 0x7fe6440ffc10 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe63bfff700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6300a0060 0x7fe6300a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe640ff9700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] conn(0x7fe6300ab000 0x7fe6300ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:39.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6300adb70 con 0x7fe6300ab000 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6340cc5e0 con 0x7fe6440ffc10 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.810+0000 7fe6437fe700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] conn(0x7fe6300ab000 0x7fe6300ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.811+0000 7fe63bfff700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6300a0060 0x7fe6300a2520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b2e0 tx=0x7fe62c00b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.811+0000 7fe6437fe700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] conn(0x7fe6300ab000 0x7fe6300ad4a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.812+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== osd.6 v2:172.21.15.3:6818/3251538985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe6300adb70 con 0x7fe6300ab000 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.829+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 --> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe644069770 con 0x7fe6300ab000 2023-08-11T14:02:39.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.830+0000 7fe640ff9700 1 -- 172.21.15.3:0/555644360 <== osd.6 v2:172.21.15.3:6818/3251538985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe644069770 con 0x7fe6300ab000 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.830+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] conn(0x7fe6300ab000 msgr2=0x7fe6300ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.830+0000 7fe6496ad700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] conn(0x7fe6300ab000 0x7fe6300ad4a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6300a0060 msgr2=0x7fe6300a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6300a0060 0x7fe6300a2520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b2e0 tx=0x7fe62c00b610 comp rx=0 tx=0).stop 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 msgr2=0x7fe64419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 --2- 172.21.15.3:0/555644360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6440ffc10 0x7fe64419c920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe63400e9a0 tx=0x7fe63400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:02:39.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6437fe700 1 -- 172.21.15.3:0/555644360 reap_dead start 2023-08-11T14:02:39.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 shutdown_connections 2023-08-11T14:02:39.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 >> 172.21.15.3:0/555644360 conn(0x7fe644076910 msgr2=0x7fe644068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:39.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 shutdown_connections 2023-08-11T14:02:39.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:38.831+0000 7fe6496ad700 1 -- 172.21.15.3:0/555644360 wait complete. 2023-08-11T14:02:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:38 smithi187 ceph-mon[130973]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:39.299 INFO:teuthology.orchestra.run.smithi003.stdout:158913790004 2023-08-11T14:02:39.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-08-11T14:02:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:39.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:38 smithi103 ceph-mon[130434]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-08-11T14:02:39.985 INFO:teuthology.orchestra.run.smithi003.stdout:55834574918 2023-08-11T14:02:39.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-08-11T14:02:40.092 INFO:teuthology.orchestra.run.smithi003.stdout:107374182462 2023-08-11T14:02:40.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-08-11T14:02:40.371 INFO:teuthology.orchestra.run.smithi003.stdout:304942678044 2023-08-11T14:02:40.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-08-11T14:02:40.415 INFO:teuthology.orchestra.run.smithi003.stdout:137438953528 2023-08-11T14:02:40.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-08-11T14:02:40.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 -- 172.21.15.3:0/2733197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 msgr2=0x7f97dc100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/2733197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc100c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009990 tx=0x7f97c8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 -- 172.21.15.3:0/2733197009 shutdown_connections 2023-08-11T14:02:40.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/2733197009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 0x7f97dc10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/2733197009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97dc106780 0x7f97dc106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/2733197009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc100c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 -- 172.21.15.3:0/2733197009 >> 172.21.15.3:0/2733197009 conn(0x7f97dc0fc2d0 msgr2=0x7f97dc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:40.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 -- 172.21.15.3:0/2733197009 shutdown_connections 2023-08-11T14:02:40.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.834+0000 7f97e1b4d700 1 -- 172.21.15.3:0/2733197009 wait complete. 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.835+0000 7f97e1b4d700 1 Processor -- start 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.835+0000 7f97e1b4d700 1 -- start start 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 0x7f97dc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97dc106780 0x7f97dc196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:40.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc196d80 con 0x7f97dc100780 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc196ef0 con 0x7f97dc106780 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e1b4d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1970b0 con 0x7f97dc101140 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48976/0 (socket says 172.21.15.3:48976) 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 -- 172.21.15.3:0/3417785493 learned_addr learned my addr 172.21.15.3:0/3417785493 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:40.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97dbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 0x7f97dc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e134c700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97dc106780 0x7f97dc196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 -- 172.21.15.3:0/3417785493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 msgr2=0x7f97dc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 0x7f97dc19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 -- 172.21.15.3:0/3417785493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97dc106780 msgr2=0x7f97dc196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97dc106780 0x7f97dc196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.836+0000 7f97e0b4b700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80095f0 con 0x7f97dc100780 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97dbfff700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97dc101140 0x7f97dc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:40.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97e0b4b700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc19c640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97d000e9a0 tx=0x7f97d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:40.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0009790 con 0x7f97dc100780 2023-08-11T14:02:40.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d0010430 con 0x7f97dc100780 2023-08-11T14:02:40.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc197390 con 0x7f97dc100780 2023-08-11T14:02:40.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0018680 con 0x7f97dc100780 2023-08-11T14:02:40.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.837+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc1a3630 con 0x7f97dc100780 2023-08-11T14:02:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.838+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97dc110a70 con 0x7f97dc100780 2023-08-11T14:02:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.840+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f97d00105a0 con 0x7f97dc100780 2023-08-11T14:02:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97d9ffb700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f97cc09ffe0 0x7f97cc0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f97d0014070 con 0x7f97dc100780 2023-08-11T14:02:40.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97dbfff700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f97cc09ffe0 0x7f97cc0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:40.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97d9ffb700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] conn(0x7f97cc0aaf80 0x7f97cc0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97cc0adaf0 con 0x7f97cc0aaf80 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97d00cbec0 con 0x7f97dc100780 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.842+0000 7f97e134c700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] conn(0x7f97cc0aaf80 0x7f97cc0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.843+0000 7f97dbfff700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f97cc09ffe0 0x7f97cc0a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b2e0 tx=0x7f97c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.843+0000 7f97e134c700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] conn(0x7f97cc0aaf80 0x7f97cc0ad420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.844+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== osd.9 v2:172.21.15.3:6826/3936829119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f97cc0adaf0 con 0x7f97cc0aaf80 2023-08-11T14:02:40.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.861+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 --> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97dc04ee70 con 0x7f97cc0aaf80 2023-08-11T14:02:40.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.861+0000 7f97d9ffb700 1 -- 172.21.15.3:0/3417785493 <== osd.9 v2:172.21.15.3:6826/3936829119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f97dc04ee70 con 0x7f97cc0aaf80 2023-08-11T14:02:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] conn(0x7f97cc0aaf80 msgr2=0x7f97cc0ad420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] conn(0x7f97cc0aaf80 0x7f97cc0ad420 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f97cc09ffe0 msgr2=0x7f97cc0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f97cc09ffe0 0x7f97cc0a24a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b2e0 tx=0x7f97c8005960 comp rx=0 tx=0).stop 2023-08-11T14:02:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 msgr2=0x7f97dc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 --2- 172.21.15.3:0/3417785493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97dc100780 0x7f97dc19c640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f97d000e9a0 tx=0x7f97d000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e134c700 1 -- 172.21.15.3:0/3417785493 reap_dead start 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 shutdown_connections 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 >> 172.21.15.3:0/3417785493 conn(0x7f97dc0fc2d0 msgr2=0x7f97dc104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 shutdown_connections 2023-08-11T14:02:40.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:40.862+0000 7f97e1b4d700 1 -- 172.21.15.3:0/3417785493 wait complete. 2023-08-11T14:02:41.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 -- 172.21.15.3:0/3506876983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304100940 msgr2=0x7f5304100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3506876983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304100940 0x7f5304100dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009990 tx=0x7f52f8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 -- 172.21.15.3:0/3506876983 shutdown_connections 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3506876983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 0x7f530410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3506876983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304106940 0x7f5304106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3506876983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304100940 0x7f5304100dc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.278+0000 7f5308ac2700 1 -- 172.21.15.3:0/3506876983 >> 172.21.15.3:0/3506876983 conn(0x7f53040fc410 msgr2=0x7f53040fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:41.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.279+0000 7f5308ac2700 1 -- 172.21.15.3:0/3506876983 shutdown_connections 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.279+0000 7f5308ac2700 1 -- 172.21.15.3:0/3506876983 wait complete. 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 Processor -- start 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 -- start start 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 0x7f5304072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 0x7f5304073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304106940 0x7f53040736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a4e80 con 0x7f5304100940 2023-08-11T14:02:41.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a4ff0 con 0x7f5304106940 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f5308ac2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041a51b0 con 0x7f5304101300 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f530259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 0x7f5304072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f530259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 0x7f5304072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49000/0 (socket says 172.21.15.3:49000) 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.280+0000 7f530259c700 1 -- 172.21.15.3:0/3325111945 learned_addr learned my addr 172.21.15.3:0/3325111945 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f5301d9b700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 0x7f5304073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f5302d9d700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304106940 0x7f53040736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:41.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 -- 172.21.15.3:0/3325111945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 msgr2=0x7f5304073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 0x7f5304073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 -- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304106940 msgr2=0x7f53040736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5304106940 0x7f53040736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80095f0 con 0x7f5304100940 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f5301d9b700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5304101300 0x7f5304073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f530259c700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 0x7f5304072c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e9a0 tx=0x7f52f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4009790 con 0x7f5304100940 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041a5490 con 0x7f5304100940 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52f400cbd0 con 0x7f5304100940 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.281+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4018680 con 0x7f5304100940 2023-08-11T14:02:41.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.282+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53041a59e0 con 0x7f5304100940 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.282+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f530404e620 con 0x7f5304100940 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.285+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f52f400cd40 con 0x7f5304100940 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f52f37fe700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52ec0a0060 0x7f52ec0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f52f4014070 con 0x7f5304100940 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f5301d9b700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52ec0a0060 0x7f52ec0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f52f37fe700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] conn(0x7f52ec0ab000 0x7f52ec0ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52ec0adb70 con 0x7f52ec0ab000 2023-08-11T14:02:41.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.286+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52f40cc360 con 0x7f5304100940 2023-08-11T14:02:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.287+0000 7f5301d9b700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52ec0a0060 0x7f52ec0a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006010 tx=0x7f52f800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.287+0000 7f5302d9d700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] conn(0x7f52ec0ab000 0x7f52ec0ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.287+0000 7f5302d9d700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] conn(0x7f52ec0ab000 0x7f52ec0ad4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.288+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== osd.7 v2:172.21.15.103:6816/3619083157 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f52ec0adb70 con 0x7f52ec0ab000 2023-08-11T14:02:41.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.306+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 --> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5304110be0 con 0x7f52ec0ab000 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f52f37fe700 1 -- 172.21.15.3:0/3325111945 <== osd.7 v2:172.21.15.103:6816/3619083157 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f52fc00f040 con 0x7f52ec0ab000 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] conn(0x7f52ec0ab000 msgr2=0x7f52ec0ad4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] conn(0x7f52ec0ab000 0x7f52ec0ad4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52ec0a0060 msgr2=0x7f52ec0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52ec0a0060 0x7f52ec0a2520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006010 tx=0x7f52f800b260 comp rx=0 tx=0).stop 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 msgr2=0x7f5304072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:41.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 --2- 172.21.15.3:0/3325111945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5304100940 0x7f5304072c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e9a0 tx=0x7f52f400ed60 comp rx=0 tx=0).stop 2023-08-11T14:02:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5302d9d700 1 -- 172.21.15.3:0/3325111945 reap_dead start 2023-08-11T14:02:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 shutdown_connections 2023-08-11T14:02:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 >> 172.21.15.3:0/3325111945 conn(0x7f53040fc410 msgr2=0x7f53040fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.307+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 shutdown_connections 2023-08-11T14:02:41.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:41.308+0000 7f5308ac2700 1 -- 172.21.15.3:0/3325111945 wait complete. 2023-08-11T14:02:41.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:41 smithi003 ceph-mon[97195]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:41.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:41 smithi187 ceph-mon[130973]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:41.547 INFO:teuthology.orchestra.run.smithi003.stdout:283467841571 2023-08-11T14:02:41.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-08-11T14:02:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:41 smithi103 ceph-mon[130434]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2023-08-11T14:02:41.974 INFO:teuthology.orchestra.run.smithi003.stdout:193273528368 2023-08-11T14:02:41.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-08-11T14:02:42.152 INFO:teuthology.orchestra.run.smithi003.stdout:214748364843 2023-08-11T14:02:42.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-08-11T14:02:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:42.465 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:42.809 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:43 smithi003 ceph-mon[97195]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-08-11T14:02:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:43 smithi103 ceph-mon[130434]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-08-11T14:02:43.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:43 smithi187 ceph-mon[130973]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-08-11T14:02:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:44.203 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:44.693 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:45 smithi003 ceph-mon[97195]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:02:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:45 smithi103 ceph-mon[130434]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:02:45.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:45 smithi187 ceph-mon[130973]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 -- 172.21.15.3:0/3950333315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0106940 msgr2=0x7fb1e0106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/3950333315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0106940 0x7fb1e0106d20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009990 tx=0x7fb1d0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 -- 172.21.15.3:0/3950333315 shutdown_connections 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/3950333315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/3950333315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0106940 0x7fb1e0106d20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/3950333315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e0100940 0x7fb1e0100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 -- 172.21.15.3:0/3950333315 >> 172.21.15.3:0/3950333315 conn(0x7fb1e00fc410 msgr2=0x7fb1e00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 -- 172.21.15.3:0/3950333315 shutdown_connections 2023-08-11T14:02:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.756+0000 7fb1e54d4700 1 -- 172.21.15.3:0/3950333315 wait complete. 2023-08-11T14:02:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.757+0000 7fb1e54d4700 1 Processor -- start 2023-08-11T14:02:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 -- start start 2023-08-11T14:02:46.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 0x7fb1e0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e0106940 0x7fb1e01a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0073a50 con 0x7fb1e0100940 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e01a51c0 con 0x7fb1e0101300 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1e54d4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e01a5330 con 0x7fb1e0106940 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1d6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 0x7fb1e0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.758+0000 7fb1deffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:58314/0 (socket says 172.21.15.3:58314) 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1d6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 0x7fb1e0072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51820/0 (socket says 172.21.15.3:51820) 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 -- 172.21.15.3:0/2129300313 learned_addr learned my addr 172.21.15.3:0/2129300313 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 -- 172.21.15.3:0/2129300313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e0106940 msgr2=0x7fb1e01a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e0106940 0x7fb1e01a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 -- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 msgr2=0x7fb1e0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 0x7fb1e0072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 -- 172.21.15.3:0/2129300313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00095f0 con 0x7fb1e0101300 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.759+0000 7fb1deffd700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800d980 tx=0x7fb1c800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.760+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8004220 con 0x7fb1e0101300 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.760+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c8007640 con 0x7fb1e0101300 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.760+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8005610 con 0x7fb1e0101300 2023-08-11T14:02:46.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.760+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e01a55b0 con 0x7fb1e0101300 2023-08-11T14:02:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.760+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e01a5a60 con 0x7fb1e0101300 2023-08-11T14:02:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.762+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e004e620 con 0x7fb1e0101300 2023-08-11T14:02:46.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.764+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fb1c8004750 con 0x7fb1e0101300 2023-08-11T14:02:46.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.766+0000 7fb1dcff9700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 0x7fb1cc0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.766+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb1c80cb4d0 con 0x7fb1e0101300 2023-08-11T14:02:46.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.767+0000 7fb1d6ffd700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 0x7fb1cc0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.767+0000 7fb1d6ffd700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 0x7fb1cc0a2520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b2e0 tx=0x7fb1d0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:46.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.770+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb1c8094da0 con 0x7fb1e0101300 2023-08-11T14:02:46.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 -- 172.21.15.3:0/1580201688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 msgr2=0x7fd828106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 --2- 172.21.15.3:0/1580201688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828106d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd824009990 tx=0x7fd824009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 -- 172.21.15.3:0/1580201688 shutdown_connections 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 --2- 172.21.15.3:0/1580201688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 0x7fd82810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 --2- 172.21.15.3:0/1580201688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828106d20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 --2- 172.21.15.3:0/1580201688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 0x7fd828100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.918+0000 7fd8302df700 1 -- 172.21.15.3:0/1580201688 >> 172.21.15.3:0/1580201688 conn(0x7fd8280fc410 msgr2=0x7fd8280fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.919+0000 7fd8302df700 1 -- 172.21.15.3:0/1580201688 shutdown_connections 2023-08-11T14:02:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.919+0000 7fd8302df700 1 -- 172.21.15.3:0/1580201688 wait complete. 2023-08-11T14:02:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 Processor -- start 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 -- start start 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 0x7fd82819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 0x7fd82819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281992c0 con 0x7fd828106940 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828199430 con 0x7fd828100940 2023-08-11T14:02:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.920+0000 7fd8302df700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281995f0 con 0x7fd828101300 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51830/0 (socket says 172.21.15.3:51830) 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 -- 172.21.15.3:0/607129767 learned_addr learned my addr 172.21.15.3:0/607129767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e07b700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 0x7fd82819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82d87a700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 0x7fd82819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 -- 172.21.15.3:0/607129767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 msgr2=0x7fd82819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 0x7fd82819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 -- 172.21.15.3:0/607129767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 msgr2=0x7fd82819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 0x7fd82819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.921+0000 7fd82e87c700 1 -- 172.21.15.3:0/607129767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240095f0 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.922+0000 7fd82e87c700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ef30 tx=0x7fd82000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.922+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820018070 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.922+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82000f040 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.922+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820007760 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.923+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd828199870 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.923+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281a5aa0 con 0x7fd828106940 2023-08-11T14:02:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.924+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82804e620 con 0x7fd828106940 2023-08-11T14:02:46.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.927+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fd82000c940 con 0x7fd828106940 2023-08-11T14:02:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.929+0000 7fd81f7fe700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 0x7fd8140a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.929+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd8200caf80 con 0x7fd828106940 2023-08-11T14:02:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.931+0000 7fd82e07b700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 0x7fd8140a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:46.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.932+0000 7fd82e07b700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 0x7fd8140a25f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd82400b2e0 tx=0x7fd824005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:46.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:46.932+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd820098360 con 0x7fd828106940 2023-08-11T14:02:47.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.062+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fb1e0110a20 con 0x7fb1e0101300 2023-08-11T14:02:47.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.063+0000 7fb1dcff9700 1 -- 172.21.15.3:0/2129300313 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb1c80983c0 con 0x7fb1e0101300 2023-08-11T14:02:47.066 INFO:teuthology.orchestra.run.smithi003.stdout:335007449115 2023-08-11T14:02:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.065+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 msgr2=0x7fb1cc0a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:47.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.065+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 0x7fb1cc0a2520 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b2e0 tx=0x7fb1d0005960 comp rx=0 tx=0).stop 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.065+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 msgr2=0x7fb1e0073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.065+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800d980 tx=0x7fb1c800dc90 comp rx=0 tx=0).stop 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 shutdown_connections 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e0106940 0x7fb1e01a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1e0100940 0x7fb1e0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb1cc0a0060 0x7fb1cc0a2520 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 --2- 172.21.15.3:0/2129300313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e0101300 0x7fb1e0073160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 >> 172.21.15.3:0/2129300313 conn(0x7fb1e00fc410 msgr2=0x7fb1e00fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 shutdown_connections 2023-08-11T14:02:47.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.066+0000 7fb1e54d4700 1 -- 172.21.15.3:0/2129300313 wait complete. 2023-08-11T14:02:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.187+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd828199fd0 con 0x7fd828106940 2023-08-11T14:02:47.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.189+0000 7fd81f7fe700 1 -- 172.21.15.3:0/607129767 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd820022020 con 0x7fd828106940 2023-08-11T14:02:47.191 INFO:teuthology.orchestra.run.smithi003.stdout:77309411395 2023-08-11T14:02:47.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 msgr2=0x7fd8140a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:47.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 0x7fd8140a25f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd82400b2e0 tx=0x7fd824005fd0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 msgr2=0x7fd828198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd82000ef30 tx=0x7fd82000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 shutdown_connections 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd828106940 0x7fd828198b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8140a0130 0x7fd8140a25f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd828100940 0x7fd82819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 --2- 172.21.15.3:0/607129767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd828101300 0x7fd82819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:47.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 >> 172.21.15.3:0/607129767 conn(0x7fd8280fc410 msgr2=0x7fd82810bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:47.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 shutdown_connections 2023-08-11T14:02:47.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:47.192+0000 7fd8302df700 1 -- 172.21.15.3:0/607129767 wait complete. 2023-08-11T14:02:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:47 smithi003 ceph-mon[97195]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:47 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2129300313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T14:02:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:47 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/607129767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T14:02:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:47 smithi103 ceph-mon[130434]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:47 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2129300313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T14:02:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:47 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/607129767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T14:02:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:47 smithi187 ceph-mon[130973]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:47 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2129300313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-08-11T14:02:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:47 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/607129767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-08-11T14:02:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.180+0000 7f56adde8700 1 -- 172.21.15.3:0/1077275614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a81037a0 msgr2=0x7f56a81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.180+0000 7f56adde8700 1 --2- 172.21.15.3:0/1077275614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a81037a0 0x7f56a81077f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f569c009990 tx=0x7f569c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 -- 172.21.15.3:0/1077275614 shutdown_connections 2023-08-11T14:02:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 --2- 172.21.15.3:0/1077275614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a8107ed0 0x7f56a810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 --2- 172.21.15.3:0/1077275614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 --2- 172.21.15.3:0/1077275614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a81037a0 0x7f56a81077f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 -- 172.21.15.3:0/1077275614 >> 172.21.15.3:0/1077275614 conn(0x7f56a80fe660 msgr2=0x7f56a8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 -- 172.21.15.3:0/1077275614 shutdown_connections 2023-08-11T14:02:48.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.181+0000 7f56adde8700 1 -- 172.21.15.3:0/1077275614 wait complete. 2023-08-11T14:02:48.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.182+0000 7f56adde8700 1 Processor -- start 2023-08-11T14:02:48.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.182+0000 7f56adde8700 1 -- start start 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 0x7f56a819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a8107ed0 0x7f56a8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81991a0 con 0x7f56a8107ed0 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8199310 con 0x7f56a8102df0 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56adde8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81994d0 con 0x7f56a81037a0 2023-08-11T14:02:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56a6ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 0x7f56a819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56a6ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 0x7f56a819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36888/0 (socket says 172.21.15.3:36888) 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56a6ffd700 1 -- 172.21.15.3:0/3931715590 learned_addr learned my addr 172.21.15.3:0/3931715590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56a77fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.183+0000 7f56a7fff700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a8107ed0 0x7f56a8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 -- 172.21.15.3:0/3931715590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 msgr2=0x7f56a819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 0x7f56a819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 -- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a8107ed0 msgr2=0x7f56a8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a8107ed0 0x7f56a8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 -- 172.21.15.3:0/3931715590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0095f0 con 0x7f56a8102df0 2023-08-11T14:02:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a77fe700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a819eab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f569800d950 tx=0x7f569800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800f890 con 0x7f56a8102df0 2023-08-11T14:02:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5698010460 con 0x7f56a8102df0 2023-08-11T14:02:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.184+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a81997b0 con 0x7f56a8102df0 2023-08-11T14:02:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.185+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a81a5aa0 con 0x7f56a8102df0 2023-08-11T14:02:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.185+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800e610 con 0x7f56a8102df0 2023-08-11T14:02:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.186+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a804e620 con 0x7f56a8102df0 2023-08-11T14:02:48.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.207+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f56980108d0 con 0x7f56a8102df0 2023-08-11T14:02:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.210+0000 7f56a4ff9700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 0x7f56900a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.210+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f56980cb7e0 con 0x7f56a8102df0 2023-08-11T14:02:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.210+0000 7f56a6ffd700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 0x7f56900a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.210+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f56980cbc20 con 0x7f56a8102df0 2023-08-11T14:02:48.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.211+0000 7f56a6ffd700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 0x7f56900a2520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f569c005cd0 tx=0x7f569c005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:48.215 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:48.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 -- 172.21.15.3:0/3087747370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 msgr2=0x7f4c3010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3087747370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c3010d810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009f90 tx=0x7f4c2800b330 comp rx=0 tx=0).stop 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 -- 172.21.15.3:0/3087747370 shutdown_connections 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3087747370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c3010d810 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3087747370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 0x7f4c30106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3087747370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 -- 172.21.15.3:0/3087747370 >> 172.21.15.3:0/3087747370 conn(0x7f4c300fc410 msgr2=0x7f4c300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:48.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 -- 172.21.15.3:0/3087747370 shutdown_connections 2023-08-11T14:02:48.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.387+0000 7f4c3758c700 1 -- 172.21.15.3:0/3087747370 wait complete. 2023-08-11T14:02:48.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.388+0000 7f4c3758c700 1 Processor -- start 2023-08-11T14:02:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 -- start start 2023-08-11T14:02:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c30072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 0x7f4c3006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3006b510 con 0x7f4c30106940 2023-08-11T14:02:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3006b680 con 0x7f4c30101300 2023-08-11T14:02:48.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.390+0000 7f4c3758c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3006b840 con 0x7f4c30100940 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c35328700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c30072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c35328700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36914/0 (socket says 172.21.15.3:36914) 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c35328700 1 -- 172.21.15.3:0/3838072896 learned_addr learned my addr 172.21.15.3:0/3838072896 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c35b29700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 0x7f4c3006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 -- 172.21.15.3:0/3838072896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 msgr2=0x7f4c30071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 -- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 msgr2=0x7f4c3006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 0x7f4c3006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.391+0000 7f4c34b27700 1 -- 172.21.15.3:0/3838072896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0095f0 con 0x7f4c30101300 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c34b27700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c30072340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e9f0 tx=0x7f4c2000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200096d0 con 0x7f4c30101300 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2000cbd0 con 0x7f4c30101300 2023-08-11T14:02:48.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20018680 con 0x7f4c30101300 2023-08-11T14:02:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c28009c40 con 0x7f4c30101300 2023-08-11T14:02:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.392+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3006fb30 con 0x7f4c30101300 2023-08-11T14:02:48.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.394+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3004e620 con 0x7f4c30101300 2023-08-11T14:02:48.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.396+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f4c2000cd40 con 0x7f4c30101300 2023-08-11T14:02:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.398+0000 7f4c267fc700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 0x7f4c1c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.398+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4c20014070 con 0x7f4c30101300 2023-08-11T14:02:48.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.398+0000 7f4c35328700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 0x7f4c1c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.399+0000 7f4c35328700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 0x7f4c1c0a24a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005d10 tx=0x7f4c2c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:48.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.399+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4c20095320 con 0x7f4c30101300 2023-08-11T14:02:48.403 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:48.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.490+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f56a8112b60 con 0x7f56a8102df0 2023-08-11T14:02:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.491+0000 7f56a4ff9700 1 -- 172.21.15.3:0/3931715590 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5698016090 con 0x7f56a8102df0 2023-08-11T14:02:48.493 INFO:teuthology.orchestra.run.smithi003.stdout:158913790007 2023-08-11T14:02:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.493+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 msgr2=0x7f56900a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.493+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 0x7f56900a2520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f569c005cd0 tx=0x7f569c005bc0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.493+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 msgr2=0x7f56a819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.493+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a819eab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f569800d950 tx=0x7f569800dc60 comp rx=0 tx=0).stop 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 shutdown_connections 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56a8107ed0 0x7f56a8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f56900a0060 0x7f56900a2520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a8102df0 0x7f56a819eab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 --2- 172.21.15.3:0/3931715590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56a81037a0 0x7f56a819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 >> 172.21.15.3:0/3931715590 conn(0x7f56a80fe660 msgr2=0x7f56a80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 shutdown_connections 2023-08-11T14:02:48.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.494+0000 7f56adde8700 1 -- 172.21.15.3:0/3931715590 wait complete. 2023-08-11T14:02:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.671+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f4c3006bb60 con 0x7f4c30101300 2023-08-11T14:02:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.672+0000 7f4c267fc700 1 -- 172.21.15.3:0/3838072896 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c20098940 con 0x7f4c30101300 2023-08-11T14:02:48.678 INFO:teuthology.orchestra.run.smithi003.stdout:249108103210 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 msgr2=0x7f4c1c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 0x7f4c1c0a24a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005d10 tx=0x7f4c2c009310 comp rx=0 tx=0).stop 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 msgr2=0x7f4c30072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c30072340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e9f0 tx=0x7f4c2000ed00 comp rx=0 tx=0).stop 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 shutdown_connections 2023-08-11T14:02:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c30106940 0x7f4c3006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c30100940 0x7f4c30071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4c1c09ffe0 0x7f4c1c0a24a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 --2- 172.21.15.3:0/3838072896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c30101300 0x7f4c30072340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 >> 172.21.15.3:0/3838072896 conn(0x7f4c300fc410 msgr2=0x7f4c300fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 shutdown_connections 2023-08-11T14:02:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:48.674+0000 7f4c3758c700 1 -- 172.21.15.3:0/3838072896 wait complete. 2023-08-11T14:02:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:49.577 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:49 smithi003 ceph-mon[97195]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:49.577 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:49 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3931715590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T14:02:49.577 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:49 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3838072896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T14:02:49.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790004 got 158913790007 for osd.5 2023-08-11T14:02:49.604 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:49 smithi103 ceph-mon[130434]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:49 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3931715590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T14:02:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:49 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3838072896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T14:02:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:49 smithi187 ceph-mon[130973]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:49 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3931715590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-08-11T14:02:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:49 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3838072896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-08-11T14:02:50.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103210 for osd.8 2023-08-11T14:02:50.113 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:50.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 -- 172.21.15.3:0/3560413744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534106940 msgr2=0x7f1534106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:50.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 --2- 172.21.15.3:0/3560413744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534106940 0x7f1534106d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1530009990 tx=0x7f1530009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 -- 172.21.15.3:0/3560413744 shutdown_connections 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 --2- 172.21.15.3:0/3560413744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534101300 0x7f153410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 --2- 172.21.15.3:0/3560413744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534106940 0x7f1534106d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 --2- 172.21.15.3:0/3560413744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 0x7f1534100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 -- 172.21.15.3:0/3560413744 >> 172.21.15.3:0/3560413744 conn(0x7f15340fc410 msgr2=0x7f15340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 -- 172.21.15.3:0/3560413744 shutdown_connections 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.956+0000 7f153b627700 1 -- 172.21.15.3:0/3560413744 wait complete. 2023-08-11T14:02:50.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.957+0000 7f153b627700 1 Processor -- start 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 -- start start 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 0x7f153419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 0x7f1534198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15341990b0 con 0x7f1534101300 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1534199220 con 0x7f1534100940 2023-08-11T14:02:50.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f153b627700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15341993e0 con 0x7f1534106940 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f1538bc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f15393c3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 0x7f153419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f1538bc2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51872/0 (socket says 172.21.15.3:51872) 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f1538bc2700 1 -- 172.21.15.3:0/2941330647 learned_addr learned my addr 172.21.15.3:0/2941330647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.958+0000 7f1539bc4700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 0x7f1534198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 -- 172.21.15.3:0/2941330647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 msgr2=0x7f1534198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:50.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 0x7f1534198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 -- 172.21.15.3:0/2941330647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 msgr2=0x7f153419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:50.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 0x7f153419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 -- 172.21.15.3:0/2941330647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15300095f0 con 0x7f1534101300 2023-08-11T14:02:50.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1539bc4700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 0x7f1534198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f1538bc2700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f152400d950 tx=0x7f152400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524009940 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15341996c0 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1524010460 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400f620 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.959+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15341a59b0 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.960+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153404e620 con 0x7f1534101300 2023-08-11T14:02:50.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.962+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f15240108d0 con 0x7f1534101300 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.964+0000 7f152a7fc700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 0x7f15200a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.964+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f15240cb590 con 0x7f1534101300 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.964+0000 7f15393c3700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 0x7f15200a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.965+0000 7f15393c3700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 0x7f15200a24a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153000b5c0 tx=0x7f1530005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:49.968+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1524094e60 con 0x7f1534101300 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.192+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15341109e0 con 0x7f1534101300 2023-08-11T14:02:50.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.193+0000 7f152a7fc700 1 -- 172.21.15.3:0/2941330647 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1524098480 con 0x7f1534101300 2023-08-11T14:02:50.296 INFO:teuthology.orchestra.run.smithi003.stdout:107374182464 2023-08-11T14:02:50.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.196+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 msgr2=0x7f15200a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:50.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.196+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 0x7f15200a24a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153000b5c0 tx=0x7f1530005960 comp rx=0 tx=0).stop 2023-08-11T14:02:50.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.196+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 msgr2=0x7f153419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:50.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.196+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f152400d950 tx=0x7f152400dc60 comp rx=0 tx=0).stop 2023-08-11T14:02:50.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 shutdown_connections 2023-08-11T14:02:50.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1534106940 0x7f1534198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f152009ffe0 0x7f15200a24a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1534100940 0x7f153419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 --2- 172.21.15.3:0/2941330647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1534101300 0x7f153419ef00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:50.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 >> 172.21.15.3:0/2941330647 conn(0x7f15340fc410 msgr2=0x7f15340fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:50.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 shutdown_connections 2023-08-11T14:02:50.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:50.197+0000 7f153b627700 1 -- 172.21.15.3:0/2941330647 wait complete. 2023-08-11T14:02:50.645 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2941330647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T14:02:50.673 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449112 got 335007449115 for osd.11 2023-08-11T14:02:50.674 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2941330647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T14:02:50.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2941330647' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-08-11T14:02:51.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182464 for osd.3 2023-08-11T14:02:51.429 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:51.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411395 for osd.2 2023-08-11T14:02:51.532 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:51.605 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:51 smithi003 ceph-mon[97195]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:02:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:51 smithi103 ceph-mon[130434]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:02:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:51 smithi187 ceph-mon[130973]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:02:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 -- 172.21.15.3:0/4241122653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050072d50 msgr2=0x7f8050073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 --2- 172.21.15.3:0/4241122653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050072d50 0x7f8050073130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8040009990 tx=0x7f8040009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 -- 172.21.15.3:0/4241122653 shutdown_connections 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 --2- 172.21.15.3:0/4241122653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f805010d180 0x7f805010f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 --2- 172.21.15.3:0/4241122653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050072d50 0x7f8050073130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 --2- 172.21.15.3:0/4241122653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 0x7f805010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 -- 172.21.15.3:0/4241122653 >> 172.21.15.3:0/4241122653 conn(0x7f80500fc410 msgr2=0x7f80500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 -- 172.21.15.3:0/4241122653 shutdown_connections 2023-08-11T14:02:52.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.200+0000 7f8055ad7700 1 -- 172.21.15.3:0/4241122653 wait complete. 2023-08-11T14:02:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.201+0000 7f8055ad7700 1 Processor -- start 2023-08-11T14:02:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.201+0000 7f8055ad7700 1 -- start start 2023-08-11T14:02:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 0x7f805019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 0x7f805019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:52.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 0x7f8050198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050199100 con 0x7f805010d180 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050199270 con 0x7f8050072d50 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f8055ad7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050199430 con 0x7f8050073670 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 0x7f805019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 0x7f8050198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 0x7f805019ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47534/0 (socket says 172.21.15.3:47534) 2023-08-11T14:02:52.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 0x7f805019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804f7fe700 1 -- 172.21.15.3:0/3578347312 learned_addr learned my addr 172.21.15.3:0/3578347312 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 -- 172.21.15.3:0/3578347312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 msgr2=0x7f805019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 0x7f805019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 -- 172.21.15.3:0/3578347312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 msgr2=0x7f805019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 0x7f805019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.202+0000 7f804ffff700 1 -- 172.21.15.3:0/3578347312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400095f0 con 0x7f805010d180 2023-08-11T14:02:52.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f804ffff700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 0x7f8050198a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f804800efc0 tx=0x7f804800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048018070 con 0x7f805010d180 2023-08-11T14:02:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f804800f040 con 0x7f805010d180 2023-08-11T14:02:52.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80501996b0 con 0x7f805010d180 2023-08-11T14:02:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804800cd40 con 0x7f805010d180 2023-08-11T14:02:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.203+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501a5a00 con 0x7f805010d180 2023-08-11T14:02:52.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.204+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8050108390 con 0x7f805010d180 2023-08-11T14:02:52.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.206+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f8048003a00 con 0x7f805010d180 2023-08-11T14:02:52.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.208+0000 7f804cff9700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 0x7f80380a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.208+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f80480cbba0 con 0x7f805010d180 2023-08-11T14:02:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.208+0000 7f804f7fe700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 0x7f80380a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.209+0000 7f804f7fe700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 0x7f80380a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8040005cd0 tx=0x7f8040005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.210+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8048095500 con 0x7f805010d180 2023-08-11T14:02:52.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.445+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f805004e620 con 0x7f805010d180 2023-08-11T14:02:52.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.445+0000 7f804cff9700 1 -- 172.21.15.3:0/3578347312 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f8048098b20 con 0x7f805010d180 2023-08-11T14:02:52.447 INFO:teuthology.orchestra.run.smithi003.stdout:304942678047 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 msgr2=0x7f80380a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 0x7f80380a2520 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8040005cd0 tx=0x7f8040005bc0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 msgr2=0x7f8050198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 0x7f8050198a90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f804800efc0 tx=0x7f804800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 shutdown_connections 2023-08-11T14:02:52.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f805010d180 0x7f8050198a90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f80380a0060 0x7f80380a2520 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8050072d50 0x7f805019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 --2- 172.21.15.3:0/3578347312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050073670 0x7f805019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 >> 172.21.15.3:0/3578347312 conn(0x7f80500fc410 msgr2=0x7f805010f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 shutdown_connections 2023-08-11T14:02:52.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:52.448+0000 7f8055ad7700 1 -- 172.21.15.3:0/3578347312 wait complete. 2023-08-11T14:02:52.946 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3578347312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:02:52.947 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:52 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:53.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3578347312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T14:02:53.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:02:53.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:02:53.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:52 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 -- 172.21.15.3:0/3211654511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 msgr2=0x7fbb9410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/3211654511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb9410d810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00b0a0 tx=0x7fbb8c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 -- 172.21.15.3:0/3211654511 shutdown_connections 2023-08-11T14:02:53.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/3211654511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb9410d810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/3211654511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 0x7fbb94106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/3211654511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 0x7fbb94100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 -- 172.21.15.3:0/3211654511 >> 172.21.15.3:0/3211654511 conn(0x7fbb940fc410 msgr2=0x7fbb940fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 -- 172.21.15.3:0/3211654511 shutdown_connections 2023-08-11T14:02:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.077+0000 7fbb9b04b700 1 -- 172.21.15.3:0/3211654511 wait complete. 2023-08-11T14:02:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 Processor -- start 2023-08-11T14:02:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 -- start start 2023-08-11T14:02:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 0x7fbb94198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 0x7fbb9419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9419dd00 con 0x7fbb94101300 2023-08-11T14:02:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9419de70 con 0x7fbb94106940 2023-08-11T14:02:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.078+0000 7fbb9b04b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9419e030 con 0x7fbb94100940 2023-08-11T14:02:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41900/0 (socket says 172.21.15.3:41900) 2023-08-11T14:02:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 -- 172.21.15.3:0/1594258070 learned_addr learned my addr 172.21.15.3:0/1594258070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb98de7700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 0x7fbb94198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb995e8700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 0x7fbb9419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 -- 172.21.15.3:0/1594258070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 msgr2=0x7fbb94198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 0x7fbb94198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 -- 172.21.15.3:0/1594258070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 msgr2=0x7fbb9419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 0x7fbb9419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 -- 172.21.15.3:0/1594258070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c009d00 con 0x7fbb94101300 2023-08-11T14:02:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.079+0000 7fbb93fff700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000d980 tx=0x7fbb8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.080+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004220 con 0x7fbb94101300 2023-08-11T14:02:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.080+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb80007640 con 0x7fbb94101300 2023-08-11T14:02:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.080+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9419e310 con 0x7fbb94101300 2023-08-11T14:02:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.080+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80005690 con 0x7fbb94101300 2023-08-11T14:02:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.080+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941a5b70 con 0x7fbb94101300 2023-08-11T14:02:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.081+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9404e620 con 0x7fbb94101300 2023-08-11T14:02:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.083+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fbb80004750 con 0x7fbb94101300 2023-08-11T14:02:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.086+0000 7fbb91ffb700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 0x7fbb840a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.086+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbb800cb4b0 con 0x7fbb94101300 2023-08-11T14:02:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.086+0000 7fbb98de7700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 0x7fbb840a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.087+0000 7fbb98de7700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 0x7fbb840a2520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009be0 tx=0x7fbb88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.090+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbb80094d80 con 0x7fbb94101300 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3578347312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:02:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:02:53.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:52 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:02:53.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 -- 172.21.15.3:0/137613880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 msgr2=0x7f34b810bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 --2- 172.21.15.3:0/137613880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b810bbe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009990 tx=0x7f34a8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 -- 172.21.15.3:0/137613880 shutdown_connections 2023-08-11T14:02:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 --2- 172.21.15.3:0/137613880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b810c2c0 0x7f34b8113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 --2- 172.21.15.3:0/137613880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b81071e0 0x7f34b81075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 --2- 172.21.15.3:0/137613880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b810bbe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 -- 172.21.15.3:0/137613880 >> 172.21.15.3:0/137613880 conn(0x7f34b8074f80 msgr2=0x7f34b80773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 -- 172.21.15.3:0/137613880 shutdown_connections 2023-08-11T14:02:53.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.309+0000 7f34c0496700 1 -- 172.21.15.3:0/137613880 wait complete. 2023-08-11T14:02:53.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 Processor -- start 2023-08-11T14:02:53.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 -- start start 2023-08-11T14:02:53.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 0x7f34b819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b810c2c0 0x7f34b81a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a2040 con 0x7f34b81071e0 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a21b0 con 0x7f34b810c2c0 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.311+0000 7f34c0496700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81a2370 con 0x7f34b8107b90 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34be232700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 0x7f34b819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34be232700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 0x7f34b819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41918/0 (socket says 172.21.15.3:41918) 2023-08-11T14:02:53.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34be232700 1 -- 172.21.15.3:0/3392713681 learned_addr learned my addr 172.21.15.3:0/3392713681 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bea33700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b810c2c0 0x7f34b81a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 -- 172.21.15.3:0/3392713681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b810c2c0 msgr2=0x7f34b81a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b810c2c0 0x7f34b81a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 -- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 msgr2=0x7f34b819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 0x7f34b819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.312+0000 7f34bda31700 1 -- 172.21.15.3:0/3392713681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80095f0 con 0x7f34b8107b90 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.313+0000 7f34bda31700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b819d5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.313+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a801d070 con 0x7f34b8107b90 2023-08-11T14:02:53.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.314+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b81a25f0 con 0x7f34b8107b90 2023-08-11T14:02:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.314+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b81a9eb0 con 0x7f34b8107b90 2023-08-11T14:02:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.314+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a8021410 con 0x7f34b8107b90 2023-08-11T14:02:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.314+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8017650 con 0x7f34b8107b90 2023-08-11T14:02:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.315+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b804e620 con 0x7f34b8107b90 2023-08-11T14:02:53.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.317+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f34a8021580 con 0x7f34b8107b90 2023-08-11T14:02:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.319+0000 7f34af7fe700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 0x7f34a40a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.319+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f34a80cc9f0 con 0x7f34b8107b90 2023-08-11T14:02:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.319+0000 7f34be232700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 0x7f34a40a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.319+0000 7f34be232700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 0x7f34a40a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d20 tx=0x7f34b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.323+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f34a8096350 con 0x7f34b8107b90 2023-08-11T14:02:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.329+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb94110a60 con 0x7fbb94101300 2023-08-11T14:02:53.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.329+0000 7fbb91ffb700 1 -- 172.21.15.3:0/1594258070 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb800983a0 con 0x7fbb94101300 2023-08-11T14:02:53.331 INFO:teuthology.orchestra.run.smithi003.stdout:55834574921 2023-08-11T14:02:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.331+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 msgr2=0x7fbb840a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 0x7fbb840a2520 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009be0 tx=0x7fbb88009260 comp rx=0 tx=0).stop 2023-08-11T14:02:53.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 msgr2=0x7fbb94199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000d980 tx=0x7fbb8000dc90 comp rx=0 tx=0).stop 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 shutdown_connections 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb840a0060 0x7fbb840a2520 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb94106940 0x7fbb9419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb94100940 0x7fbb94198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 --2- 172.21.15.3:0/1594258070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb94101300 0x7fbb94199280 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 >> 172.21.15.3:0/1594258070 conn(0x7fbb940fc410 msgr2=0x7fbb9410bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:53.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 shutdown_connections 2023-08-11T14:02:53.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.332+0000 7fbb9b04b700 1 -- 172.21.15.3:0/1594258070 wait complete. 2023-08-11T14:02:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.561+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f34b819e3e0 con 0x7f34b8107b90 2023-08-11T14:02:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.562+0000 7f34af7fe700 1 -- 172.21.15.3:0/3392713681 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34a8026090 con 0x7f34b8107b90 2023-08-11T14:02:53.563 INFO:teuthology.orchestra.run.smithi003.stdout:34359738447 2023-08-11T14:02:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.564+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 msgr2=0x7f34a40a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.564+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 0x7f34a40a2520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009d20 tx=0x7f34b4009260 comp rx=0 tx=0).stop 2023-08-11T14:02:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.564+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 msgr2=0x7f34b819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.564+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b819d5c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34a8000c00 tx=0x7f34a8005740 comp rx=0 tx=0).stop 2023-08-11T14:02:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 shutdown_connections 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f34a40a0060 0x7f34a40a2520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34b810c2c0 0x7f34b81a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34b81071e0 0x7f34b819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 --2- 172.21.15.3:0/3392713681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34b8107b90 0x7f34b819d5c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 >> 172.21.15.3:0/3392713681 conn(0x7f34b8074f80 msgr2=0x7f34b8075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 shutdown_connections 2023-08-11T14:02:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.565+0000 7f34c0496700 1 -- 172.21.15.3:0/3392713681 wait complete. 2023-08-11T14:02:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 -- 172.21.15.3:0/2001945185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3040105390 msgr2=0x7f3040105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2001945185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3040105390 0x7f3040105810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f30340099f0 tx=0x7f3034009d00 comp rx=0 tx=0).stop 2023-08-11T14:02:53.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 -- 172.21.15.3:0/2001945185 shutdown_connections 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2001945185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105d50 0x7f304010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2001945185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30400688e0 0x7f3040068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2001945185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3040105390 0x7f3040105810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.813+0000 7f3045eb8700 1 -- 172.21.15.3:0/2001945185 >> 172.21.15.3:0/2001945185 conn(0x7f3040076920 msgr2=0x7f3040076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.814+0000 7f3045eb8700 1 -- 172.21.15.3:0/2001945185 shutdown_connections 2023-08-11T14:02:53.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.814+0000 7f3045eb8700 1 -- 172.21.15.3:0/2001945185 wait complete. 2023-08-11T14:02:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.815+0000 7f3045eb8700 1 Processor -- start 2023-08-11T14:02:53.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.815+0000 7f3045eb8700 1 -- start start 2023-08-11T14:02:53.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105390 0x7f30400ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 0x7f3040100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a4ea0 con 0x7f3040105390 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a5010 con 0x7f30400688e0 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f3045eb8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a51d0 con 0x7f3040105d50 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f303f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f303f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47580/0 (socket says 172.21.15.3:47580) 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f303f7fe700 1 -- 172.21.15.3:0/2565471098 learned_addr learned my addr 172.21.15.3:0/2565471098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f303effd700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105390 0x7f30400ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.816+0000 7f303ffff700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 0x7f3040100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 -- 172.21.15.3:0/2565471098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 msgr2=0x7f3040100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 0x7f3040100390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 -- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105390 msgr2=0x7f30400ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105390 0x7f30400ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:53.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 -- 172.21.15.3:0/2565471098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009c40 con 0x7f30400688e0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303ffff700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 0x7f3040100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303f7fe700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9a0 tx=0x7f303000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030009790 con 0x7f30400688e0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303000cbd0 con 0x7f30400688e0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30340095f0 con 0x7f30400688e0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030018680 con 0x7f30400688e0 2023-08-11T14:02:53.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.817+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401a5820 con 0x7f30400688e0 2023-08-11T14:02:53.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.818+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304004e620 con 0x7f30400688e0 2023-08-11T14:02:53.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.821+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f303000cd40 con 0x7f30400688e0 2023-08-11T14:02:53.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.822+0000 7f303cff9700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 0x7f30280a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:53.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.822+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3030014070 con 0x7f30400688e0 2023-08-11T14:02:53.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.822+0000 7f303effd700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 0x7f30280a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:53.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.823+0000 7f303effd700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 0x7f30280a24a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3034006010 tx=0x7f303401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:53.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:53.824+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f30300954e0 con 0x7f30400688e0 2023-08-11T14:02:53.922 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1594258070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T14:02:53.922 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3392713681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T14:02:53.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678044 got 304942678047 for osd.10 2023-08-11T14:02:53.998 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.059+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3040110a20 con 0x7f30400688e0 2023-08-11T14:02:54.062 INFO:teuthology.orchestra.run.smithi003.stdout:137438953532 2023-08-11T14:02:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.060+0000 7f303cff9700 1 -- 172.21.15.3:0/2565471098 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3030098b00 con 0x7f30400688e0 2023-08-11T14:02:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 msgr2=0x7f30280a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 0x7f30280a24a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3034006010 tx=0x7f303401a040 comp rx=0 tx=0).stop 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 msgr2=0x7f30400ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9a0 tx=0x7f303000ed60 comp rx=0 tx=0).stop 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 shutdown_connections 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3040105d50 0x7f3040100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f302809ffe0 0x7f30280a24a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30400688e0 0x7f30400ff8d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 --2- 172.21.15.3:0/2565471098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3040105390 0x7f30400ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 >> 172.21.15.3:0/2565471098 conn(0x7f3040076920 msgr2=0x7f30400fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 shutdown_connections 2023-08-11T14:02:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.062+0000 7f3045eb8700 1 -- 172.21.15.3:0/2565471098 wait complete. 2023-08-11T14:02:54.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1594258070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T14:02:54.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3392713681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T14:02:54.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1594258070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-08-11T14:02:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3392713681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-08-11T14:02:54.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 -- 172.21.15.3:0/3824584587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8106940 msgr2=0x7f4ab8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 --2- 172.21.15.3:0/3824584587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8106940 0x7f4ab8106d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009990 tx=0x7f4aa8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 -- 172.21.15.3:0/3824584587 shutdown_connections 2023-08-11T14:02:54.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 --2- 172.21.15.3:0/3824584587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8101300 0x7f4ab810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 --2- 172.21.15.3:0/3824584587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8106940 0x7f4ab8106d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 --2- 172.21.15.3:0/3824584587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8100940 0x7f4ab8100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.397+0000 7f4abe264700 1 -- 172.21.15.3:0/3824584587 >> 172.21.15.3:0/3824584587 conn(0x7f4ab80fc410 msgr2=0x7f4ab80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.398+0000 7f4abe264700 1 -- 172.21.15.3:0/3824584587 shutdown_connections 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.398+0000 7f4abe264700 1 -- 172.21.15.3:0/3824584587 wait complete. 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.398+0000 7f4abe264700 1 Processor -- start 2023-08-11T14:02:54.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 -- start start 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8100940 0x7f4ab8072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 0x7f4ab8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 0x7f4ab80736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab81a9420 con 0x7f4ab8101300 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab81a9560 con 0x7f4ab8106940 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abe264700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab81a96a0 con 0x7f4ab8100940 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4ab7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 0x7f4ab8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:54.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4abc801700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 0x7f4ab80736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4ab7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 0x7f4ab8073160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41958/0 (socket says 172.21.15.3:41958) 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4ab7fff700 1 -- 172.21.15.3:0/1692675084 learned_addr learned my addr 172.21.15.3:0/1692675084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.399+0000 7f4aaffff700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8100940 0x7f4ab8072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 -- 172.21.15.3:0/1692675084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8100940 msgr2=0x7f4ab8072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8100940 0x7f4ab8072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 -- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 msgr2=0x7f4ab8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 0x7f4ab8073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 -- 172.21.15.3:0/1692675084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80095f0 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abc801700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 0x7f4ab80736a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000efc0 tx=0x7f4ab000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0018070 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab81a9920 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ab000f040 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000cd60 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.400+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab81a9dd0 con 0x7f4ab8106940 2023-08-11T14:02:54.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.401+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab804e620 con 0x7f4ab8106940 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.404+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f4ab0003a00 con 0x7f4ab8106940 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.405+0000 7f4ab5ffb700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 0x7f4aa40a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.405+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4ab00cbb00 con 0x7f4ab8106940 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.405+0000 7f4aaffff700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 0x7f4aa40a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.406+0000 7f4aaffff700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 0x7f4aa40a24a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b2e0 tx=0x7f4aa8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.407+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4ab0095460 con 0x7f4ab8106940 2023-08-11T14:02:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.621+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4ab8074150 con 0x7f4ab8106940 2023-08-11T14:02:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.621+0000 7f4ab5ffb700 1 -- 172.21.15.3:0/1692675084 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ab0098a80 con 0x7f4ab8106940 2023-08-11T14:02:54.623 INFO:teuthology.orchestra.run.smithi003.stdout:283467841573 2023-08-11T14:02:54.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 msgr2=0x7f4aa40a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 0x7f4aa40a24a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b2e0 tx=0x7f4aa8005fd0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 msgr2=0x7f4ab80736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 0x7f4ab80736a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000efc0 tx=0x7f4ab000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 shutdown_connections 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4aa409ffe0 0x7f4aa40a24a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab8106940 0x7f4ab80736a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ab8100940 0x7f4ab8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 --2- 172.21.15.3:0/1692675084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ab8101300 0x7f4ab8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 >> 172.21.15.3:0/1692675084 conn(0x7f4ab80fc410 msgr2=0x7f4ab80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 shutdown_connections 2023-08-11T14:02:54.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:54.624+0000 7f4abe264700 1 -- 172.21.15.3:0/1692675084 wait complete. 2023-08-11T14:02:54.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574921 for osd.1 2023-08-11T14:02:54.754 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:54.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738447 for osd.0 2023-08-11T14:02:54.810 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:54.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2565471098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T14:02:54.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:54 smithi003 ceph-mon[97195]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2023-08-11T14:02:54.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1692675084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T14:02:55.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841571 got 283467841573 for osd.9 2023-08-11T14:02:55.139 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:55.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2565471098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T14:02:55.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:54 smithi103 ceph-mon[130434]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2023-08-11T14:02:55.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1692675084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T14:02:55.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2565471098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-08-11T14:02:55.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:54 smithi187 ceph-mon[130973]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2023-08-11T14:02:55.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1692675084' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-08-11T14:02:55.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953528 got 137438953532 for osd.4 2023-08-11T14:02:55.642 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:55.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 -- 172.21.15.3:0/21372316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 msgr2=0x7f3f68106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:55.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 --2- 172.21.15.3:0/21372316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f68106d20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3f640098f0 tx=0x7f3f64009c00 comp rx=0 tx=0).stop 2023-08-11T14:02:55.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 -- 172.21.15.3:0/21372316 shutdown_connections 2023-08-11T14:02:55.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 --2- 172.21.15.3:0/21372316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f6810d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:55.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 --2- 172.21.15.3:0/21372316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f68106d20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:55.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 --2- 172.21.15.3:0/21372316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:55.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 -- 172.21.15.3:0/21372316 >> 172.21.15.3:0/21372316 conn(0x7f3f680fc410 msgr2=0x7f3f680fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:55.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 -- 172.21.15.3:0/21372316 shutdown_connections 2023-08-11T14:02:55.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.826+0000 7f3f70478700 1 -- 172.21.15.3:0/21372316 wait complete. 2023-08-11T14:02:55.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.827+0000 7f3f70478700 1 Processor -- start 2023-08-11T14:02:55.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.827+0000 7f3f70478700 1 -- start start 2023-08-11T14:02:55.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f68199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f6819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819cbc0 con 0x7f3f68101300 2023-08-11T14:02:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819cd30 con 0x7f3f68106940 2023-08-11T14:02:55.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f70478700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819cef0 con 0x7f3f68100940 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f6e214700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f6e214700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44850/0 (socket says 172.21.15.3:44850) 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f6e214700 1 -- 172.21.15.3:0/442431925 learned_addr learned my addr 172.21.15.3:0/442431925 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f6ea15700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f6819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.828+0000 7f3f6da13700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f68199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:55.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 -- 172.21.15.3:0/442431925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 msgr2=0x7f3f6819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:55.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f6819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:55.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 -- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 msgr2=0x7f3f68199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:55.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f68199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:55.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 -- 172.21.15.3:0/442431925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64009520 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6da13700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f68199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f6e214700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009810 tx=0x7f3f6400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6401c070 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6819d170 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6400bbd0 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f640049c0 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.829+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f681a5b20 con 0x7f3f68100940 2023-08-11T14:02:55.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.830+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6804e620 con 0x7f3f68100940 2023-08-11T14:02:55.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.833+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f3f64020070 con 0x7f3f68100940 2023-08-11T14:02:55.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.834+0000 7f3f5f7fe700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 0x7f3f540a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:55.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.834+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3f640cc850 con 0x7f3f68100940 2023-08-11T14:02:55.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.834+0000 7f3f6da13700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 0x7f3f540a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:55.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.835+0000 7f3f6da13700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 0x7f3f540a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f6819a2a0 tx=0x7f3f58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:55.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:55.836+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3f640961b0 con 0x7f3f68100940 2023-08-11T14:02:56.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.057+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3f68110920 con 0x7f3f68100940 2023-08-11T14:02:56.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.057+0000 7f3f5f7fe700 1 -- 172.21.15.3:0/442431925 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f640997d0 con 0x7f3f68100940 2023-08-11T14:02:56.059 INFO:teuthology.orchestra.run.smithi003.stdout:214748364845 2023-08-11T14:02:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 msgr2=0x7f3f540a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 0x7f3f540a24a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f6819a2a0 tx=0x7f3f58009260 comp rx=0 tx=0).stop 2023-08-11T14:02:56.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 msgr2=0x7f3f68198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009810 tx=0x7f3f6400fa00 comp rx=0 tx=0).stop 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 shutdown_connections 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f5409ffe0 0x7f3f540a24a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68106940 0x7f3f6819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f68100940 0x7f3f68198cc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 --2- 172.21.15.3:0/442431925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f68101300 0x7f3f68199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 >> 172.21.15.3:0/442431925 conn(0x7f3f680fc410 msgr2=0x7f3f6810be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 shutdown_connections 2023-08-11T14:02:56.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.060+0000 7f3f70478700 1 -- 172.21.15.3:0/442431925 wait complete. 2023-08-11T14:02:56.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 -- 172.21.15.3:0/207058308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 msgr2=0x7fba8410f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 --2- 172.21.15.3:0/207058308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8410f5f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b0a0 tx=0x7fba7c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 -- 172.21.15.3:0/207058308 shutdown_connections 2023-08-11T14:02:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 --2- 172.21.15.3:0/207058308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8410f5f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 --2- 172.21.15.3:0/207058308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84072d50 0x7fba84073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 --2- 172.21.15.3:0/207058308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84073670 0x7fba8410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.162+0000 7fba8b33f700 1 -- 172.21.15.3:0/207058308 >> 172.21.15.3:0/207058308 conn(0x7fba840fc410 msgr2=0x7fba840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.163+0000 7fba8b33f700 1 -- 172.21.15.3:0/207058308 shutdown_connections 2023-08-11T14:02:56.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.163+0000 7fba8b33f700 1 -- 172.21.15.3:0/207058308 wait complete. 2023-08-11T14:02:56.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 Processor -- start 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 -- start start 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419dd50 con 0x7fba8410d180 2023-08-11T14:02:56.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419dec0 con 0x7fba84073670 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.164+0000 7fba8b33f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8419e080 con 0x7fba84072d50 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba898dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba888da700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44864/0 (socket says 172.21.15.3:44864) 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba898dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41992/0 (socket says 172.21.15.3:41992) 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 -- 172.21.15.3:0/4237941308 learned_addr learned my addr 172.21.15.3:0/4237941308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:56.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba888da700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47620/0 (socket says 172.21.15.3:47620) 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 -- 172.21.15.3:0/4237941308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 msgr2=0x7fba841992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 -- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 msgr2=0x7fba8419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 -- 172.21.15.3:0/4237941308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009d00 con 0x7fba84072d50 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba888da700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.165+0000 7fba890db700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba8000e9a0 tx=0x7fba8000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:56.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.166+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80009790 con 0x7fba84072d50 2023-08-11T14:02:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.166+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8419e360 con 0x7fba84072d50 2023-08-11T14:02:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.166+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba80010430 con 0x7fba84072d50 2023-08-11T14:02:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.166+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80018680 con 0x7fba84072d50 2023-08-11T14:02:56.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.166+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba841a5bc0 con 0x7fba84072d50 2023-08-11T14:02:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.167+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8404e620 con 0x7fba84072d50 2023-08-11T14:02:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.169+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7fba800105a0 con 0x7fba84072d50 2023-08-11T14:02:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.170+0000 7fba7a7fc700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 0x7fba700a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.171+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fba80014070 con 0x7fba84072d50 2023-08-11T14:02:56.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.171+0000 7fba888da700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 0x7fba700a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:56.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.171+0000 7fba888da700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 0x7fba700a24a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fba8419a3b0 tx=0x7fba74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:56.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.175+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fba80051c30 con 0x7fba84072d50 2023-08-11T14:02:56.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:56 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/442431925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T14:02:56.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.398+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fba8419a0f0 con 0x7fba84072d50 2023-08-11T14:02:56.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.399+0000 7fba7a7fc700 1 -- 172.21.15.3:0/4237941308 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fba800951c0 con 0x7fba84072d50 2023-08-11T14:02:56.400 INFO:teuthology.orchestra.run.smithi003.stdout:193273528372 2023-08-11T14:02:56.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 msgr2=0x7fba700a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 0x7fba700a24a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fba8419a3b0 tx=0x7fba74009260 comp rx=0 tx=0).stop 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 msgr2=0x7fba84198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba8000e9a0 tx=0x7fba8000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 shutdown_connections 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba8410d180 0x7fba8419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba84072d50 0x7fba84198d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba7009ffe0 0x7fba700a24a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 --2- 172.21.15.3:0/4237941308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba84073670 0x7fba841992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 >> 172.21.15.3:0/4237941308 conn(0x7fba840fc410 msgr2=0x7fba8410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 shutdown_connections 2023-08-11T14:02:56.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:56.401+0000 7fba8b33f700 1 -- 172.21.15.3:0/4237941308 wait complete. 2023-08-11T14:02:56.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:56 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/442431925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T14:02:56.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:56 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/442431925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-08-11T14:02:56.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364843 got 214748364845 for osd.7 2023-08-11T14:02:56.683 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:56.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528368 got 193273528372 for osd.6 2023-08-11T14:02:56.988 DEBUG:teuthology.parallel:result is None 2023-08-11T14:02:56.988 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-08-11T14:02:56.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph pg dump --format=json 2023-08-11T14:02:57.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:57 smithi003 ceph-mon[97195]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:57.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4237941308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T14:02:57.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:57 smithi103 ceph-mon[130434]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:57.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4237941308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T14:02:57.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:57 smithi187 ceph-mon[130973]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:57.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4237941308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-08-11T14:02:57.730 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:02:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 -- 172.21.15.3:0/2897558134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 msgr2=0x7f4a34100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 --2- 172.21.15.3:0/2897558134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34100dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009990 tx=0x7f4a28009ca0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 -- 172.21.15.3:0/2897558134 shutdown_connections 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 --2- 172.21.15.3:0/2897558134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34101300 0x7f4a3410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 --2- 172.21.15.3:0/2897558134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34106940 0x7f4a34106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 --2- 172.21.15.3:0/2897558134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34100dc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 -- 172.21.15.3:0/2897558134 >> 172.21.15.3:0/2897558134 conn(0x7f4a340fc410 msgr2=0x7f4a340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 -- 172.21.15.3:0/2897558134 shutdown_connections 2023-08-11T14:02:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.299+0000 7f4a39769700 1 -- 172.21.15.3:0/2897558134 wait complete. 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.300+0000 7f4a39769700 1 Processor -- start 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 -- start start 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34101300 0x7f4a34073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34106940 0x7f4a340736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341a4eb0 con 0x7f4a34100940 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341a5020 con 0x7f4a34106940 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a39769700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a341a51e0 con 0x7f4a34101300 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a32ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a32ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42014/0 (socket says 172.21.15.3:42014) 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a32ffd700 1 -- 172.21.15.3:0/4050749720 learned_addr learned my addr 172.21.15.3:0/4050749720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a337fe700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34106940 0x7f4a340736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.301+0000 7f4a327fc700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34101300 0x7f4a34073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 -- 172.21.15.3:0/4050749720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34101300 msgr2=0x7f4a34073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34101300 0x7f4a34073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 -- 172.21.15.3:0/4050749720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34106940 msgr2=0x7f4a340736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34106940 0x7f4a340736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 -- 172.21.15.3:0/4050749720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a280095f0 con 0x7f4a34100940 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a32ffd700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400e9a0 tx=0x7f4a2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24009790 con 0x7f4a34100940 2023-08-11T14:02:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a24010430 con 0x7f4a34100940 2023-08-11T14:02:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a341a54c0 con 0x7f4a34100940 2023-08-11T14:02:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.302+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24018680 con 0x7f4a34100940 2023-08-11T14:02:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.303+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a341a59c0 con 0x7f4a34100940 2023-08-11T14:02:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.303+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3404e620 con 0x7f4a34100940 2023-08-11T14:02:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.306+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) v1 ==== 134721+0+0 (secure 0 0 0) 0x7f4a240105a0 con 0x7f4a34100940 2023-08-11T14:02:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.307+0000 7f4a1bfff700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 0x7f4a1c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:02:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.307+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4a24014070 con 0x7f4a34100940 2023-08-11T14:02:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.307+0000 7f4a327fc700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 0x7f4a1c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:02:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.308+0000 7f4a327fc700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 0x7f4a1c0a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4a34074410 tx=0x7f4a28005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:02:59.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.309+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4a24095370 con 0x7f4a34100940 2023-08-11T14:02:59.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.517+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a34110b40 con 0x7f4a1c09ffe0 2023-08-11T14:02:59.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.531+0000 7f4a1bfff700 1 -- 172.21.15.3:0/4050749720 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426543 (secure 0 0 0) 0x7f4a34110b40 con 0x7f4a1c09ffe0 2023-08-11T14:02:59.540 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:02:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 msgr2=0x7f4a1c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 0x7f4a1c0a24a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4a34074410 tx=0x7f4a28005fb0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 msgr2=0x7f4a34072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:02:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400e9a0 tx=0x7f4a2400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 shutdown_connections 2023-08-11T14:02:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a1c09ffe0 0x7f4a1c0a24a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a34106940 0x7f4a340736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a34100940 0x7f4a34072c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 --2- 172.21.15.3:0/4050749720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a34101300 0x7f4a34073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:02:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 >> 172.21.15.3:0/4050749720 conn(0x7f4a340fc410 msgr2=0x7f4a340fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:02:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 shutdown_connections 2023-08-11T14:02:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:02:59.538+0000 7f4a39769700 1 -- 172.21.15.3:0/4050749720 wait complete. 2023-08-11T14:02:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:02:59.564 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:02:59 smithi003 ceph-mon[97195]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:02:59 smithi103 ceph-mon[130434]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:02:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:02:59 smithi187 ceph-mon[130973]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:00.026 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-08-11T14:02:58.185203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657867,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":698,"num_read_kb":465,"num_write":421,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338164,"kb_used_data":13932,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603660,"statfs":{"total":1151940427776,"available":1151594147840,"internally_reserved":0,"allocated":14266368,"data_stored":7822493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003399"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.339952+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:02:03.339952+0000","last_peered":"2023-08-11T14:02:03.339952+0000","last_clean":"2023-08-11T14:02:03.339952+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:02:03.339952+0000","last_undegraded":"2023-08-11T14:02:03.339952+0000","last_fullsized":"2023-08-11T14:02:03.339952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:07:02.851636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514145+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:03.514145+0000","last_peered":"2023-08-11T14:02:03.514145+0000","last_clean":"2023-08-11T14:02:03.514145+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:03.514145+0000","last_undegraded":"2023-08-11T14:02:03.514145+0000","last_fullsized":"2023-08-11T14:02:03.514145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:30.192305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514351+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:03.514351+0000","last_peered":"2023-08-11T14:02:03.514351+0000","last_clean":"2023-08-11T14:02:03.514351+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:03.514351+0000","last_undegraded":"2023-08-11T14:02:03.514351+0000","last_fullsized":"2023-08-11T14:02:03.514351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:53:22.056513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330963+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:03.330963+0000","last_peered":"2023-08-11T14:02:03.330963+0000","last_clean":"2023-08-11T14:02:03.330963+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:03.330963+0000","last_undegraded":"2023-08-11T14:02:03.330963+0000","last_fullsized":"2023-08-11T14:02:03.330963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:51.334309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:51.405796+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:51.405796+0000","last_peered":"2023-08-11T14:02:51.405796+0000","last_clean":"2023-08-11T14:02:51.405796+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:51.405796+0000","last_undegraded":"2023-08-11T14:02:51.405796+0000","last_fullsized":"2023-08-11T14:02:51.405796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:49.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513358+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:03.513358+0000","last_peered":"2023-08-11T14:02:03.513358+0000","last_clean":"2023-08-11T14:02:03.513358+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:03.513358+0000","last_undegraded":"2023-08-11T14:02:03.513358+0000","last_fullsized":"2023-08-11T14:02:03.513358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:08.181790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:55.402711+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:55.402711+0000","last_peered":"2023-08-11T14:02:55.402711+0000","last_clean":"2023-08-11T14:02:55.402711+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:55.402711+0000","last_undegraded":"2023-08-11T14:02:55.402711+0000","last_fullsized":"2023-08-11T14:02:55.402711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330576+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:03.330576+0000","last_peered":"2023-08-11T14:02:03.330576+0000","last_clean":"2023-08-11T14:02:03.330576+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:03.330576+0000","last_undegraded":"2023-08-11T14:02:03.330576+0000","last_fullsized":"2023-08-11T14:02:03.330576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:21.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":134,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:53.399237+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:53.399237+0000","last_peered":"2023-08-11T14:02:53.399237+0000","last_clean":"2023-08-11T14:02:53.399237+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:53.399237+0000","last_undegraded":"2023-08-11T14:02:53.399237+0000","last_fullsized":"2023-08-11T14:02:53.399237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514217+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:03.514217+0000","last_peered":"2023-08-11T14:02:03.514217+0000","last_clean":"2023-08-11T14:02:03.514217+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:03.514217+0000","last_undegraded":"2023-08-11T14:02:03.514217+0000","last_fullsized":"2023-08-11T14:02:03.514217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:49:01.488839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330205+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:03.330205+0000","last_peered":"2023-08-11T14:02:03.330205+0000","last_clean":"2023-08-11T14:02:03.330205+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:03.330205+0000","last_undegraded":"2023-08-11T14:02:03.330205+0000","last_fullsized":"2023-08-11T14:02:03.330205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:32.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513781+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:03.513781+0000","last_peered":"2023-08-11T14:02:03.513781+0000","last_clean":"2023-08-11T14:02:03.513781+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:03.513781+0000","last_undegraded":"2023-08-11T14:02:03.513781+0000","last_fullsized":"2023-08-11T14:02:03.513781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:08.653787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330092+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:03.330092+0000","last_peered":"2023-08-11T14:02:03.330092+0000","last_clean":"2023-08-11T14:02:03.330092+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:03.330092+0000","last_undegraded":"2023-08-11T14:02:03.330092+0000","last_fullsized":"2023-08-11T14:02:03.330092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:16.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:53.411676+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:53.411676+0000","last_peered":"2023-08-11T14:02:53.411676+0000","last_clean":"2023-08-11T14:02:53.411676+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:53.411676+0000","last_undegraded":"2023-08-11T14:02:53.411676+0000","last_fullsized":"2023-08-11T14:02:53.411676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":117,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":88,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:55.414049+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:02:55.414049+0000","last_peered":"2023-08-11T14:02:55.414049+0000","last_clean":"2023-08-11T14:02:55.414049+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:02:55.414049+0000","last_undegraded":"2023-08-11T14:02:55.414049+0000","last_fullsized":"2023-08-11T14:02:55.414049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515481+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:02:03.515481+0000","last_peered":"2023-08-11T14:02:03.515481+0000","last_clean":"2023-08-11T14:02:03.515481+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:02:03.515481+0000","last_undegraded":"2023-08-11T14:02:03.515481+0000","last_fullsized":"2023-08-11T14:02:03.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:07:44.577236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330437+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:03.330437+0000","last_peered":"2023-08-11T14:02:03.330437+0000","last_clean":"2023-08-11T14:02:03.330437+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:03.330437+0000","last_undegraded":"2023-08-11T14:02:03.330437+0000","last_fullsized":"2023-08-11T14:02:03.330437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:52.907857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":86,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:53.409773+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:53.409773+0000","last_peered":"2023-08-11T14:02:53.409773+0000","last_clean":"2023-08-11T14:02:53.409773+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:53.409773+0000","last_undegraded":"2023-08-11T14:02:53.409773+0000","last_fullsized":"2023-08-11T14:02:53.409773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":85,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:52.408645+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:52.408645+0000","last_peered":"2023-08-11T14:02:52.408645+0000","last_clean":"2023-08-11T14:02:52.408645+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:52.408645+0000","last_undegraded":"2023-08-11T14:02:52.408645+0000","last_fullsized":"2023-08-11T14:02:52.408645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:07.097233+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:07.097233+0000","last_peered":"2023-08-11T14:02:07.097233+0000","last_clean":"2023-08-11T14:02:07.097233+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:07.097233+0000","last_undegraded":"2023-08-11T14:02:07.097233+0000","last_fullsized":"2023-08-11T14:02:07.097233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:08:56.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514463+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:03.514463+0000","last_peered":"2023-08-11T14:02:03.514463+0000","last_clean":"2023-08-11T14:02:03.514463+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:03.514463+0000","last_undegraded":"2023-08-11T14:02:03.514463+0000","last_fullsized":"2023-08-11T14:02:03.514463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:19.003199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330601+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:03.330601+0000","last_peered":"2023-08-11T14:02:03.330601+0000","last_clean":"2023-08-11T14:02:03.330601+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:03.330601+0000","last_undegraded":"2023-08-11T14:02:03.330601+0000","last_fullsized":"2023-08-11T14:02:03.330601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:32:17.015669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513303+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:03.513303+0000","last_peered":"2023-08-11T14:02:03.513303+0000","last_clean":"2023-08-11T14:02:03.513303+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:03.513303+0000","last_undegraded":"2023-08-11T14:02:03.513303+0000","last_fullsized":"2023-08-11T14:02:03.513303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:16.532299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":85,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:53.415288+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:53.415288+0000","last_peered":"2023-08-11T14:02:53.415288+0000","last_clean":"2023-08-11T14:02:53.415288+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:53.415288+0000","last_undegraded":"2023-08-11T14:02:53.415288+0000","last_fullsized":"2023-08-11T14:02:53.415288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514220+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:03.514220+0000","last_peered":"2023-08-11T14:02:03.514220+0000","last_clean":"2023-08-11T14:02:03.514220+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:03.514220+0000","last_undegraded":"2023-08-11T14:02:03.514220+0000","last_fullsized":"2023-08-11T14:02:03.514220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:06.901263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67253,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":572,"num_read_kb":356,"num_write":238,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449117,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.56999999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.88,"15min":0.88},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.296,"15min":1.296},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":1.208,"15min":1.208},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.048,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.876,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.72099999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0109999999999999,"5min":1.27,"15min":1.27},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0720000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97399999999999998,"5min":1.139,"15min":1.139},"last":0.755}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1137719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.194,"5min":1.526,"15min":1.526},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.307,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0309999999999999,"5min":1.516,"15min":1.516},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.147,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80100000000000005,"5min":1.446,"15min":1.446},"last":0.433},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81299999999999994,"5min":1.571,"15min":1.571},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88800000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.504},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":1.133,"15min":1.133},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.91900000000000004,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.01,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.091,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2,"5min":1.605,"15min":1.605},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.62,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.873,"15min":0.89300000000000002},"min":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1100000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.71299999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1719999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.54600000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99299999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.621}]},{"osd":11,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.59899999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76000000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841574,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":494098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.498},{"interface":"front","average":{"1min":0.495,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1,"15min":1},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97599999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.73099999999999998,"15min":0.75700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93799999999999994,"5min":1.123,"15min":1.123},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.72899999999999998,"15min":0.755},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.878,"15min":0.90700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.075,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":3.2149999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.93400000000000005,"15min":0.96499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0289999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.93100000000000005,"15min":0.96399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1,"15min":1.038},"min":{"1min":0.54600000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1679999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":1.006,"15min":1.042},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1240000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1759999999999999,"5min":1.484,"15min":1.484},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.103,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.51,"15min":1.51},"last":3.2440000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.99199999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1479999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.84999999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.97799999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103212,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.504,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":1.03},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47099999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59299999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.41399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.45400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.51000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.61299999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":1.022,"15min":1.022},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.56399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.58299999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.53700000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.86399999999999999,"5min":1.101,"15min":1.101},"last":0.67900000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.76700000000000002,"15min":0.75},"min":{"1min":0.74099999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.089,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.69399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0349999999999999,"5min":1.042,"15min":1.042},"last":0.755}]},{"osd":7,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.70099999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.083,"15min":1.083},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.66200000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96999999999999997,"5min":1.29,"15min":1.29},"last":0.83699999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.91900000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.95999999999999996}]},{"osd":11,"last update":"Fri Aug 11 14:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":1.0029999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.752},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71699999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":1.034,"15min":1.034},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71999999999999997,"15min":0.69799999999999995},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":6.798}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.92400000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":0.95799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92000000000000004,"5min":1.143,"15min":1.143},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92800000000000005,"5min":1.173,"15min":1.173},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":1.095,"15min":1.095},"last":0.88},{"interface":"front","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.57199999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.91400000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.94099999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.876},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.60699999999999998}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97399999999999998,"5min":1.032,"15min":1.032},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86799999999999999,"15min":0.86599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.90100000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.83999999999999997,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":1.663,"15min":1.663},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81899999999999995,"15min":0.83199999999999996},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.151,"5min":1.252,"15min":1.252},"last":0.70199999999999996}]},{"osd":9,"last update":"Fri Aug 11 14:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2130000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55700000000000005}]},{"osd":10,"last update":"Fri Aug 11 14:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84699999999999998,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92500000000000004,"15min":0.92700000000000005},"min":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.212,"5min":1.363,"15min":1.363},"last":0.94299999999999995}]},{"osd":11,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.751},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.96099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":546704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":1.054,"15min":1.054},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90800000000000003,"5min":1.234,"15min":1.234},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.46300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.48799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:01:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85099999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63500000000000001,"15min":0.59299999999999997},"min":{"1min":0.58199999999999996,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.89300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.44900000000000001,"15min":0.40500000000000003},"min":{"1min":0.35399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.61199999999999999,"15min":0.56100000000000005},"min":{"1min":0.58199999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.64900000000000002,"15min":0.58499999999999996},"min":{"1min":0.57199999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.58199999999999996,"15min":0.53900000000000003},"min":{"1min":0.504,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.623,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.177,"5min":1.345,"15min":1.345},"last":0.754}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0940000000000001,"5min":1.302,"15min":1.302},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.73799999999999999,"15min":0.70199999999999996},"min":{"1min":0.53200000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.90900000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.90200000000000002},"min":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95599999999999996,"5min":3.194,"15min":3.194},"last":0.87},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2370000000000001,"5min":1.393,"15min":1.393},"last":0.97199999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.84699999999999998},"min":{"1min":0.66500000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2250000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.92800000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.57999999999999996},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.434,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Aug 11 14:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.052,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.71899999999999997,"15min":0.63700000000000001},"min":{"1min":0.46700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.2910000000000004,"5min":6.2910000000000004,"15min":6.2910000000000004},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.3279999999999998,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.627},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95899999999999996,"5min":1.097,"15min":1.097},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63100000000000001,"15min":0.58699999999999997},"min":{"1min":0.501,"5min":0.317,"15min":0.317},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.745}]},{"osd":5,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.76700000000000002,"15min":0.71699999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":6.25,"5min":6.25,"15min":6.25},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.71099999999999997,"15min":0.65200000000000002},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.93300000000000005,"5min":1.115,"15min":1.115},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.96399999999999997,"5min":1.165,"15min":1.165},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Aug 11 14:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.254,"5min":3.254,"15min":3.254},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.69399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.877},{"interface":"front","average":{"1min":1.224,"5min":0.90400000000000003,"15min":0.85799999999999998},"min":{"1min":0.68100000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.89100000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":0.80000000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.85499999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.878,"15min":0.85999999999999999},"min":{"1min":0.747,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":6.3739999999999997,"5min":6.3739999999999997,"15min":6.3739999999999997},"last":0.92600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":547129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.42599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.52300000000000002},"min":{"1min":0.40100000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46300000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46700000000000003,"15min":0.40899999999999997},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.80500000000000005,"5min":3.218,"15min":3.218},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47099999999999997,"15min":0.42899999999999999},"min":{"1min":0.39200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.496}]},{"osd":7,"last update":"Fri Aug 11 14:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.75700000000000001}]},{"osd":8,"last update":"Fri Aug 11 14:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61399999999999999,"15min":0.57499999999999996},"min":{"1min":0.46600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.014,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.64700000000000002,"15min":0.60699999999999998},"min":{"1min":0.52300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0740000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.59799999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.59199999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.052,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.60599999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.82299999999999995}]},{"osd":11,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.42199999999999999},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.876}]},{"osd":1,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.68600000000000005,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.423,"5min":6.423,"15min":6.423},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.61299999999999999},"min":{"1min":0.57199999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":2.548,"15min":2.548},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.72499999999999998,"15min":0.65500000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":6.5010000000000003,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85299999999999998,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.363,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.622},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72999999999999998,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Aug 11 14:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.80600000000000005},"min":{"1min":0.626,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96699999999999997,"5min":1.165,"15min":1.165},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.032,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:02:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.97699999999999998,"15min":1.0009999999999999},"min":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0409999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0649999999999999,"5min":1.891,"15min":1.891},"last":0.56000000000000005}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1379999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.83999999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.59899999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63200000000000001,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.40500000000000003,"15min":0.379},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59999999999999998,"5min":0.875,"15min":0.875},"last":0.432}]},{"osd":2,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.49399999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.60099999999999998},"min":{"1min":0.41099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68799999999999994}]},{"osd":8,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95599999999999996,"5min":1.02,"15min":1.02},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.753,"15min":0.747},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.748},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64800000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99399999999999999,"5min":1.075,"15min":1.075},"last":0.77000000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67500000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.46,"15min":1.46},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.81399999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.39500000000000002},"min":{"1min":0.317,"5min":0.251,"15min":0.251},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.41899999999999998},"min":{"1min":0.33900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":1.409,"15min":1.409},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.63400000000000001,"15min":0.59399999999999997},"min":{"1min":0.46800000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.4900000000000002,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90400000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.75,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.55800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:03:00.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph pg dump --format=json 2023-08-11T14:03:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:01.619 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:01 smithi003 ceph-mon[97195]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:01.619 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:01 smithi003 ceph-mon[97195]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:01 smithi103 ceph-mon[130434]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:01.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:01 smithi103 ceph-mon[130434]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:01.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:01 smithi187 ceph-mon[130973]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:01.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:01 smithi187 ceph-mon[130973]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.388+0000 7fc5b52a1700 1 -- 172.21.15.3:0/125446728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 msgr2=0x7fc5b010f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.388+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/125446728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b010f5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800b0a0 tx=0x7fc5a800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 -- 172.21.15.3:0/125446728 shutdown_connections 2023-08-11T14:03:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/125446728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b010f5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/125446728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/125446728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 -- 172.21.15.3:0/125446728 >> 172.21.15.3:0/125446728 conn(0x7fc5b00fc410 msgr2=0x7fc5b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 -- 172.21.15.3:0/125446728 shutdown_connections 2023-08-11T14:03:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.389+0000 7fc5b52a1700 1 -- 172.21.15.3:0/125446728 wait complete. 2023-08-11T14:03:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 Processor -- start 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 -- start start 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019dd50 con 0x7fc5b010d180 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019dec0 con 0x7fc5b0072d50 2023-08-11T14:03:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.390+0000 7fc5b52a1700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b019e080 con 0x7fc5b0073670 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5aeffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5aeffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39384/0 (socket says 172.21.15.3:39384) 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5aeffd700 1 -- 172.21.15.3:0/1659359487 learned_addr learned my addr 172.21.15.3:0/1659359487 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 -- 172.21.15.3:0/1659359487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 msgr2=0x7fc5b0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 -- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 msgr2=0x7fc5b019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 -- 172.21.15.3:0/1659359487 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8009d00 con 0x7fc5b0073670 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5af7fe700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.391+0000 7fc5ae7fc700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b01992d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e9f0 tx=0x7fc5a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.392+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400c9c0 con 0x7fc5b0073670 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.392+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b019e360 con 0x7fc5b0073670 2023-08-11T14:03:02.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.392+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a4010430 con 0x7fc5b0073670 2023-08-11T14:03:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.392+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4018680 con 0x7fc5b0073670 2023-08-11T14:03:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.392+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b01a5bc0 con 0x7fc5b0073670 2023-08-11T14:03:02.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.393+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b004e620 con 0x7fc5b0073670 2023-08-11T14:03:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.395+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc5a4010860 con 0x7fc5b0073670 2023-08-11T14:03:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.396+0000 7fc597fff700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 0x7fc5980a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.396+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc5a4014070 con 0x7fc5b0073670 2023-08-11T14:03:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.396+0000 7fc5aeffd700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 0x7fc5980a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:02.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.397+0000 7fc5aeffd700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 0x7fc5980a2410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0007c90 tx=0x7fc5a0007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:02.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.400+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc5a4051c30 con 0x7fc5b0073670 2023-08-11T14:03:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.603+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5b0112da0 con 0x7fc59809ff50 2023-08-11T14:03:02.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.616+0000 7fc597fff700 1 -- 172.21.15.3:0/1659359487 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426521 (secure 0 0 0) 0x7fc5b0112da0 con 0x7fc59809ff50 2023-08-11T14:03:02.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:02.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 msgr2=0x7fc5980a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 0x7fc5980a2410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0007c90 tx=0x7fc5a0007480 comp rx=0 tx=0).stop 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 msgr2=0x7fc5b01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b01992d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e9f0 tx=0x7fc5a400ed00 comp rx=0 tx=0).stop 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 shutdown_connections 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5b010d180 0x7fc5b019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc59809ff50 0x7fc5980a2410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5b0072d50 0x7fc5b0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 --2- 172.21.15.3:0/1659359487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5b0073670 0x7fc5b01992d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 >> 172.21.15.3:0/1659359487 conn(0x7fc5b00fc410 msgr2=0x7fc5b010f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 shutdown_connections 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:02.622+0000 7fc5b52a1700 1 -- 172.21.15.3:0/1659359487 wait complete. 2023-08-11T14:03:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:03:03.100 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2023-08-11T14:03:02.186537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657867,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":743,"num_read_kb":490,"num_write":421,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338164,"kb_used_data":13932,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603660,"statfs":{"total":1151940427776,"available":1151594147840,"internally_reserved":0,"allocated":14266368,"data_stored":7822493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331792+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:02:03.331792+0000","last_peered":"2023-08-11T14:02:03.331792+0000","last_clean":"2023-08-11T14:02:03.331792+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:02:03.331792+0000","last_undegraded":"2023-08-11T14:02:03.331792+0000","last_fullsized":"2023-08-11T14:02:03.331792+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:51:58.428642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337871+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:02:03.337871+0000","last_peered":"2023-08-11T14:02:03.337871+0000","last_clean":"2023-08-11T14:02:03.337871+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:02:03.337871+0000","last_undegraded":"2023-08-11T14:02:03.337871+0000","last_fullsized":"2023-08-11T14:02:03.337871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:27.291943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337376+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:02:03.337376+0000","last_peered":"2023-08-11T14:02:03.337376+0000","last_clean":"2023-08-11T14:02:03.337376+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:02:03.337376+0000","last_undegraded":"2023-08-11T14:02:03.337376+0000","last_fullsized":"2023-08-11T14:02:03.337376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:58:11.187863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334817+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:02:03.334817+0000","last_peered":"2023-08-11T14:02:03.334817+0000","last_clean":"2023-08-11T14:02:03.334817+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:02:03.334817+0000","last_undegraded":"2023-08-11T14:02:03.334817+0000","last_fullsized":"2023-08-11T14:02:03.334817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:57:40.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334219+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:02:03.334219+0000","last_peered":"2023-08-11T14:02:03.334219+0000","last_clean":"2023-08-11T14:02:03.334219+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:02:03.334219+0000","last_undegraded":"2023-08-11T14:02:03.334219+0000","last_fullsized":"2023-08-11T14:02:03.334219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:18:50.744427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331881+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:02:03.331881+0000","last_peered":"2023-08-11T14:02:03.331881+0000","last_clean":"2023-08-11T14:02:03.331881+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:02:03.331881+0000","last_undegraded":"2023-08-11T14:02:03.331881+0000","last_fullsized":"2023-08-11T14:02:03.331881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:46:55.482691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513583+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:02:03.513583+0000","last_peered":"2023-08-11T14:02:03.513583+0000","last_clean":"2023-08-11T14:02:03.513583+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:02:03.513583+0000","last_undegraded":"2023-08-11T14:02:03.513583+0000","last_fullsized":"2023-08-11T14:02:03.513583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:35:40.721439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.339952+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:02:03.339952+0000","last_peered":"2023-08-11T14:02:03.339952+0000","last_clean":"2023-08-11T14:02:03.339952+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:02:03.339952+0000","last_undegraded":"2023-08-11T14:02:03.339952+0000","last_fullsized":"2023-08-11T14:02:03.339952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:07:02.851636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514145+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:02:03.514145+0000","last_peered":"2023-08-11T14:02:03.514145+0000","last_clean":"2023-08-11T14:02:03.514145+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:02:03.514145+0000","last_undegraded":"2023-08-11T14:02:03.514145+0000","last_fullsized":"2023-08-11T14:02:03.514145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:17:30.192305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333065+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:02:03.333065+0000","last_peered":"2023-08-11T14:02:03.333065+0000","last_clean":"2023-08-11T14:02:03.333065+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:02:03.333065+0000","last_undegraded":"2023-08-11T14:02:03.333065+0000","last_fullsized":"2023-08-11T14:02:03.333065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:08:46.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514583+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:02:03.514583+0000","last_peered":"2023-08-11T14:02:03.514583+0000","last_clean":"2023-08-11T14:02:03.514583+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:02:03.514583+0000","last_undegraded":"2023-08-11T14:02:03.514583+0000","last_fullsized":"2023-08-11T14:02:03.514583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:45:58.389232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334716+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:02:03.334716+0000","last_peered":"2023-08-11T14:02:03.334716+0000","last_clean":"2023-08-11T14:02:03.334716+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:02:03.334716+0000","last_undegraded":"2023-08-11T14:02:03.334716+0000","last_fullsized":"2023-08-11T14:02:03.334716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:26:02.515253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331544+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:02:03.331544+0000","last_peered":"2023-08-11T14:02:03.331544+0000","last_clean":"2023-08-11T14:02:03.331544+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:02:03.331544+0000","last_undegraded":"2023-08-11T14:02:03.331544+0000","last_fullsized":"2023-08-11T14:02:03.331544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:19.525319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335946+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:02:03.335946+0000","last_peered":"2023-08-11T14:02:03.335946+0000","last_clean":"2023-08-11T14:02:03.335946+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:02:03.335946+0000","last_undegraded":"2023-08-11T14:02:03.335946+0000","last_fullsized":"2023-08-11T14:02:03.335946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:47.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335730+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:02:03.335730+0000","last_peered":"2023-08-11T14:02:03.335730+0000","last_clean":"2023-08-11T14:02:03.335730+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:02:03.335730+0000","last_undegraded":"2023-08-11T14:02:03.335730+0000","last_fullsized":"2023-08-11T14:02:03.335730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:22.148404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514351+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:02:03.514351+0000","last_peered":"2023-08-11T14:02:03.514351+0000","last_clean":"2023-08-11T14:02:03.514351+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:02:03.514351+0000","last_undegraded":"2023-08-11T14:02:03.514351+0000","last_fullsized":"2023-08-11T14:02:03.514351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:53:22.056513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331885+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:02:03.331885+0000","last_peered":"2023-08-11T14:02:03.331885+0000","last_clean":"2023-08-11T14:02:03.331885+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:02:03.331885+0000","last_undegraded":"2023-08-11T14:02:03.331885+0000","last_fullsized":"2023-08-11T14:02:03.331885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:15:42.811935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330963+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:02:03.330963+0000","last_peered":"2023-08-11T14:02:03.330963+0000","last_clean":"2023-08-11T14:02:03.330963+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:02:03.330963+0000","last_undegraded":"2023-08-11T14:02:03.330963+0000","last_fullsized":"2023-08-11T14:02:03.330963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:55:51.334309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329428+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:02:03.329428+0000","last_peered":"2023-08-11T14:02:03.329428+0000","last_clean":"2023-08-11T14:02:03.329428+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:02:03.329428+0000","last_undegraded":"2023-08-11T14:02:03.329428+0000","last_fullsized":"2023-08-11T14:02:03.329428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:25:20.846946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328746+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:02:03.328746+0000","last_peered":"2023-08-11T14:02:03.328746+0000","last_clean":"2023-08-11T14:02:03.328746+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:02:03.328746+0000","last_undegraded":"2023-08-11T14:02:03.328746+0000","last_fullsized":"2023-08-11T14:02:03.328746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:45.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:56.412886+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:02:56.412886+0000","last_peered":"2023-08-11T14:02:56.412886+0000","last_clean":"2023-08-11T14:02:56.412886+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:02:56.412886+0000","last_undegraded":"2023-08-11T14:02:56.412886+0000","last_fullsized":"2023-08-11T14:02:56.412886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:03:49.543185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336557+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:02:03.336557+0000","last_peered":"2023-08-11T14:02:03.336557+0000","last_clean":"2023-08-11T14:02:03.336557+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:02:03.336557+0000","last_undegraded":"2023-08-11T14:02:03.336557+0000","last_fullsized":"2023-08-11T14:02:03.336557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:20:05.473927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332900+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:02:03.332900+0000","last_peered":"2023-08-11T14:02:03.332900+0000","last_clean":"2023-08-11T14:02:03.332900+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:02:03.332900+0000","last_undegraded":"2023-08-11T14:02:03.332900+0000","last_fullsized":"2023-08-11T14:02:03.332900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:07.976859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336699+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:02:03.336699+0000","last_peered":"2023-08-11T14:02:03.336699+0000","last_clean":"2023-08-11T14:02:03.336699+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:02:03.336699+0000","last_undegraded":"2023-08-11T14:02:03.336699+0000","last_fullsized":"2023-08-11T14:02:03.336699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:11:49.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336948+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:02:03.336948+0000","last_peered":"2023-08-11T14:02:03.336948+0000","last_clean":"2023-08-11T14:02:03.336948+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:02:03.336948+0000","last_undegraded":"2023-08-11T14:02:03.336948+0000","last_fullsized":"2023-08-11T14:02:03.336948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:59:07.982750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334019+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:02:03.334019+0000","last_peered":"2023-08-11T14:02:03.334019+0000","last_clean":"2023-08-11T14:02:03.334019+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:02:03.334019+0000","last_undegraded":"2023-08-11T14:02:03.334019+0000","last_fullsized":"2023-08-11T14:02:03.334019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:02:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334878+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:02:03.334878+0000","last_peered":"2023-08-11T14:02:03.334878+0000","last_clean":"2023-08-11T14:02:03.334878+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:02:03.334878+0000","last_undegraded":"2023-08-11T14:02:03.334878+0000","last_fullsized":"2023-08-11T14:02:03.334878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:53:51.516259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513358+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:02:03.513358+0000","last_peered":"2023-08-11T14:02:03.513358+0000","last_clean":"2023-08-11T14:02:03.513358+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:02:03.513358+0000","last_undegraded":"2023-08-11T14:02:03.513358+0000","last_fullsized":"2023-08-11T14:02:03.513358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:08.181790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333088+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:02:03.333088+0000","last_peered":"2023-08-11T14:02:03.333088+0000","last_clean":"2023-08-11T14:02:03.333088+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:02:03.333088+0000","last_undegraded":"2023-08-11T14:02:03.333088+0000","last_fullsized":"2023-08-11T14:02:03.333088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:57.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334372+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:02:03.334372+0000","last_peered":"2023-08-11T14:02:03.334372+0000","last_clean":"2023-08-11T14:02:03.334372+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:02:03.334372+0000","last_undegraded":"2023-08-11T14:02:03.334372+0000","last_fullsized":"2023-08-11T14:02:03.334372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:25:15.703177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336854+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:02:03.336854+0000","last_peered":"2023-08-11T14:02:03.336854+0000","last_clean":"2023-08-11T14:02:03.336854+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:02:03.336854+0000","last_undegraded":"2023-08-11T14:02:03.336854+0000","last_fullsized":"2023-08-11T14:02:03.336854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:04:24.882002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329257+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:02:03.329257+0000","last_peered":"2023-08-11T14:02:03.329257+0000","last_clean":"2023-08-11T14:02:03.329257+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:02:03.329257+0000","last_undegraded":"2023-08-11T14:02:03.329257+0000","last_fullsized":"2023-08-11T14:02:03.329257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:02.214605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:55.402711+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:02:55.402711+0000","last_peered":"2023-08-11T14:02:55.402711+0000","last_clean":"2023-08-11T14:02:55.402711+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:02:55.402711+0000","last_undegraded":"2023-08-11T14:02:55.402711+0000","last_fullsized":"2023-08-11T14:02:55.402711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:15.794565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334377+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:02:03.334377+0000","last_peered":"2023-08-11T14:02:03.334377+0000","last_clean":"2023-08-11T14:02:03.334377+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:02:03.334377+0000","last_undegraded":"2023-08-11T14:02:03.334377+0000","last_fullsized":"2023-08-11T14:02:03.334377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:18:18.096428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329041+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:02:03.329041+0000","last_peered":"2023-08-11T14:02:03.329041+0000","last_clean":"2023-08-11T14:02:03.329041+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:02:03.329041+0000","last_undegraded":"2023-08-11T14:02:03.329041+0000","last_fullsized":"2023-08-11T14:02:03.329041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:48:20.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331831+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:02:03.331831+0000","last_peered":"2023-08-11T14:02:03.331831+0000","last_clean":"2023-08-11T14:02:03.331831+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:02:03.331831+0000","last_undegraded":"2023-08-11T14:02:03.331831+0000","last_fullsized":"2023-08-11T14:02:03.331831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:20:40.181885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337337+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:02:03.337337+0000","last_peered":"2023-08-11T14:02:03.337337+0000","last_clean":"2023-08-11T14:02:03.337337+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:02:03.337337+0000","last_undegraded":"2023-08-11T14:02:03.337337+0000","last_fullsized":"2023-08-11T14:02:03.337337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:24:08.010819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331299+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:02:03.331299+0000","last_peered":"2023-08-11T14:02:03.331299+0000","last_clean":"2023-08-11T14:02:03.331299+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:02:03.331299+0000","last_undegraded":"2023-08-11T14:02:03.331299+0000","last_fullsized":"2023-08-11T14:02:03.331299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:23:42.594346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331880+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:02:03.331880+0000","last_peered":"2023-08-11T14:02:03.331880+0000","last_clean":"2023-08-11T14:02:03.331880+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:02:03.331880+0000","last_undegraded":"2023-08-11T14:02:03.331880+0000","last_fullsized":"2023-08-11T14:02:03.331880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:31.945260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335468+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:02:03.335468+0000","last_peered":"2023-08-11T14:02:03.335468+0000","last_clean":"2023-08-11T14:02:03.335468+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:02:03.335468+0000","last_undegraded":"2023-08-11T14:02:03.335468+0000","last_fullsized":"2023-08-11T14:02:03.335468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:11.769236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337591+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:02:03.337591+0000","last_peered":"2023-08-11T14:02:03.337591+0000","last_clean":"2023-08-11T14:02:03.337591+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:02:03.337591+0000","last_undegraded":"2023-08-11T14:02:03.337591+0000","last_fullsized":"2023-08-11T14:02:03.337591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:42:20.068831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336838+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:02:03.336838+0000","last_peered":"2023-08-11T14:02:03.336838+0000","last_clean":"2023-08-11T14:02:03.336838+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:02:03.336838+0000","last_undegraded":"2023-08-11T14:02:03.336838+0000","last_fullsized":"2023-08-11T14:02:03.336838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:02.833607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330576+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:02:03.330576+0000","last_peered":"2023-08-11T14:02:03.330576+0000","last_clean":"2023-08-11T14:02:03.330576+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:02:03.330576+0000","last_undegraded":"2023-08-11T14:02:03.330576+0000","last_fullsized":"2023-08-11T14:02:03.330576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:21.461662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337143+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:02:03.337143+0000","last_peered":"2023-08-11T14:02:03.337143+0000","last_clean":"2023-08-11T14:02:03.337143+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:02:03.337143+0000","last_undegraded":"2023-08-11T14:02:03.337143+0000","last_fullsized":"2023-08-11T14:02:03.337143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:32:51.835743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:58.406643+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:02:58.406643+0000","last_peered":"2023-08-11T14:02:58.406643+0000","last_clean":"2023-08-11T14:02:58.406643+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:02:58.406643+0000","last_undegraded":"2023-08-11T14:02:58.406643+0000","last_fullsized":"2023-08-11T14:02:58.406643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:21:36.864881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514217+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:02:03.514217+0000","last_peered":"2023-08-11T14:02:03.514217+0000","last_clean":"2023-08-11T14:02:03.514217+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:02:03.514217+0000","last_undegraded":"2023-08-11T14:02:03.514217+0000","last_fullsized":"2023-08-11T14:02:03.514217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:49:01.488839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330205+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:02:03.330205+0000","last_peered":"2023-08-11T14:02:03.330205+0000","last_clean":"2023-08-11T14:02:03.330205+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:02:03.330205+0000","last_undegraded":"2023-08-11T14:02:03.330205+0000","last_fullsized":"2023-08-11T14:02:03.330205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:44:32.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514239+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:02:03.514239+0000","last_peered":"2023-08-11T14:02:03.514239+0000","last_clean":"2023-08-11T14:02:03.514239+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:02:03.514239+0000","last_undegraded":"2023-08-11T14:02:03.514239+0000","last_fullsized":"2023-08-11T14:02:03.514239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:20.455636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.342277+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:02:03.342277+0000","last_peered":"2023-08-11T14:02:03.342277+0000","last_clean":"2023-08-11T14:02:03.342277+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:02:03.342277+0000","last_undegraded":"2023-08-11T14:02:03.342277+0000","last_fullsized":"2023-08-11T14:02:03.342277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:58:39.191618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334375+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:02:03.334375+0000","last_peered":"2023-08-11T14:02:03.334375+0000","last_clean":"2023-08-11T14:02:03.334375+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:02:03.334375+0000","last_undegraded":"2023-08-11T14:02:03.334375+0000","last_fullsized":"2023-08-11T14:02:03.334375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:13.333071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336657+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:02:03.336657+0000","last_peered":"2023-08-11T14:02:03.336657+0000","last_clean":"2023-08-11T14:02:03.336657+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:02:03.336657+0000","last_undegraded":"2023-08-11T14:02:03.336657+0000","last_fullsized":"2023-08-11T14:02:03.336657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:50:48.206539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513781+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:02:03.513781+0000","last_peered":"2023-08-11T14:02:03.513781+0000","last_clean":"2023-08-11T14:02:03.513781+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:02:03.513781+0000","last_undegraded":"2023-08-11T14:02:03.513781+0000","last_fullsized":"2023-08-11T14:02:03.513781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:08.653787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330092+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:02:03.330092+0000","last_peered":"2023-08-11T14:02:03.330092+0000","last_clean":"2023-08-11T14:02:03.330092+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:02:03.330092+0000","last_undegraded":"2023-08-11T14:02:03.330092+0000","last_fullsized":"2023-08-11T14:02:03.330092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:39:16.568599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":182,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:58.417960+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:02:58.417960+0000","last_peered":"2023-08-11T14:02:58.417960+0000","last_clean":"2023-08-11T14:02:58.417960+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:02:58.417960+0000","last_undegraded":"2023-08-11T14:02:58.417960+0000","last_fullsized":"2023-08-11T14:02:58.417960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:27:42.493851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":122,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334731+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:02:03.334731+0000","last_peered":"2023-08-11T14:02:03.334731+0000","last_clean":"2023-08-11T14:02:03.334731+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:02:03.334731+0000","last_undegraded":"2023-08-11T14:02:03.334731+0000","last_fullsized":"2023-08-11T14:02:03.334731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:13.286447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337433+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:02:03.337433+0000","last_peered":"2023-08-11T14:02:03.337433+0000","last_clean":"2023-08-11T14:02:03.337433+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:02:03.337433+0000","last_undegraded":"2023-08-11T14:02:03.337433+0000","last_fullsized":"2023-08-11T14:02:03.337433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:51:30.827438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.351142+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:02:03.351142+0000","last_peered":"2023-08-11T14:02:03.351142+0000","last_clean":"2023-08-11T14:02:03.351142+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:02:03.351142+0000","last_undegraded":"2023-08-11T14:02:03.351142+0000","last_fullsized":"2023-08-11T14:02:03.351142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:52:44.646062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331432+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:02:03.331432+0000","last_peered":"2023-08-11T14:02:03.331432+0000","last_clean":"2023-08-11T14:02:03.331432+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:02:03.331432+0000","last_undegraded":"2023-08-11T14:02:03.331432+0000","last_fullsized":"2023-08-11T14:02:03.331432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:53:07.295009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515889+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:02:03.515889+0000","last_peered":"2023-08-11T14:02:03.515889+0000","last_clean":"2023-08-11T14:02:03.515889+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:02:03.515889+0000","last_undegraded":"2023-08-11T14:02:03.515889+0000","last_fullsized":"2023-08-11T14:02:03.515889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:46:12.408393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336996+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:02:03.336996+0000","last_peered":"2023-08-11T14:02:03.336996+0000","last_clean":"2023-08-11T14:02:03.336996+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:02:03.336996+0000","last_undegraded":"2023-08-11T14:02:03.336996+0000","last_fullsized":"2023-08-11T14:02:03.336996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:38:48.870033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514281+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:02:03.514281+0000","last_peered":"2023-08-11T14:02:03.514281+0000","last_clean":"2023-08-11T14:02:03.514281+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:02:03.514281+0000","last_undegraded":"2023-08-11T14:02:03.514281+0000","last_fullsized":"2023-08-11T14:02:03.514281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:49:45.249610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.516664+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:02:03.516664+0000","last_peered":"2023-08-11T14:02:03.516664+0000","last_clean":"2023-08-11T14:02:03.516664+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:02:03.516664+0000","last_undegraded":"2023-08-11T14:02:03.516664+0000","last_fullsized":"2023-08-11T14:02:03.516664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:12:50.860375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:03:00.420730+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:03:00.420730+0000","last_peered":"2023-08-11T14:03:00.420730+0000","last_clean":"2023-08-11T14:03:00.420730+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:03:00.420730+0000","last_undegraded":"2023-08-11T14:03:00.420730+0000","last_fullsized":"2023-08-11T14:03:00.420730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:55:02.003859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513709+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:02:03.513709+0000","last_peered":"2023-08-11T14:02:03.513709+0000","last_clean":"2023-08-11T14:02:03.513709+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:02:03.513709+0000","last_undegraded":"2023-08-11T14:02:03.513709+0000","last_fullsized":"2023-08-11T14:02:03.513709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:17:14.428403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.341361+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:02:03.341361+0000","last_peered":"2023-08-11T14:02:03.341361+0000","last_clean":"2023-08-11T14:02:03.341361+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:02:03.341361+0000","last_undegraded":"2023-08-11T14:02:03.341361+0000","last_fullsized":"2023-08-11T14:02:03.341361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:52:45.568837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333872+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:02:03.333872+0000","last_peered":"2023-08-11T14:02:03.333872+0000","last_clean":"2023-08-11T14:02:03.333872+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:02:03.333872+0000","last_undegraded":"2023-08-11T14:02:03.333872+0000","last_fullsized":"2023-08-11T14:02:03.333872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:07:09.384734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.515481+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:02:03.515481+0000","last_peered":"2023-08-11T14:02:03.515481+0000","last_clean":"2023-08-11T14:02:03.515481+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:02:03.515481+0000","last_undegraded":"2023-08-11T14:02:03.515481+0000","last_fullsized":"2023-08-11T14:02:03.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:07:44.577236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336675+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:02:03.336675+0000","last_peered":"2023-08-11T14:02:03.336675+0000","last_clean":"2023-08-11T14:02:03.336675+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:02:03.336675+0000","last_undegraded":"2023-08-11T14:02:03.336675+0000","last_fullsized":"2023-08-11T14:02:03.336675+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:37:47.264671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335006+0000","last_change":"2023-08-11T14:01:32.297291+0000","last_active":"2023-08-11T14:02:03.335006+0000","last_peered":"2023-08-11T14:02:03.335006+0000","last_clean":"2023-08-11T14:02:03.335006+0000","last_became_active":"2023-08-11T14:01:32.296546+0000","last_became_peered":"2023-08-11T14:01:32.296546+0000","last_unstale":"2023-08-11T14:02:03.335006+0000","last_undegraded":"2023-08-11T14:02:03.335006+0000","last_fullsized":"2023-08-11T14:02:03.335006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:01:59.092217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331497+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:02:03.331497+0000","last_peered":"2023-08-11T14:02:03.331497+0000","last_clean":"2023-08-11T14:02:03.331497+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:02:03.331497+0000","last_undegraded":"2023-08-11T14:02:03.331497+0000","last_fullsized":"2023-08-11T14:02:03.331497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:57:13.361686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334195+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:02:03.334195+0000","last_peered":"2023-08-11T14:02:03.334195+0000","last_clean":"2023-08-11T14:02:03.334195+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:02:03.334195+0000","last_undegraded":"2023-08-11T14:02:03.334195+0000","last_fullsized":"2023-08-11T14:02:03.334195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:30:24.527602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330437+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:02:03.330437+0000","last_peered":"2023-08-11T14:02:03.330437+0000","last_clean":"2023-08-11T14:02:03.330437+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:02:03.330437+0000","last_undegraded":"2023-08-11T14:02:03.330437+0000","last_fullsized":"2023-08-11T14:02:03.330437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:52.907857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333207+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:02:03.333207+0000","last_peered":"2023-08-11T14:02:03.333207+0000","last_clean":"2023-08-11T14:02:03.333207+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:02:03.333207+0000","last_undegraded":"2023-08-11T14:02:03.333207+0000","last_fullsized":"2023-08-11T14:02:03.333207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:57:24.232067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331907+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:02:03.331907+0000","last_peered":"2023-08-11T14:02:03.331907+0000","last_clean":"2023-08-11T14:02:03.331907+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:02:03.331907+0000","last_undegraded":"2023-08-11T14:02:03.331907+0000","last_fullsized":"2023-08-11T14:02:03.331907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:36:27.156211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:58.416507+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:02:58.416507+0000","last_peered":"2023-08-11T14:02:58.416507+0000","last_clean":"2023-08-11T14:02:58.416507+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:02:58.416507+0000","last_undegraded":"2023-08-11T14:02:58.416507+0000","last_fullsized":"2023-08-11T14:02:58.416507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:47:00.825364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332087+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:02:03.332087+0000","last_peered":"2023-08-11T14:02:03.332087+0000","last_clean":"2023-08-11T14:02:03.332087+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:02:03.332087+0000","last_undegraded":"2023-08-11T14:02:03.332087+0000","last_fullsized":"2023-08-11T14:02:03.332087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:51:19.363482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332389+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:02:03.332389+0000","last_peered":"2023-08-11T14:02:03.332389+0000","last_clean":"2023-08-11T14:02:03.332389+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:02:03.332389+0000","last_undegraded":"2023-08-11T14:02:03.332389+0000","last_fullsized":"2023-08-11T14:02:03.332389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:32:17.902481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337152+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:02:03.337152+0000","last_peered":"2023-08-11T14:02:03.337152+0000","last_clean":"2023-08-11T14:02:03.337152+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:02:03.337152+0000","last_undegraded":"2023-08-11T14:02:03.337152+0000","last_fullsized":"2023-08-11T14:02:03.337152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:14.305302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:57.416500+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:02:57.416500+0000","last_peered":"2023-08-11T14:02:57.416500+0000","last_clean":"2023-08-11T14:02:57.416500+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:02:57.416500+0000","last_undegraded":"2023-08-11T14:02:57.416500+0000","last_fullsized":"2023-08-11T14:02:57.416500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:03:26.301499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514250+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:02:03.514250+0000","last_peered":"2023-08-11T14:02:03.514250+0000","last_clean":"2023-08-11T14:02:03.514250+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:02:03.514250+0000","last_undegraded":"2023-08-11T14:02:03.514250+0000","last_fullsized":"2023-08-11T14:02:03.514250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:10:32.366645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335798+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:02:03.335798+0000","last_peered":"2023-08-11T14:02:03.335798+0000","last_clean":"2023-08-11T14:02:03.335798+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:02:03.335798+0000","last_undegraded":"2023-08-11T14:02:03.335798+0000","last_fullsized":"2023-08-11T14:02:03.335798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:42.873872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334095+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:02:03.334095+0000","last_peered":"2023-08-11T14:02:03.334095+0000","last_clean":"2023-08-11T14:02:03.334095+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:02:03.334095+0000","last_undegraded":"2023-08-11T14:02:03.334095+0000","last_fullsized":"2023-08-11T14:02:03.334095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:58:57.384759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513140+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:02:03.513140+0000","last_peered":"2023-08-11T14:02:03.513140+0000","last_clean":"2023-08-11T14:02:03.513140+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:02:03.513140+0000","last_undegraded":"2023-08-11T14:02:03.513140+0000","last_fullsized":"2023-08-11T14:02:03.513140+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:00:21.425175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336924+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:02:03.336924+0000","last_peered":"2023-08-11T14:02:03.336924+0000","last_clean":"2023-08-11T14:02:03.336924+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:02:03.336924+0000","last_undegraded":"2023-08-11T14:02:03.336924+0000","last_fullsized":"2023-08-11T14:02:03.336924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:12:00.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335496+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:02:03.335496+0000","last_peered":"2023-08-11T14:02:03.335496+0000","last_clean":"2023-08-11T14:02:03.335496+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:02:03.335496+0000","last_undegraded":"2023-08-11T14:02:03.335496+0000","last_fullsized":"2023-08-11T14:02:03.335496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:55:24.249331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513547+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:02:03.513547+0000","last_peered":"2023-08-11T14:02:03.513547+0000","last_clean":"2023-08-11T14:02:03.513547+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:02:03.513547+0000","last_undegraded":"2023-08-11T14:02:03.513547+0000","last_fullsized":"2023-08-11T14:02:03.513547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:43:14.258870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331490+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:02:03.331490+0000","last_peered":"2023-08-11T14:02:03.331490+0000","last_clean":"2023-08-11T14:02:03.331490+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:02:03.331490+0000","last_undegraded":"2023-08-11T14:02:03.331490+0000","last_fullsized":"2023-08-11T14:02:03.331490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:03:13.665333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334559+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:02:03.334559+0000","last_peered":"2023-08-11T14:02:03.334559+0000","last_clean":"2023-08-11T14:02:03.334559+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:02:03.334559+0000","last_undegraded":"2023-08-11T14:02:03.334559+0000","last_fullsized":"2023-08-11T14:02:03.334559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:54:12.781265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332044+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:02:03.332044+0000","last_peered":"2023-08-11T14:02:03.332044+0000","last_clean":"2023-08-11T14:02:03.332044+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:02:03.332044+0000","last_undegraded":"2023-08-11T14:02:03.332044+0000","last_fullsized":"2023-08-11T14:02:03.332044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:31:49.139627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335327+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:02:03.335327+0000","last_peered":"2023-08-11T14:02:03.335327+0000","last_clean":"2023-08-11T14:02:03.335327+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:02:03.335327+0000","last_undegraded":"2023-08-11T14:02:03.335327+0000","last_fullsized":"2023-08-11T14:02:03.335327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:09:53.699206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'10","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:07.097233+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:02:07.097233+0000","last_peered":"2023-08-11T14:02:07.097233+0000","last_clean":"2023-08-11T14:02:07.097233+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:02:07.097233+0000","last_undegraded":"2023-08-11T14:02:07.097233+0000","last_fullsized":"2023-08-11T14:02:07.097233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:08:56.731924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335320+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:02:03.335320+0000","last_peered":"2023-08-11T14:02:03.335320+0000","last_clean":"2023-08-11T14:02:03.335320+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:02:03.335320+0000","last_undegraded":"2023-08-11T14:02:03.335320+0000","last_fullsized":"2023-08-11T14:02:03.335320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:01:44.639732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328579+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:02:03.328579+0000","last_peered":"2023-08-11T14:02:03.328579+0000","last_clean":"2023-08-11T14:02:03.328579+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:02:03.328579+0000","last_undegraded":"2023-08-11T14:02:03.328579+0000","last_fullsized":"2023-08-11T14:02:03.328579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:30:22.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334471+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:02:03.334471+0000","last_peered":"2023-08-11T14:02:03.334471+0000","last_clean":"2023-08-11T14:02:03.334471+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:02:03.334471+0000","last_undegraded":"2023-08-11T14:02:03.334471+0000","last_fullsized":"2023-08-11T14:02:03.334471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:15:41.175031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337934+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:02:03.337934+0000","last_peered":"2023-08-11T14:02:03.337934+0000","last_clean":"2023-08-11T14:02:03.337934+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:02:03.337934+0000","last_undegraded":"2023-08-11T14:02:03.337934+0000","last_fullsized":"2023-08-11T14:02:03.337934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:02:19.969087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.338205+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:02:03.338205+0000","last_peered":"2023-08-11T14:02:03.338205+0000","last_clean":"2023-08-11T14:02:03.338205+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:02:03.338205+0000","last_undegraded":"2023-08-11T14:02:03.338205+0000","last_fullsized":"2023-08-11T14:02:03.338205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:34:25.756753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514463+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:02:03.514463+0000","last_peered":"2023-08-11T14:02:03.514463+0000","last_clean":"2023-08-11T14:02:03.514463+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:02:03.514463+0000","last_undegraded":"2023-08-11T14:02:03.514463+0000","last_fullsized":"2023-08-11T14:02:03.514463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:33:19.003199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.333842+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:02:03.333842+0000","last_peered":"2023-08-11T14:02:03.333842+0000","last_clean":"2023-08-11T14:02:03.333842+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:02:03.333842+0000","last_undegraded":"2023-08-11T14:02:03.333842+0000","last_fullsized":"2023-08-11T14:02:03.333842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:33:45.029273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330601+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:02:03.330601+0000","last_peered":"2023-08-11T14:02:03.330601+0000","last_clean":"2023-08-11T14:02:03.330601+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:02:03.330601+0000","last_undegraded":"2023-08-11T14:02:03.330601+0000","last_fullsized":"2023-08-11T14:02:03.330601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:32:17.015669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337272+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:02:03.337272+0000","last_peered":"2023-08-11T14:02:03.337272+0000","last_clean":"2023-08-11T14:02:03.337272+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:02:03.337272+0000","last_undegraded":"2023-08-11T14:02:03.337272+0000","last_fullsized":"2023-08-11T14:02:03.337272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:49:35.317791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330540+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:02:03.330540+0000","last_peered":"2023-08-11T14:02:03.330540+0000","last_clean":"2023-08-11T14:02:03.330540+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:02:03.330540+0000","last_undegraded":"2023-08-11T14:02:03.330540+0000","last_fullsized":"2023-08-11T14:02:03.330540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:20.537811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334277+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:02:03.334277+0000","last_peered":"2023-08-11T14:02:03.334277+0000","last_clean":"2023-08-11T14:02:03.334277+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:02:03.334277+0000","last_undegraded":"2023-08-11T14:02:03.334277+0000","last_fullsized":"2023-08-11T14:02:03.334277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:02.518435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328837+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:02:03.328837+0000","last_peered":"2023-08-11T14:02:03.328837+0000","last_clean":"2023-08-11T14:02:03.328837+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:02:03.328837+0000","last_undegraded":"2023-08-11T14:02:03.328837+0000","last_fullsized":"2023-08-11T14:02:03.328837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:07:32.066321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335561+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:02:03.335561+0000","last_peered":"2023-08-11T14:02:03.335561+0000","last_clean":"2023-08-11T14:02:03.335561+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:02:03.335561+0000","last_undegraded":"2023-08-11T14:02:03.335561+0000","last_fullsized":"2023-08-11T14:02:03.335561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:27:34.867342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334192+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:02:03.334192+0000","last_peered":"2023-08-11T14:02:03.334192+0000","last_clean":"2023-08-11T14:02:03.334192+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:02:03.334192+0000","last_undegraded":"2023-08-11T14:02:03.334192+0000","last_fullsized":"2023-08-11T14:02:03.334192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:39.062659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.334116+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:02:03.334116+0000","last_peered":"2023-08-11T14:02:03.334116+0000","last_clean":"2023-08-11T14:02:03.334116+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:02:03.334116+0000","last_undegraded":"2023-08-11T14:02:03.334116+0000","last_fullsized":"2023-08-11T14:02:03.334116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:35:43.633835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513915+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:02:03.513915+0000","last_peered":"2023-08-11T14:02:03.513915+0000","last_clean":"2023-08-11T14:02:03.513915+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:02:03.513915+0000","last_undegraded":"2023-08-11T14:02:03.513915+0000","last_fullsized":"2023-08-11T14:02:03.513915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:29:09.762764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.513303+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:02:03.513303+0000","last_peered":"2023-08-11T14:02:03.513303+0000","last_clean":"2023-08-11T14:02:03.513303+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:02:03.513303+0000","last_undegraded":"2023-08-11T14:02:03.513303+0000","last_fullsized":"2023-08-11T14:02:03.513303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:16.532299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:58.421919+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:02:58.421919+0000","last_peered":"2023-08-11T14:02:58.421919+0000","last_clean":"2023-08-11T14:02:58.421919+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:02:58.421919+0000","last_undegraded":"2023-08-11T14:02:58.421919+0000","last_fullsized":"2023-08-11T14:02:58.421919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:20:36.499383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514603+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:02:03.514603+0000","last_peered":"2023-08-11T14:02:03.514603+0000","last_clean":"2023-08-11T14:02:03.514603+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:02:03.514603+0000","last_undegraded":"2023-08-11T14:02:03.514603+0000","last_fullsized":"2023-08-11T14:02:03.514603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:51.701185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337600+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:02:03.337600+0000","last_peered":"2023-08-11T14:02:03.337600+0000","last_clean":"2023-08-11T14:02:03.337600+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:02:03.337600+0000","last_undegraded":"2023-08-11T14:02:03.337600+0000","last_fullsized":"2023-08-11T14:02:03.337600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:03:29.935737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337020+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:02:03.337020+0000","last_peered":"2023-08-11T14:02:03.337020+0000","last_clean":"2023-08-11T14:02:03.337020+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:02:03.337020+0000","last_undegraded":"2023-08-11T14:02:03.337020+0000","last_fullsized":"2023-08-11T14:02:03.337020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:18:36.246178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514628+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:02:03.514628+0000","last_peered":"2023-08-11T14:02:03.514628+0000","last_clean":"2023-08-11T14:02:03.514628+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:02:03.514628+0000","last_undegraded":"2023-08-11T14:02:03.514628+0000","last_fullsized":"2023-08-11T14:02:03.514628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:08:49.752360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336382+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:02:03.336382+0000","last_peered":"2023-08-11T14:02:03.336382+0000","last_clean":"2023-08-11T14:02:03.336382+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:02:03.336382+0000","last_undegraded":"2023-08-11T14:02:03.336382+0000","last_fullsized":"2023-08-11T14:02:03.336382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:17:23.708072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332219+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:02:03.332219+0000","last_peered":"2023-08-11T14:02:03.332219+0000","last_clean":"2023-08-11T14:02:03.332219+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:02:03.332219+0000","last_undegraded":"2023-08-11T14:02:03.332219+0000","last_fullsized":"2023-08-11T14:02:03.332219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:31:46.922439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514566+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:02:03.514566+0000","last_peered":"2023-08-11T14:02:03.514566+0000","last_clean":"2023-08-11T14:02:03.514566+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:02:03.514566+0000","last_undegraded":"2023-08-11T14:02:03.514566+0000","last_fullsized":"2023-08-11T14:02:03.514566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:15:20.919187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330668+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:02:03.330668+0000","last_peered":"2023-08-11T14:02:03.330668+0000","last_clean":"2023-08-11T14:02:03.330668+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:02:03.330668+0000","last_undegraded":"2023-08-11T14:02:03.330668+0000","last_fullsized":"2023-08-11T14:02:03.330668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:47:54.099330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328891+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:02:03.328891+0000","last_peered":"2023-08-11T14:02:03.328891+0000","last_clean":"2023-08-11T14:02:03.328891+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:02:03.328891+0000","last_undegraded":"2023-08-11T14:02:03.328891+0000","last_fullsized":"2023-08-11T14:02:03.328891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:34:26.962324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.328645+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:02:03.328645+0000","last_peered":"2023-08-11T14:02:03.328645+0000","last_clean":"2023-08-11T14:02:03.328645+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:02:03.328645+0000","last_undegraded":"2023-08-11T14:02:03.328645+0000","last_fullsized":"2023-08-11T14:02:03.328645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:18:53.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337742+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:02:03.337742+0000","last_peered":"2023-08-11T14:02:03.337742+0000","last_clean":"2023-08-11T14:02:03.337742+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:02:03.337742+0000","last_undegraded":"2023-08-11T14:02:03.337742+0000","last_fullsized":"2023-08-11T14:02:03.337742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:37:45.078263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.332002+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:02:03.332002+0000","last_peered":"2023-08-11T14:02:03.332002+0000","last_clean":"2023-08-11T14:02:03.332002+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:02:03.332002+0000","last_undegraded":"2023-08-11T14:02:03.332002+0000","last_fullsized":"2023-08-11T14:02:03.332002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:09.878054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331306+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:02:03.331306+0000","last_peered":"2023-08-11T14:02:03.331306+0000","last_clean":"2023-08-11T14:02:03.331306+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:02:03.331306+0000","last_undegraded":"2023-08-11T14:02:03.331306+0000","last_fullsized":"2023-08-11T14:02:03.331306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:35:53.346832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.329500+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:02:03.329500+0000","last_peered":"2023-08-11T14:02:03.329500+0000","last_clean":"2023-08-11T14:02:03.329500+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:02:03.329500+0000","last_undegraded":"2023-08-11T14:02:03.329500+0000","last_fullsized":"2023-08-11T14:02:03.329500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:42:47.090015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.335469+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:02:03.335469+0000","last_peered":"2023-08-11T14:02:03.335469+0000","last_clean":"2023-08-11T14:02:03.335469+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:02:03.335469+0000","last_undegraded":"2023-08-11T14:02:03.335469+0000","last_fullsized":"2023-08-11T14:02:03.335469+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:55:38.674030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.336925+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:02:03.336925+0000","last_peered":"2023-08-11T14:02:03.336925+0000","last_clean":"2023-08-11T14:02:03.336925+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:02:03.336925+0000","last_undegraded":"2023-08-11T14:02:03.336925+0000","last_fullsized":"2023-08-11T14:02:03.336925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:29:22.915773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.337390+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:02:03.337390+0000","last_peered":"2023-08-11T14:02:03.337390+0000","last_clean":"2023-08-11T14:02:03.337390+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:02:03.337390+0000","last_undegraded":"2023-08-11T14:02:03.337390+0000","last_fullsized":"2023-08-11T14:02:03.337390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:47.323970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.514220+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:02:03.514220+0000","last_peered":"2023-08-11T14:02:03.514220+0000","last_clean":"2023-08-11T14:02:03.514220+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:02:03.514220+0000","last_undegraded":"2023-08-11T14:02:03.514220+0000","last_fullsized":"2023-08-11T14:02:03.514220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:16:06.901263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.331590+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:02:03.331590+0000","last_peered":"2023-08-11T14:02:03.331590+0000","last_clean":"2023-08-11T14:02:03.331590+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:02:03.331590+0000","last_undegraded":"2023-08-11T14:02:03.331590+0000","last_fullsized":"2023-08-11T14:02:03.331590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:55.654954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-08-11T14:02:03.330961+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:02:03.330961+0000","last_peered":"2023-08-11T14:02:03.330961+0000","last_clean":"2023-08-11T14:02:03.330961+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:02:03.330961+0000","last_undegraded":"2023-08-11T14:02:03.330961+0000","last_fullsized":"2023-08-11T14:02:03.330961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:29.354569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67253,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":617,"num_read_kb":381,"num_write":238,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":490140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.56999999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.88,"15min":0.88},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":1.0089999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.296,"15min":1.296},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":1.208,"15min":1.208},"last":0.83699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.048,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.876,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.81799999999999995}]},{"osd":8,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":1.056}]},{"osd":9,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0109999999999999,"5min":1.27,"15min":1.27},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0720000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.079}]},{"osd":10,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97399999999999998,"5min":1.139,"15min":1.139},"last":0.95699999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678049,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1137719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.52200000000000002,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.194,"5min":1.526,"15min":1.526},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.307,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.0309999999999999,"5min":1.516,"15min":1.516},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.147,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80100000000000005,"5min":1.446,"15min":1.446},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81299999999999994,"5min":1.571,"15min":1.571},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88800000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":1.133,"15min":1.133},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.91900000000000004,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.01,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.091,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2,"5min":1.605,"15min":1.605},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.62,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.873,"15min":0.89300000000000002},"min":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1100000000000001,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.77900000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.873,"15min":0.88700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1719999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.83699999999999997,"15min":0.85299999999999998},"min":{"1min":0.54600000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99299999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.73199999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.59899999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.748}]}]},{"osd":9,"up_from":66,"seq":283467841575,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":494098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.505},{"interface":"front","average":{"1min":0.495,"5min":0.621,"15min":0.64200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1,"5min":1,"15min":1},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97599999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.79600000000000004,"15min":0.81399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.998,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.62}]},{"osd":3,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.73099999999999998,"15min":0.75700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93799999999999994,"5min":1.123,"15min":1.123},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.72899999999999998,"15min":0.755},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.876,"5min":1.165,"15min":1.165},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.878,"15min":0.90700000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.075,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92300000000000004,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.93400000000000005,"15min":0.96499999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0289999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.93100000000000005,"15min":0.96399999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1,"15min":1.038},"min":{"1min":0.54600000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1679999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":1.006,"15min":1.042},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1240000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.98499999999999999,"15min":1.014},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1759999999999999,"5min":1.484,"15min":1.484},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82799999999999996,"5min":1.0249999999999999,"15min":1.0580000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.103,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.78600000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.94499999999999995,"15min":0.96699999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1279999999999999,"5min":1.51,"15min":1.51},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.99199999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1479999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.76000000000000001}]},{"osd":11,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.80100000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.504,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47099999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.59299999999999997,"15min":0.57199999999999995},"min":{"1min":0.46899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.41399999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.45400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.51000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.748},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.61299999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.73399999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":1.022,"15min":1.022},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.56399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.58299999999999996,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59999999999999998,"15min":0.59099999999999997},"min":{"1min":0.53700000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.86399999999999999,"5min":1.101,"15min":1.101},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.76700000000000002,"15min":0.75},"min":{"1min":0.74099999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.089,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.69399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0349999999999999,"5min":1.042,"15min":1.042},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.70099999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.083,"15min":1.083},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.749,"15min":0.74099999999999999},"min":{"1min":0.66200000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96999999999999997,"5min":1.29,"15min":1.29},"last":0.88900000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79400000000000004,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.87,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.872},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.86099999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1086603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68799999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64300000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.75}]},{"osd":3,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.442,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83499999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52400000000000002,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71699999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.35099999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.51100000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":1.034,"15min":1.034},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71999999999999997,"15min":0.69799999999999995},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81699999999999995}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.74399999999999999},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.92400000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":0.95799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.92000000000000004,"5min":1.143,"15min":1.143},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92800000000000005,"5min":1.173,"15min":1.173},"last":0.85499999999999998}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.48099999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":1.095,"15min":1.095},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.57199999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.91400000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70599999999999996}]},{"osd":11,"last update":"Fri Aug 11 14:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.872},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.78700000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528373,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":485506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88,"5min":1.107,"15min":1.107},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.087,"5min":1.131,"15min":1.131},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.754}]},{"osd":4,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.82299999999999995,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.61899999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":1.081,"15min":1.081},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93200000000000005,"5min":1.198,"15min":1.198},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.97399999999999998,"5min":1.032,"15min":1.032},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86799999999999999,"15min":0.86599999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80100000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.83999999999999997,"15min":0.879},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98799999999999999,"5min":1.663,"15min":1.663},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81899999999999995,"15min":0.83199999999999996},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.151,"5min":1.252,"15min":1.252},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Aug 11 14:02:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.2130000000000001,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90900000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.89700000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84699999999999998,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.92500000000000004,"15min":0.92700000000000005},"min":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.212,"5min":1.363,"15min":1.363},"last":0.77600000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.72399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":546704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":1.054,"15min":1.054},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90800000000000003,"5min":1.234,"15min":1.234},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.46300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.48799999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:02:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.51300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.60599999999999998},"min":{"1min":0.55800000000000005,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Aug 11 14:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.44900000000000001,"15min":0.40500000000000003},"min":{"1min":0.35399999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.44400000000000001,"15min":0.42199999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.69299999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.441}]},{"osd":5,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.61199999999999999,"15min":0.56100000000000005},"min":{"1min":0.58199999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.64900000000000002,"15min":0.58499999999999996},"min":{"1min":0.57199999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66000000000000003,"15min":0.60899999999999999},"min":{"1min":0.46500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.878,"5min":3.153,"15min":3.153},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.44500000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97099999999999997,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Aug 11 14:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.58199999999999996,"15min":0.53900000000000003},"min":{"1min":0.504,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.623,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.177,"5min":1.345,"15min":1.345},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.69299999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0940000000000001,"5min":1.302,"15min":1.302},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.73799999999999999,"15min":0.70199999999999996},"min":{"1min":0.53200000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.66700000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:02:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.88900000000000001,"15min":0.90200000000000002},"min":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95599999999999996,"5min":3.194,"15min":3.194},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.2370000000000001,"5min":1.393,"15min":1.393},"last":0.82799999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.84699999999999998},"min":{"1min":0.66500000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2250000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0569999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77700000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.81200000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":486962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.61399999999999999,"15min":0.57999999999999996},"min":{"1min":0.45300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.434,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.64000000000000001,"15min":0.59199999999999997},"min":{"1min":0.38200000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.052,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.71899999999999997,"15min":0.63700000000000001},"min":{"1min":0.46700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":6.2910000000000004,"5min":6.2910000000000004,"15min":6.2910000000000004},"last":0.94199999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.3279999999999998,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.622}]},{"osd":4,"last update":"Fri Aug 11 14:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.48899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95899999999999996,"5min":1.097,"15min":1.097},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63100000000000001,"15min":0.58699999999999997},"min":{"1min":0.501,"5min":0.317,"15min":0.317},"max":{"1min":0.83499999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:02:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.76700000000000002,"15min":0.71699999999999997},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.44},"max":{"1min":6.25,"5min":6.25,"15min":6.25},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.71099999999999997,"15min":0.65200000000000002},"min":{"1min":0.57899999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Aug 11 14:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.52300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79000000000000004,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":0.48999999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.254,"5min":3.254,"15min":3.254},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.77400000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.84199999999999997,"15min":0.84099999999999997},"min":{"1min":0.69399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.224,"5min":0.90400000000000003,"15min":0.85799999999999998},"min":{"1min":0.68100000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.3339999999999996,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90100000000000002,"15min":0.91400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.018,"5min":1.274,"15min":1.274},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":6.2060000000000004,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.91300000000000003,"15min":0.89600000000000002},"min":{"1min":0.65800000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.745}]},{"osd":11,"last update":"Fri Aug 11 14:02:47 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.878,"15min":0.85999999999999999},"min":{"1min":0.747,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.871},{"interface":"front","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":6.3739999999999997,"5min":6.3739999999999997,"15min":6.3739999999999997},"last":0.89700000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":547129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.879,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55000000000000004,"15min":0.52300000000000002},"min":{"1min":0.42599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:02:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.41699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.57399999999999995,"15min":0.52300000000000002},"min":{"1min":0.40100000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.624}]},{"osd":3,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.55400000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.52500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.751}]},{"osd":5,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46700000000000003,"15min":0.40899999999999997},"min":{"1min":0.36499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.80500000000000005,"5min":3.218,"15min":3.218},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.47099999999999997,"15min":0.42899999999999999},"min":{"1min":0.39200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Aug 11 14:02:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:02:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.61399999999999999,"15min":0.57499999999999996},"min":{"1min":0.46600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.014,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.64700000000000002,"15min":0.60699999999999998},"min":{"1min":0.52300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0740000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.77200000000000002}]},{"osd":9,"last update":"Fri Aug 11 14:02:06 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.57999999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.59199999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.052,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.60599999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80500000000000005}]},{"osd":11,"last update":"Fri Aug 11 14:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84299999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":485214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.42199999999999999},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.628}]},{"osd":1,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.68600000000000005,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.423,"5min":6.423,"15min":6.423},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.61299999999999999},"min":{"1min":0.57199999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":2.548,"15min":2.548},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.72499999999999998,"15min":0.65500000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":6.5010000000000003,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:02:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.61899999999999999},"min":{"1min":0.51500000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85299999999999998,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.999}]},{"osd":5,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.66300000000000003},"min":{"1min":0.56299999999999994,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.363,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72999999999999998,"15min":0.69599999999999995},"min":{"1min":0.54700000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70899999999999996,"15min":0.67400000000000004},"min":{"1min":0.5,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0469999999999999,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":1.109},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95699999999999996,"5min":1.302,"15min":1.302},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81100000000000005,"15min":0.80600000000000005},"min":{"1min":0.626,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96699999999999997,"5min":1.165,"15min":1.165},"last":1.153},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.032,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:02:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.97699999999999998,"15min":1.0009999999999999},"min":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.0409999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0649999999999999,"5min":1.891,"15min":1.891},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Aug 11 14:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1379999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.93000000000000005},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.1879999999999999}]},{"osd":11,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.254},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86699999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":489970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.63200000000000001,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.40500000000000003,"15min":0.379},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59999999999999998,"5min":0.875,"15min":0.875},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57399999999999995,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Aug 11 14:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79000000000000004,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.49399999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:02:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91900000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Aug 11 14:02:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56999999999999995,"15min":0.55200000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61599999999999999,"15min":0.60099999999999998},"min":{"1min":0.41099999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.628}]},{"osd":8,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91300000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.73499999999999999,"15min":0.71899999999999997},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95599999999999996,"5min":1.02,"15min":1.02},"last":0.876}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.753,"15min":0.747},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76600000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80400000000000005,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99399999999999999,"5min":1.075,"15min":1.075},"last":0.90400000000000003}]},{"osd":11,"last update":"Fri Aug 11 14:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.83899999999999997}]}]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1082068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":1.46,"15min":1.46},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.33800000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72699999999999998,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.38900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78500000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.436,"5min":0.318,"15min":0.318},"max":{"1min":0.81399999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.40899999999999997,"15min":0.39500000000000002},"min":{"1min":0.317,"5min":0.251,"15min":0.251},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.41899999999999998},"min":{"1min":0.33900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68500000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":1.409,"15min":1.409},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.63400000000000001,"15min":0.59399999999999997},"min":{"1min":0.46800000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.4900000000000002,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Aug 11 14:02:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.60599999999999998,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:01:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Aug 11 14:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.54800000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90400000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:02:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Aug 11 14:02:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.75,"15min":0.749},"min":{"1min":0.50900000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83099999999999996}]},{"osd":10,"last update":"Fri Aug 11 14:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.55800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.78300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:03:03.112 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-08-11T14:03:03.112 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-11T14:03:03.112 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-08-11T14:03:03.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph health --format=json 2023-08-11T14:03:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:03 smithi003 ceph-mon[97195]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:03 smithi003 ceph-mon[97195]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T14:03:03.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-08-11T14:03:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:03 smithi103 ceph-mon[130434]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:03 smithi103 ceph-mon[130434]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T14:03:03.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-08-11T14:03:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:03 smithi187 ceph-mon[130973]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:03 smithi187 ceph-mon[130973]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-08-11T14:03:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-08-11T14:03:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:04.435 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:04 smithi003 ceph-mon[97195]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:04.436 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:04 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-08-11T14:03:04.436 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:04 smithi003 ceph-mon[97195]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T14:03:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:04 smithi103 ceph-mon[130434]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:04 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-08-11T14:03:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:04 smithi103 ceph-mon[130434]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T14:03:04.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:04 smithi187 ceph-mon[130973]: from='client.25069 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:04.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:04 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-08-11T14:03:04.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:04 smithi187 ceph-mon[130973]: osdmap e90: 12 total, 12 up, 12 in 2023-08-11T14:03:05.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 -- 172.21.15.3:0/3602061172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 msgr2=0x7f988c100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 --2- 172.21.15.3:0/3602061172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c100dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9874009990 tx=0x7f9874009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 -- 172.21.15.3:0/3602061172 shutdown_connections 2023-08-11T14:03:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 --2- 172.21.15.3:0/3602061172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 0x7f988c10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 --2- 172.21.15.3:0/3602061172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 0x7f988c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.459+0000 7f989104b700 1 --2- 172.21.15.3:0/3602061172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c100dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.460+0000 7f989104b700 1 -- 172.21.15.3:0/3602061172 >> 172.21.15.3:0/3602061172 conn(0x7f988c0fc410 msgr2=0x7f988c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.460+0000 7f989104b700 1 -- 172.21.15.3:0/3602061172 shutdown_connections 2023-08-11T14:03:05.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.460+0000 7f989104b700 1 -- 172.21.15.3:0/3602061172 wait complete. 2023-08-11T14:03:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 Processor -- start 2023-08-11T14:03:05.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 -- start start 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 0x7f988c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 0x7f988c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c196ec0 con 0x7f988c100940 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c197030 con 0x7f988c101300 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f989104b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c1971f0 con 0x7f988c106940 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f9882d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:05.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.461+0000 7f9882d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36000/0 (socket says 172.21.15.3:36000) 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 -- 172.21.15.3:0/361771620 learned_addr learned my addr 172.21.15.3:0/361771620 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f988ad9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 0x7f988c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f988b59e700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 0x7f988c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 -- 172.21.15.3:0/361771620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 msgr2=0x7f988c196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 0x7f988c196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 -- 172.21.15.3:0/361771620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 msgr2=0x7f988c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:05.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 0x7f988c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 -- 172.21.15.3:0/361771620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740095f0 con 0x7f988c100940 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9882d9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e9a0 tx=0x7f987c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.462+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c009790 con 0x7f988c100940 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.463+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987c010430 con 0x7f988c100940 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.463+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c1974d0 con 0x7f988c100940 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.463+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c018680 con 0x7f988c100940 2023-08-11T14:03:05.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.463+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c1a37c0 con 0x7f988c100940 2023-08-11T14:03:05.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.464+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c04e620 con 0x7f988c100940 2023-08-11T14:03:05.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.466+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f987c0105a0 con 0x7f988c100940 2023-08-11T14:03:05.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.468+0000 7f9888d99700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 0x7f98780a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:05.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.468+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f987c014070 con 0x7f988c100940 2023-08-11T14:03:05.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.468+0000 7f988ad9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 0x7f98780a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:05.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.469+0000 7f988ad9d700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 0x7f98780a2410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f987400b6f0 tx=0x7f987400b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:05.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.471+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f987c051c30 con 0x7f988c100940 2023-08-11T14:03:05.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:05 smithi003 ceph-mon[97195]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2023-08-11T14:03:05.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:05 smithi003 ceph-mon[97195]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T14:03:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:05 smithi103 ceph-mon[130434]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2023-08-11T14:03:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:05 smithi103 ceph-mon[130434]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T14:03:05.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.726+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f988c110a40 con 0x7f988c100940 2023-08-11T14:03:05.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.727+0000 7f9888d99700 1 -- 172.21.15.3:0/361771620 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f987c0950f0 con 0x7f988c100940 2023-08-11T14:03:05.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:05.729 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-11T14:03:05.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 msgr2=0x7f98780a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:05.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 0x7f98780a2410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f987400b6f0 tx=0x7f987400b280 comp rx=0 tx=0).stop 2023-08-11T14:03:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 msgr2=0x7f988c19c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f987c00e9a0 tx=0x7f987c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 shutdown_connections 2023-08-11T14:03:05.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f988c106940 0x7f988c196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f988c100940 0x7f988c19c7d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f987809ff50 0x7f98780a2410 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 --2- 172.21.15.3:0/361771620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f988c101300 0x7f988c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 >> 172.21.15.3:0/361771620 conn(0x7f988c0fc410 msgr2=0x7f988c0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.729+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 shutdown_connections 2023-08-11T14:03:05.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:05.730+0000 7f989104b700 1 -- 172.21.15.3:0/361771620 wait complete. 2023-08-11T14:03:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:05 smithi187 ceph-mon[130973]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2023-08-11T14:03:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:05 smithi187 ceph-mon[130973]: osdmap e91: 12 total, 12 up, 12 in 2023-08-11T14:03:06.279 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-08-11T14:03:06.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:06.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 -- 172.21.15.3:0/2807460373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41087e0 msgr2=0x7f49f410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 --2- 172.21.15.3:0/2807460373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41087e0 0x7f49f410ac70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00b0a0 tx=0x7f49ec00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 -- 172.21.15.3:0/2807460373 shutdown_connections 2023-08-11T14:03:06.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 --2- 172.21.15.3:0/2807460373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41087e0 0x7f49f410ac70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 --2- 172.21.15.3:0/2807460373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4074960 0x7f49f4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 --2- 172.21.15.3:0/2807460373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4073380 0x7f49f4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 -- 172.21.15.3:0/2807460373 >> 172.21.15.3:0/2807460373 conn(0x7f49f40fbe50 msgr2=0x7f49f40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:06.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 -- 172.21.15.3:0/2807460373 shutdown_connections 2023-08-11T14:03:06.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.491+0000 7f49fbbff700 1 -- 172.21.15.3:0/2807460373 wait complete. 2023-08-11T14:03:06.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.493+0000 7f49fbbff700 1 Processor -- start 2023-08-11T14:03:06.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.493+0000 7f49fbbff700 1 -- start start 2023-08-11T14:03:06.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.493+0000 7f49fbbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 0x7f49f41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.493+0000 7f49fbbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 0x7f49f4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.493+0000 7f49fbbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fbbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41a0450 con 0x7f49f41087e0 2023-08-11T14:03:06.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fbbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41a0590 con 0x7f49f4074960 2023-08-11T14:03:06.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fbbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41a0720 con 0x7f49f4073380 2023-08-11T14:03:06.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49f999b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 0x7f49f41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36026/0 (socket says 172.21.15.3:36026) 2023-08-11T14:03:06.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 -- 172.21.15.3:0/861604611 learned_addr learned my addr 172.21.15.3:0/861604611 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49f919a700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 0x7f49f4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 -- 172.21.15.3:0/861604611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 msgr2=0x7f49f41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 0x7f49f41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 -- 172.21.15.3:0/861604611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 msgr2=0x7f49f4198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 0x7f49f4198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 -- 172.21.15.3:0/861604611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec009d00 con 0x7f49f41087e0 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49f999b700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 0x7f49f41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49f919a700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 0x7f49f4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.494+0000 7f49fa19c700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fd0 tx=0x7f49ec00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.495+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00e040 con 0x7f49f41087e0 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.495+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f41a09a0 con 0x7f49f41087e0 2023-08-11T14:03:06.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.495+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ec0094a0 con 0x7f49f41087e0 2023-08-11T14:03:06.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.495+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec01b6a0 con 0x7f49f41087e0 2023-08-11T14:03:06.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.495+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f41a0dd0 con 0x7f49f41087e0 2023-08-11T14:03:06.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.496+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f4106e90 con 0x7f49f41087e0 2023-08-11T14:03:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.498+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f49ec01d470 con 0x7f49f41087e0 2023-08-11T14:03:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.499+0000 7f49eaffd700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 0x7f49e00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.499+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f49ec0cc700 con 0x7f49f41087e0 2023-08-11T14:03:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.499+0000 7f49f999b700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 0x7f49e00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.500+0000 7f49f999b700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 0x7f49e00a2460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49f0011490 tx=0x7f49f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:06.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.503+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f49ec0961a0 con 0x7f49f41087e0 2023-08-11T14:03:06.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:06 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/361771620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T14:03:06.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:06 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/361771620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T14:03:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.714+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49f410e360 con 0x7f49f41087e0 2023-08-11T14:03:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.716+0000 7f49eaffd700 1 -- 172.21.15.3:0/861604611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f49ec0997c0 con 0x7f49f41087e0 2023-08-11T14:03:06.719 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:06.719 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 msgr2=0x7f49e00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 0x7f49e00a2460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f49f0011490 tx=0x7f49f0009260 comp rx=0 tx=0).stop 2023-08-11T14:03:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 msgr2=0x7f49f419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009fd0 tx=0x7f49ec00bce0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 shutdown_connections 2023-08-11T14:03:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41087e0 0x7f49f419fe80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4073380 0x7f49f41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ffa0 0x7f49e00a2460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 --2- 172.21.15.3:0/861604611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4074960 0x7f49f4198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 >> 172.21.15.3:0/861604611 conn(0x7f49f40fbe50 msgr2=0x7f49f40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 shutdown_connections 2023-08-11T14:03:06.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.718+0000 7f49fbbff700 1 -- 172.21.15.3:0/861604611 wait complete. 2023-08-11T14:03:06.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:06.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:06.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:06 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/361771620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-08-11T14:03:06.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 -- 172.21.15.3:0/489383840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 msgr2=0x7fbfcc108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/489383840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc108a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009f90 tx=0x7fbfc400b330 comp rx=0 tx=0).stop 2023-08-11T14:03:06.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 -- 172.21.15.3:0/489383840 shutdown_connections 2023-08-11T14:03:06.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/489383840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc108a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/489383840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc101390 0x7fbfcc103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/489383840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc1060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 -- 172.21.15.3:0/489383840 >> 172.21.15.3:0/489383840 conn(0x7fbfcc0fad80 msgr2=0x7fbfcc0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:06.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 -- 172.21.15.3:0/489383840 shutdown_connections 2023-08-11T14:03:06.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.945+0000 7fbfd21ff700 1 -- 172.21.15.3:0/489383840 wait complete. 2023-08-11T14:03:06.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.946+0000 7fbfd21ff700 1 Processor -- start 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 -- start start 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc101390 0x7fbfcc1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19e240 con 0x7fbfcc103cc0 2023-08-11T14:03:06.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19e380 con 0x7fbfcc1065f0 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfd21ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc19e510 con 0x7fbfcc101390 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfcaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfcaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36046/0 (socket says 172.21.15.3:36046) 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfcaffd700 1 -- 172.21.15.3:0/1264936480 learned_addr learned my addr 172.21.15.3:0/1264936480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfcb7fe700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc101390 0x7fbfcc1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.947+0000 7fbfcbfff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 -- 172.21.15.3:0/1264936480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc101390 msgr2=0x7fbfcc1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc101390 0x7fbfcc1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 -- 172.21.15.3:0/1264936480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 msgr2=0x7fbfcc19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 -- 172.21.15.3:0/1264936480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009c40 con 0x7fbfcc103cc0 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfcaffd700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e9a0 tx=0x7fbfc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0009790 con 0x7fbfcc103cc0 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc19e7f0 con 0x7fbfcc103cc0 2023-08-11T14:03:06.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc000ce80 con 0x7fbfcc103cc0 2023-08-11T14:03:06.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0018680 con 0x7fbfcc103cc0 2023-08-11T14:03:06.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.948+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc19ed50 con 0x7fbfcc103cc0 2023-08-11T14:03:06.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.949+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfcc0fc900 con 0x7fbfcc103cc0 2023-08-11T14:03:06.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.951+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbfc0010430 con 0x7fbfcc103cc0 2023-08-11T14:03:06.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.953+0000 7fbfc8ff9700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 0x7fbfb40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:06.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.953+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbfc0014070 con 0x7fbfcc103cc0 2023-08-11T14:03:06.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.953+0000 7fbfcb7fe700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 0x7fbfb40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:06.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.954+0000 7fbfcb7fe700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 0x7fbfb40a2410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005fd0 tx=0x7fbfbc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:06.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:06.955+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbfc0095540 con 0x7fbfcc103cc0 2023-08-11T14:03:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.166+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfcc10c0f0 con 0x7fbfcc103cc0 2023-08-11T14:03:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.167+0000 7fbfc8ff9700 1 -- 172.21.15.3:0/1264936480 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfc00260a0 con 0x7fbfcc103cc0 2023-08-11T14:03:07.169 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 msgr2=0x7fbfb40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 0x7fbfb40a2410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005fd0 tx=0x7fbfbc009310 comp rx=0 tx=0).stop 2023-08-11T14:03:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 msgr2=0x7fbfcc196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000e9a0 tx=0x7fbfc000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 shutdown_connections 2023-08-11T14:03:07.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbfb409ff50 0x7fbfb40a2410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfcc1065f0 0x7fbfcc19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfcc101390 0x7fbfcc1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 --2- 172.21.15.3:0/1264936480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfcc103cc0 0x7fbfcc196c00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 >> 172.21.15.3:0/1264936480 conn(0x7fbfcc0fad80 msgr2=0x7fbfcc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:07.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 shutdown_connections 2023-08-11T14:03:07.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.169+0000 7fbfd21ff700 1 -- 172.21.15.3:0/1264936480 wait complete. 2023-08-11T14:03:07.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:07.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 -- 172.21.15.3:0/3422122328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c1024c0 msgr2=0x7fa30c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 --2- 172.21.15.3:0/3422122328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c1024c0 0x7fa30c108d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b0d0 tx=0x7fa30400b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 -- 172.21.15.3:0/3422122328 shutdown_connections 2023-08-11T14:03:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 --2- 172.21.15.3:0/3422122328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c1024c0 0x7fa30c108d70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 --2- 172.21.15.3:0/3422122328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c100930 0x7fa30c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 --2- 172.21.15.3:0/3422122328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 0x7fa30c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 -- 172.21.15.3:0/3422122328 >> 172.21.15.3:0/3422122328 conn(0x7fa30c0fbea0 msgr2=0x7fa30c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 -- 172.21.15.3:0/3422122328 shutdown_connections 2023-08-11T14:03:07.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.396+0000 7fa312fff700 1 -- 172.21.15.3:0/3422122328 wait complete. 2023-08-11T14:03:07.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.397+0000 7fa312fff700 1 Processor -- start 2023-08-11T14:03:07.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.397+0000 7fa312fff700 1 -- start start 2023-08-11T14:03:07.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 0x7fa30c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c1024c0 0x7fa30c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1a0490 con 0x7fa30c100930 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1a05d0 con 0x7fa30c101b20 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa312fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1a0760 con 0x7fa30c1024c0 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36074/0 (socket says 172.21.15.3:36074) 2023-08-11T14:03:07.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 -- 172.21.15.3:0/1515318687 learned_addr learned my addr 172.21.15.3:0/1515318687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa31159c700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c1024c0 0x7fa30c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa30bfff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 0x7fa30c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 -- 172.21.15.3:0/1515318687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c1024c0 msgr2=0x7fa30c19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c1024c0 0x7fa30c19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 -- 172.21.15.3:0/1515318687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 msgr2=0x7fa30c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 0x7fa30c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.398+0000 7fa310d9b700 1 -- 172.21.15.3:0/1515318687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009d00 con 0x7fa30c100930 2023-08-11T14:03:07.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa310d9b700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d980 tx=0x7fa30000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300004020 con 0x7fa30c100930 2023-08-11T14:03:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30000f690 con 0x7fa30c100930 2023-08-11T14:03:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c1a0a40 con 0x7fa30c100930 2023-08-11T14:03:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300003a60 con 0x7fa30c100930 2023-08-11T14:03:07.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.399+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c1a0fa0 con 0x7fa30c100930 2023-08-11T14:03:07.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.400+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c10c1a0 con 0x7fa30c100930 2023-08-11T14:03:07.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.402+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa30000fb00 con 0x7fa30c100930 2023-08-11T14:03:07.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.405+0000 7fa309ffb700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 0x7fa2fc0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.405+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa3000cb0f0 con 0x7fa30c100930 2023-08-11T14:03:07.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.405+0000 7fa30bfff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 0x7fa2fc0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.405+0000 7fa30bfff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 0x7fa2fc0a2490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007be0 tx=0x7fa2f80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:07.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.409+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa300094b00 con 0x7fa30c100930 2023-08-11T14:03:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:07 smithi003 ceph-mon[97195]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:07.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:07 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/861604611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:07.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:07 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1264936480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:07.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.621+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa30c1a0bd0 con 0x7fa30c100930 2023-08-11T14:03:07.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.622+0000 7fa309ffb700 1 -- 172.21.15.3:0/1515318687 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa30c1a0bd0 con 0x7fa30c100930 2023-08-11T14:03:07.624 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:07.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 msgr2=0x7fa2fc0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 0x7fa2fc0a2490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007be0 tx=0x7fa2f80073d0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 msgr2=0x7fa30c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d980 tx=0x7fa30000dc90 comp rx=0 tx=0).stop 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 shutdown_connections 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa30c1024c0 0x7fa30c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa30c100930 0x7fa30c198910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa2fc09ffd0 0x7fa2fc0a2490 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 --2- 172.21.15.3:0/1515318687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa30c101b20 0x7fa30c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 >> 172.21.15.3:0/1515318687 conn(0x7fa30c0fbea0 msgr2=0x7fa30c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 shutdown_connections 2023-08-11T14:03:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.624+0000 7fa312fff700 1 -- 172.21.15.3:0/1515318687 wait complete. 2023-08-11T14:03:07.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:07 smithi103 ceph-mon[130434]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:07 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/861604611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:07 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1264936480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:07.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:07 smithi187 ceph-mon[130973]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:07.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:07 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/861604611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:07.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:07 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1264936480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:07.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 -- 172.21.15.3:0/355432514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab741024a0 msgr2=0x7fab74108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/355432514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab741024a0 0x7fab74108d50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b0d0 tx=0x7fab6c00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 -- 172.21.15.3:0/355432514 shutdown_connections 2023-08-11T14:03:07.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/355432514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab741024a0 0x7fab74108d50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/355432514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 0x7fab74100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/355432514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab74101b00 0x7fab74101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 -- 172.21.15.3:0/355432514 >> 172.21.15.3:0/355432514 conn(0x7fab740fbe60 msgr2=0x7fab740fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 -- 172.21.15.3:0/355432514 shutdown_connections 2023-08-11T14:03:07.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.850+0000 7fab7c7ff700 1 -- 172.21.15.3:0/355432514 wait complete. 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.851+0000 7fab7c7ff700 1 Processor -- start 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.851+0000 7fab7c7ff700 1 -- start start 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 0x7fab740746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab741024a0 0x7fab74073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740739f0 con 0x7fab74101b00 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74073b60 con 0x7fab741024a0 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7c7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741a0790 con 0x7fab74100910 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36100/0 (socket says 172.21.15.3:36100) 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 -- 172.21.15.3:0/985937146 learned_addr learned my addr 172.21.15.3:0/985937146 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7a59b700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 0x7fab740746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab7ad9c700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab741024a0 0x7fab74073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 -- 172.21.15.3:0/985937146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 msgr2=0x7fab740746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 0x7fab740746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 -- 172.21.15.3:0/985937146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab741024a0 msgr2=0x7fab74073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab741024a0 0x7fab74073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.852+0000 7fab79d9a700 1 -- 172.21.15.3:0/985937146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009d00 con 0x7fab74101b00 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab79d9a700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9a0 tx=0x7fab6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64009790 con 0x7fab74101b00 2023-08-11T14:03:07.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab64010430 con 0x7fab74101b00 2023-08-11T14:03:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab741a0a70 con 0x7fab74101b00 2023-08-11T14:03:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64018680 con 0x7fab74101b00 2023-08-11T14:03:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.853+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab741a0f80 con 0x7fab74101b00 2023-08-11T14:03:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.854+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7410c180 con 0x7fab74101b00 2023-08-11T14:03:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.856+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fab640105a0 con 0x7fab74101b00 2023-08-11T14:03:07.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.858+0000 7fab6b7fe700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 0x7fab600a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:07.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.858+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fab64014070 con 0x7fab74101b00 2023-08-11T14:03:07.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.858+0000 7fab7a59b700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 0x7fab600a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:07.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.859+0000 7fab7a59b700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 0x7fab600a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab74101960 tx=0x7fab70009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:07.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:07.860+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fab64095380 con 0x7fab74101b00 2023-08-11T14:03:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.070+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab741077f0 con 0x7fab74101b00 2023-08-11T14:03:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.070+0000 7fab6b7fe700 1 -- 172.21.15.3:0/985937146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab640260a0 con 0x7fab74101b00 2023-08-11T14:03:08.072 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.072+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 msgr2=0x7fab600a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.072+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 0x7fab600a2410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab74101960 tx=0x7fab70009260 comp rx=0 tx=0).stop 2023-08-11T14:03:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.072+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 msgr2=0x7fab74072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.072+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab6400e9a0 tx=0x7fab6400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 shutdown_connections 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab6009ff50 0x7fab600a2410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab741024a0 0x7fab74073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab74100910 0x7fab740746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 --2- 172.21.15.3:0/985937146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab74101b00 0x7fab74072d10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 >> 172.21.15.3:0/985937146 conn(0x7fab740fbe60 msgr2=0x7fab740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 shutdown_connections 2023-08-11T14:03:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.073+0000 7fab7c7ff700 1 -- 172.21.15.3:0/985937146 wait complete. 2023-08-11T14:03:08.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:08.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 -- 172.21.15.3:0/3949878488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4101b20 msgr2=0x7f49f4101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 --2- 172.21.15.3:0/3949878488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4101b20 0x7f49f4101f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0099c0 tx=0x7f49dc009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 -- 172.21.15.3:0/3949878488 shutdown_connections 2023-08-11T14:03:08.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 --2- 172.21.15.3:0/3949878488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f41024c0 0x7f49f4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 --2- 172.21.15.3:0/3949878488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 --2- 172.21.15.3:0/3949878488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f4101b20 0x7f49f4101f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.298+0000 7f49f8bff700 1 -- 172.21.15.3:0/3949878488 >> 172.21.15.3:0/3949878488 conn(0x7f49f40fbea0 msgr2=0x7f49f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:08.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.299+0000 7f49f8bff700 1 -- 172.21.15.3:0/3949878488 shutdown_connections 2023-08-11T14:03:08.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.299+0000 7f49f8bff700 1 -- 172.21.15.3:0/3949878488 wait complete. 2023-08-11T14:03:08.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.299+0000 7f49f8bff700 1 Processor -- start 2023-08-11T14:03:08.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 -- start start 2023-08-11T14:03:08.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f40746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41024c0 0x7f49f4073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40739f0 con 0x7f49f4101b20 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f4073b60 con 0x7f49f41024c0 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f8bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41a0720 con 0x7f49f4100930 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f40746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f40746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35954/0 (socket says 172.21.15.3:35954) 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36128/0 (socket says 172.21.15.3:36128) 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f259c700 1 -- 172.21.15.3:0/936614983 learned_addr learned my addr 172.21.15.3:0/936614983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.300+0000 7f49f2d9d700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41024c0 0x7f49f4073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 -- 172.21.15.3:0/936614983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 msgr2=0x7f49f40746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f40746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 -- 172.21.15.3:0/936614983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41024c0 msgr2=0x7f49f4073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41024c0 0x7f49f4073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 -- 172.21.15.3:0/936614983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4009520 con 0x7f49f4101b20 2023-08-11T14:03:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49ebfff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009fd0 tx=0x7f49dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc01d070 con 0x7f49f4101b20 2023-08-11T14:03:08.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0095f0 con 0x7f49f4101b20 2023-08-11T14:03:08.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49dc005050 con 0x7f49f4101b20 2023-08-11T14:03:08.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc004a70 con 0x7f49f4101b20 2023-08-11T14:03:08.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.301+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f41a0c90 con 0x7f49f4101b20 2023-08-11T14:03:08.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.303+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f4107f10 con 0x7f49f4101b20 2023-08-11T14:03:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.304+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f49dc005450 con 0x7f49f4101b20 2023-08-11T14:03:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.306+0000 7f49eb7fe700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 0x7f49e00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.306+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f49dc0cc290 con 0x7f49f4101b20 2023-08-11T14:03:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.306+0000 7f49f259c700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 0x7f49e00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.307+0000 7f49f259c700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 0x7f49e00a2410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005950 tx=0x7f49e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:08.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.310+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f49dc095d30 con 0x7f49f4101b20 2023-08-11T14:03:08.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.522+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f49f402cc70 con 0x7f49f4101b20 2023-08-11T14:03:08.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.522+0000 7f49eb7fe700 1 -- 172.21.15.3:0/936614983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f49dc099350 con 0x7f49f4101b20 2023-08-11T14:03:08.524 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 msgr2=0x7f49e00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 0x7f49e00a2410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f49e4005950 tx=0x7f49e4009260 comp rx=0 tx=0).stop 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 msgr2=0x7f49f4072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009fd0 tx=0x7f49dc00f690 comp rx=0 tx=0).stop 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 shutdown_connections 2023-08-11T14:03:08.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f49e009ff50 0x7f49e00a2410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49f41024c0 0x7f49f4073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f4100930 0x7f49f40746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 --2- 172.21.15.3:0/936614983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49f4101b20 0x7f49f4072d10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.524+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 >> 172.21.15.3:0/936614983 conn(0x7f49f40fbea0 msgr2=0x7f49f4105050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.525+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 shutdown_connections 2023-08-11T14:03:08.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.525+0000 7f49f8bff700 1 -- 172.21.15.3:0/936614983 wait complete. 2023-08-11T14:03:08.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:08.606 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:08 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1515318687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:08.607 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:08 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/985937146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:08 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1515318687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:08.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:08 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/985937146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:08.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.748+0000 7f753c9ff700 1 -- 172.21.15.3:0/1271408274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 msgr2=0x7f7538100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.748+0000 7f753c9ff700 1 --2- 172.21.15.3:0/1271408274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f7538100d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75280099c0 tx=0x7f7528009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 -- 172.21.15.3:0/1271408274 shutdown_connections 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 --2- 172.21.15.3:0/1271408274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75381024b0 0x7f7538108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 --2- 172.21.15.3:0/1271408274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f7538100d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 --2- 172.21.15.3:0/1271408274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7538101b10 0x7f7538101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 -- 172.21.15.3:0/1271408274 >> 172.21.15.3:0/1271408274 conn(0x7f75380fbe50 msgr2=0x7f75380fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 -- 172.21.15.3:0/1271408274 shutdown_connections 2023-08-11T14:03:08.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.749+0000 7f753c9ff700 1 -- 172.21.15.3:0/1271408274 wait complete. 2023-08-11T14:03:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 Processor -- start 2023-08-11T14:03:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 -- start start 2023-08-11T14:03:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f75380746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7538101b10 0x7f7538072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380739f0 con 0x7f75381024b0 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7538073b60 con 0x7f7538100920 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f753c9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381a06a0 con 0x7f7538101b10 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f7536d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.750+0000 7f7536d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36144/0 (socket says 172.21.15.3:36144) 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 -- 172.21.15.3:0/2160348030 learned_addr learned my addr 172.21.15.3:0/2160348030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f753659c700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f75380746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7535d9b700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7538101b10 0x7f7538072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 -- 172.21.15.3:0/2160348030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7538101b10 msgr2=0x7f7538072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7538101b10 0x7f7538072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 -- 172.21.15.3:0/2160348030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 msgr2=0x7f75380746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f75380746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 -- 172.21.15.3:0/2160348030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c009520 con 0x7f75381024b0 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f7536d9d700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f753000ef30 tx=0x7f753000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530018070 con 0x7f75381024b0 2023-08-11T14:03:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f753000f040 con 0x7f75381024b0 2023-08-11T14:03:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75280095f0 con 0x7f75381024b0 2023-08-11T14:03:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.751+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753000cd00 con 0x7f75381024b0 2023-08-11T14:03:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.752+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75381a0c50 con 0x7f75381024b0 2023-08-11T14:03:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.753+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753810c180 con 0x7f75381024b0 2023-08-11T14:03:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.755+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7530003a00 con 0x7f75381024b0 2023-08-11T14:03:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.756+0000 7f75277fe700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 0x7f75200a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:08.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.756+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f75300cb110 con 0x7f75381024b0 2023-08-11T14:03:08.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.756+0000 7f753659c700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 0x7f75200a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:08.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.757+0000 7f753659c700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 0x7f75200a2460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:08.760+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7530094bb0 con 0x7f75381024b0 2023-08-11T14:03:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:08 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1515318687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:08 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/985937146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:09.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.016+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f753810c390 con 0x7f75381024b0 2023-08-11T14:03:09.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.017+0000 7f75277fe700 1 -- 172.21.15.3:0/2160348030 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f75300981d0 con 0x7f75381024b0 2023-08-11T14:03:09.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:09.019 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:09.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 msgr2=0x7f75200a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 0x7f75200a2460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 msgr2=0x7f7538073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f753000ef30 tx=0x7f753000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 shutdown_connections 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75381024b0 0x7f7538073250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f752009ffa0 0x7f75200a2460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7538100920 0x7f75380746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 --2- 172.21.15.3:0/2160348030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7538101b10 0x7f7538072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 >> 172.21.15.3:0/2160348030 conn(0x7f75380fbe50 msgr2=0x7f75380fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 shutdown_connections 2023-08-11T14:03:09.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.020+0000 7f753c9ff700 1 -- 172.21.15.3:0/2160348030 wait complete. 2023-08-11T14:03:09.023 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:09.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:09.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 -- 172.21.15.3:0/3162671389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88101b20 msgr2=0x7f5c88101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 --2- 172.21.15.3:0/3162671389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88101b20 0x7f5c88101f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c780098f0 tx=0x7f5c78009c00 comp rx=0 tx=0).stop 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 -- 172.21.15.3:0/3162671389 shutdown_connections 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 --2- 172.21.15.3:0/3162671389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c881024c0 0x7f5c88108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 --2- 172.21.15.3:0/3162671389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c88100930 0x7f5c88100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 --2- 172.21.15.3:0/3162671389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88101b20 0x7f5c88101f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 -- 172.21.15.3:0/3162671389 >> 172.21.15.3:0/3162671389 conn(0x7f5c880fbe60 msgr2=0x7f5c880fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 -- 172.21.15.3:0/3162671389 shutdown_connections 2023-08-11T14:03:09.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.244+0000 7f5c907ff700 1 -- 172.21.15.3:0/3162671389 wait complete. 2023-08-11T14:03:09.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 Processor -- start 2023-08-11T14:03:09.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 -- start start 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 0x7f5c88198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c881024c0 0x7f5c8819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881a0340 con 0x7f5c88101b20 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881a0480 con 0x7f5c881024c0 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.245+0000 7f5c907ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881a0610 con 0x7f5c88100930 2023-08-11T14:03:09.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8dd9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 0x7f5c88198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35982/0 (socket says 172.21.15.3:35982) 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 -- 172.21.15.3:0/1321867036 learned_addr learned my addr 172.21.15.3:0/1321867036 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8ed9c700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c881024c0 0x7f5c8819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 -- 172.21.15.3:0/1321867036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c881024c0 msgr2=0x7f5c8819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c881024c0 0x7f5c8819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 -- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 msgr2=0x7f5c88198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 0x7f5c88198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 -- 172.21.15.3:0/1321867036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840095f0 con 0x7f5c88100930 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8dd9a700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 0x7f5c88198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.246+0000 7f5c8e59b700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ebf0 tx=0x7f5c8400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.247+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84009a10 con 0x7f5c88100930 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.247+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c78009520 con 0x7f5c88100930 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.247+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c8400cc90 con 0x7f5c88100930 2023-08-11T14:03:09.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.247+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84018750 con 0x7f5c88100930 2023-08-11T14:03:09.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.247+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c881a0c70 con 0x7f5c88100930 2023-08-11T14:03:09.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.248+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88192f30 con 0x7f5c88100930 2023-08-11T14:03:09.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.250+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5c840188b0 con 0x7f5c88100930 2023-08-11T14:03:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.251+0000 7f5c7f7fe700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 0x7f5c740a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.251+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5c84014070 con 0x7f5c88100930 2023-08-11T14:03:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.252+0000 7f5c8dd9a700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 0x7f5c740a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.252+0000 7f5c8dd9a700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 0x7f5c740a2460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800fa70 tx=0x7f5c78019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:09.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.253+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5c84095d00 con 0x7f5c88100930 2023-08-11T14:03:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.468+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c8810c140 con 0x7f5c88100930 2023-08-11T14:03:09.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.471+0000 7f5c7f7fe700 1 -- 172.21.15.3:0/1321867036 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f5c84099320 con 0x7f5c88100930 2023-08-11T14:03:09.472 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:09.473 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 msgr2=0x7f5c740a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 0x7f5c740a2460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800fa70 tx=0x7f5c78019040 comp rx=0 tx=0).stop 2023-08-11T14:03:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 msgr2=0x7f5c881987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ebf0 tx=0x7f5c8400ef00 comp rx=0 tx=0).stop 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 shutdown_connections 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5c7409ffa0 0x7f5c740a2460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c881024c0 0x7f5c8819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5c88100930 0x7f5c881987c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 --2- 172.21.15.3:0/1321867036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c88101b20 0x7f5c88198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 >> 172.21.15.3:0/1321867036 conn(0x7f5c880fbe60 msgr2=0x7f5c88107400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 shutdown_connections 2023-08-11T14:03:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.473+0000 7f5c907ff700 1 -- 172.21.15.3:0/1321867036 wait complete. 2023-08-11T14:03:09.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:09.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:09 smithi103 ceph-mon[130434]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:09 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/936614983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:09 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2160348030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:09.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 -- 172.21.15.3:0/1731933835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8100910 msgr2=0x7f32f8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 --2- 172.21.15.3:0/1731933835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8100910 0x7f32f8100d30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0099c0 tx=0x7f32ec009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 -- 172.21.15.3:0/1731933835 shutdown_connections 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 --2- 172.21.15.3:0/1731933835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 --2- 172.21.15.3:0/1731933835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8100910 0x7f32f8100d30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 --2- 172.21.15.3:0/1731933835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8101b00 0x7f32f8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 -- 172.21.15.3:0/1731933835 >> 172.21.15.3:0/1731933835 conn(0x7f32f80fbe60 msgr2=0x7f32f80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 -- 172.21.15.3:0/1731933835 shutdown_connections 2023-08-11T14:03:09.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.700+0000 7f32febff700 1 -- 172.21.15.3:0/1731933835 wait complete. 2023-08-11T14:03:09.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.701+0000 7f32febff700 1 Processor -- start 2023-08-11T14:03:09.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 -- start start 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 0x7f32f8072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80739f0 con 0x7f32f8101b00 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8073b60 con 0x7f32f81024a0 2023-08-11T14:03:09.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.702+0000 7f32febff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81a06e0 con 0x7f32f8100910 2023-08-11T14:03:09.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 0x7f32f8072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36006/0 (socket says 172.21.15.3:36006) 2023-08-11T14:03:09.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 -- 172.21.15.3:0/2847437360 learned_addr learned my addr 172.21.15.3:0/2847437360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:09.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fd19c700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 -- 172.21.15.3:0/2847437360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 msgr2=0x7f32f8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 -- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 msgr2=0x7f32f8072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 0x7f32f8072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 -- 172.21.15.3:0/2847437360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0095f0 con 0x7f32f8100910 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fd19c700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32f7fff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 0x7f32f8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:09.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.703+0000 7f32fc99b700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0098e0 tx=0x7f32ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.704+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01d070 con 0x7f32f8100910 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.704+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f81a0960 con 0x7f32f8100910 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.704+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32ec005050 con 0x7f32f8100910 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.704+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004b40 con 0x7f32f8100910 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.704+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f81a0d80 con 0x7f32f8100910 2023-08-11T14:03:09.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.705+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f8107ef0 con 0x7f32f8100910 2023-08-11T14:03:09.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.707+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f32ec004ce0 con 0x7f32f8100910 2023-08-11T14:03:09.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.708+0000 7f32f5ffb700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 0x7f32e80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:09.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.708+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f32ec0cc380 con 0x7f32f8100910 2023-08-11T14:03:09.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.708+0000 7f32f7fff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 0x7f32e80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:09.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.709+0000 7f32f7fff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 0x7f32e80a2410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005950 tx=0x7f32e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:09.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.710+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f32ec095f40 con 0x7f32f8100910 2023-08-11T14:03:09.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:09 smithi003 ceph-mon[97195]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:09.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:09 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/936614983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:09.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:09 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2160348030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:09.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:09 smithi187 ceph-mon[130973]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:09.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:09 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/936614983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:09.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:09 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2160348030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.915+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32f810c3c0 con 0x7f32f8100910 2023-08-11T14:03:09.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.917+0000 7f32f5ffb700 1 -- 172.21.15.3:0/2847437360 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32ec099560 con 0x7f32f8100910 2023-08-11T14:03:09.918 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.918+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 msgr2=0x7f32e80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.918+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 0x7f32e80a2410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005950 tx=0x7f32e400b410 comp rx=0 tx=0).stop 2023-08-11T14:03:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.918+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 msgr2=0x7f32f80746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.918+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0098e0 tx=0x7f32ec00f690 comp rx=0 tx=0).stop 2023-08-11T14:03:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 shutdown_connections 2023-08-11T14:03:09.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f32e809ff50 0x7f32e80a2410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32f81024a0 0x7f32f8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f8100910 0x7f32f80746d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 --2- 172.21.15.3:0/2847437360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101b00 0x7f32f8072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 >> 172.21.15.3:0/2847437360 conn(0x7f32f80fbe60 msgr2=0x7f32f8105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 shutdown_connections 2023-08-11T14:03:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:09.919+0000 7f32febff700 1 -- 172.21.15.3:0/2847437360 wait complete. 2023-08-11T14:03:09.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:10.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 -- 172.21.15.3:0/3803297762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c101b00 msgr2=0x7f110c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 --2- 172.21.15.3:0/3803297762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c101b00 0x7f110c101f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009920 tx=0x7f10f4009c30 comp rx=0 tx=0).stop 2023-08-11T14:03:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 -- 172.21.15.3:0/3803297762 shutdown_connections 2023-08-11T14:03:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 --2- 172.21.15.3:0/3803297762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 --2- 172.21.15.3:0/3803297762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c100910 0x7f110c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 --2- 172.21.15.3:0/3803297762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c101b00 0x7f110c101f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 -- 172.21.15.3:0/3803297762 >> 172.21.15.3:0/3803297762 conn(0x7f110c0fbe60 msgr2=0x7f110c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 -- 172.21.15.3:0/3803297762 shutdown_connections 2023-08-11T14:03:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.145+0000 7f11109ff700 1 -- 172.21.15.3:0/3803297762 wait complete. 2023-08-11T14:03:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.146+0000 7f11109ff700 1 Processor -- start 2023-08-11T14:03:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.146+0000 7f11109ff700 1 -- start start 2023-08-11T14:03:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.146+0000 7f11109ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 0x7f110c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.146+0000 7f11109ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f11109ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f11109ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1a0410 con 0x7f110c101b00 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f11109ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1a0550 con 0x7f110c100910 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f11109ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1a06e0 con 0x7f110c1024a0 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f1103fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f1103fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36194/0 (socket says 172.21.15.3:36194) 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f1103fff700 1 -- 172.21.15.3:0/3420603953 learned_addr learned my addr 172.21.15.3:0/3420603953 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110a59c700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 0x7f110c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 -- 172.21.15.3:0/3420603953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 msgr2=0x7f110c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 0x7f110c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 -- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 msgr2=0x7f110c198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 -- 172.21.15.3:0/3420603953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4009520 con 0x7f110c1024a0 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110a59c700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 0x7f110c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f1103fff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.147+0000 7f110ad9d700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c19fe40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f110400efc0 tx=0x7f110400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.148+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104018070 con 0x7f110c1024a0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.148+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c1a0960 con 0x7f110c1024a0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.148+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110400f040 con 0x7f110c1024a0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.148+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400cd30 con 0x7f110c1024a0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.148+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c1a0e70 con 0x7f110c1024a0 2023-08-11T14:03:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.149+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c10c120 con 0x7f110c1024a0 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.151+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1104003a00 con 0x7f110c1024a0 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.152+0000 7f11037fe700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 0x7f10f80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.152+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f11040cbac0 con 0x7f110c1024a0 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.153+0000 7f110a59c700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 0x7f10f80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.153+0000 7f110a59c700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 0x7f10f80a2410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f110c101960 tx=0x7f10fc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.154+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1104095560 con 0x7f110c1024a0 2023-08-11T14:03:10.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.366+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f110c1a1060 con 0x7f110c1024a0 2023-08-11T14:03:10.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.367+0000 7f11037fe700 1 -- 172.21.15.3:0/3420603953 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1104098b80 con 0x7f110c1024a0 2023-08-11T14:03:10.369 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:10.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 msgr2=0x7f10f80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 0x7f10f80a2410 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f110c101960 tx=0x7f10fc009310 comp rx=0 tx=0).stop 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 msgr2=0x7f110c19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c19fe40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f110400efc0 tx=0x7f110400c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 shutdown_connections 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110c1024a0 0x7f110c19fe40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f10f809ff50 0x7f10f80a2410 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f110c100910 0x7f110c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 --2- 172.21.15.3:0/3420603953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101b00 0x7f110c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 >> 172.21.15.3:0/3420603953 conn(0x7f110c0fbe60 msgr2=0x7f110c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:10.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 shutdown_connections 2023-08-11T14:03:10.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.370+0000 7f11109ff700 1 -- 172.21.15.3:0/3420603953 wait complete. 2023-08-11T14:03:10.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 -- 172.21.15.3:0/1464432705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4101390 msgr2=0x7ffbd4103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 --2- 172.21.15.3:0/1464432705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4101390 0x7ffbd4103780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009990 tx=0x7ffbc8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 -- 172.21.15.3:0/1464432705 shutdown_connections 2023-08-11T14:03:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 --2- 172.21.15.3:0/1464432705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd41065f0 0x7ffbd4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 --2- 172.21.15.3:0/1464432705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd4101390 0x7ffbd4103780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 --2- 172.21.15.3:0/1464432705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd41060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 -- 172.21.15.3:0/1464432705 >> 172.21.15.3:0/1464432705 conn(0x7ffbd40fad80 msgr2=0x7ffbd40fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:10.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 -- 172.21.15.3:0/1464432705 shutdown_connections 2023-08-11T14:03:10.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.596+0000 7ffbdadff700 1 -- 172.21.15.3:0/1464432705 wait complete. 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.597+0000 7ffbdadff700 1 Processor -- start 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.597+0000 7ffbdadff700 1 -- start start 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd4101390 0x7ffbd4100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd41065f0 0x7ffbd40ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4100020 con 0x7ffbd4103cc0 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4100190 con 0x7ffbd41065f0 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbdadff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41a0670 con 0x7ffbd4101390 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbd3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbd3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36224/0 (socket says 172.21.15.3:36224) 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbd8b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd4101390 0x7ffbd4100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbd3fff700 1 -- 172.21.15.3:0/2379885659 learned_addr learned my addr 172.21.15.3:0/2379885659 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.598+0000 7ffbd939c700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd41065f0 0x7ffbd40ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 -- 172.21.15.3:0/2379885659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd4101390 msgr2=0x7ffbd4100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd4101390 0x7ffbd4100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 -- 172.21.15.3:0/2379885659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd41065f0 msgr2=0x7ffbd40ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd41065f0 0x7ffbd40ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 -- 172.21.15.3:0/2379885659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80095f0 con 0x7ffbd4103cc0 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd3fff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000e9a0 tx=0x7ffbc000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0009790 con 0x7ffbd4103cc0 2023-08-11T14:03:10.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd41a0950 con 0x7ffbd4103cc0 2023-08-11T14:03:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbc0010430 con 0x7ffbd4103cc0 2023-08-11T14:03:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0018680 con 0x7ffbd4103cc0 2023-08-11T14:03:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.599+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd41a0eb0 con 0x7ffbd4103cc0 2023-08-11T14:03:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.600+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd40fc900 con 0x7ffbd4103cc0 2023-08-11T14:03:10.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.602+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ffbc00105a0 con 0x7ffbd4103cc0 2023-08-11T14:03:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.603+0000 7ffbd1ffb700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 0x7ffbc40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.604+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffbc0014070 con 0x7ffbd4103cc0 2023-08-11T14:03:10.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.604+0000 7ffbd8b9b700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 0x7ffbc40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:10.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.604+0000 7ffbd8b9b700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 0x7ffbc40a2410 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b2e0 tx=0x7ffbc8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:10.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.607+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffbc0051c30 con 0x7ffbd4103cc0 2023-08-11T14:03:10.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:10 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1321867036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:10.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:10 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2847437360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1321867036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:10 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2847437360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:10.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:10 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1321867036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:10.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:10 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2847437360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.819+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbd410c310 con 0x7ffbd4103cc0 2023-08-11T14:03:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.820+0000 7ffbd1ffb700 1 -- 172.21.15.3:0/2379885659 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbc0095180 con 0x7ffbd4103cc0 2023-08-11T14:03:10.822 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.822+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 msgr2=0x7ffbc40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 0x7ffbc40a2410 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b2e0 tx=0x7ffbc8005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 msgr2=0x7ffbd40ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000e9a0 tx=0x7ffbc000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 shutdown_connections 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffbc409ff50 0x7ffbc40a2410 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd41065f0 0x7ffbd40ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbd4101390 0x7ffbd4100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 --2- 172.21.15.3:0/2379885659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd4103cc0 0x7ffbd40ff340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 >> 172.21.15.3:0/2379885659 conn(0x7ffbd40fad80 msgr2=0x7ffbd4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 shutdown_connections 2023-08-11T14:03:10.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:10.823+0000 7ffbdadff700 1 -- 172.21.15.3:0/2379885659 wait complete. 2023-08-11T14:03:10.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 -- 172.21.15.3:0/1578785330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54100910 msgr2=0x7f4d54100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1578785330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54100910 0x7f4d54100d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4d440099c0 tx=0x7f4d44009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 -- 172.21.15.3:0/1578785330 shutdown_connections 2023-08-11T14:03:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1578785330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d54108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1578785330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54100910 0x7f4d54100d30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1578785330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54101b00 0x7f4d54101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 -- 172.21.15.3:0/1578785330 >> 172.21.15.3:0/1578785330 conn(0x7f4d540fbe60 msgr2=0x7f4d540fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 -- 172.21.15.3:0/1578785330 shutdown_connections 2023-08-11T14:03:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.047+0000 7f4d58fff700 1 -- 172.21.15.3:0/1578785330 wait complete. 2023-08-11T14:03:11.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.048+0000 7f4d58fff700 1 Processor -- start 2023-08-11T14:03:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 -- start start 2023-08-11T14:03:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100910 0x7f4d54198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d5419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541a0410 con 0x7f4d54101b00 2023-08-11T14:03:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541a0550 con 0x7f4d541024a0 2023-08-11T14:03:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d58fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541a06e0 con 0x7f4d54100910 2023-08-11T14:03:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d4bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d52d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d5419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d4bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36246/0 (socket says 172.21.15.3:36246) 2023-08-11T14:03:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d4bfff700 1 -- 172.21.15.3:0/1869428160 learned_addr learned my addr 172.21.15.3:0/1869428160 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.049+0000 7f4d5259c700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100910 0x7f4d54198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 -- 172.21.15.3:0/1869428160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100910 msgr2=0x7f4d54198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100910 0x7f4d54198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 -- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 msgr2=0x7f4d54198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 -- 172.21.15.3:0/1869428160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440095f0 con 0x7f4d541024a0 2023-08-11T14:03:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d4bfff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d52d9d700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d5419fe40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00efc0 tx=0x7f4d4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c018070 con 0x7f4d541024a0 2023-08-11T14:03:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d4c00f040 con 0x7f4d541024a0 2023-08-11T14:03:11.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.050+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00cd40 con 0x7f4d541024a0 2023-08-11T14:03:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.051+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d541a0960 con 0x7f4d541024a0 2023-08-11T14:03:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.051+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d541a0e20 con 0x7f4d541024a0 2023-08-11T14:03:11.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.052+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d54192f60 con 0x7f4d541024a0 2023-08-11T14:03:11.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.054+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4d4c003a00 con 0x7f4d541024a0 2023-08-11T14:03:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.055+0000 7f4d4b7fe700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 0x7f4d400a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.055+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4d4c0cbb50 con 0x7f4d541024a0 2023-08-11T14:03:11.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.055+0000 7f4d5259c700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 0x7f4d400a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.056+0000 7f4d5259c700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 0x7f4d400a2410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009990 tx=0x7f4d44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.057+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4d4c0955f0 con 0x7f4d541024a0 2023-08-11T14:03:11.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.266+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4d5410c300 con 0x7f4d541024a0 2023-08-11T14:03:11.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.267+0000 7f4d4b7fe700 1 -- 172.21.15.3:0/1869428160 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d4c098c10 con 0x7f4d541024a0 2023-08-11T14:03:11.268 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 msgr2=0x7f4d400a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 0x7f4d400a2410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009990 tx=0x7f4d44005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 msgr2=0x7f4d5419fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d5419fe40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00efc0 tx=0x7f4d4c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 shutdown_connections 2023-08-11T14:03:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d4009ff50 0x7f4d400a2410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d541024a0 0x7f4d5419fe40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100910 0x7f4d54198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 --2- 172.21.15.3:0/1869428160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d54101b00 0x7f4d54198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.269+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 >> 172.21.15.3:0/1869428160 conn(0x7f4d540fbe60 msgr2=0x7f4d54107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.270+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 shutdown_connections 2023-08-11T14:03:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.270+0000 7f4d58fff700 1 -- 172.21.15.3:0/1869428160 wait complete. 2023-08-11T14:03:11.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 -- 172.21.15.3:0/1284166492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0074960 msgr2=0x7ffab0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 --2- 172.21.15.3:0/1284166492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0074960 0x7ffab0072db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009990 tx=0x7ffaac009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 -- 172.21.15.3:0/1284166492 shutdown_connections 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 --2- 172.21.15.3:0/1284166492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab01087e0 0x7ffab010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 --2- 172.21.15.3:0/1284166492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0074960 0x7ffab0072db0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 --2- 172.21.15.3:0/1284166492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab0073380 0x7ffab0073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.493+0000 7ffab83ff700 1 -- 172.21.15.3:0/1284166492 >> 172.21.15.3:0/1284166492 conn(0x7ffab00fbe50 msgr2=0x7ffab00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.494+0000 7ffab83ff700 1 -- 172.21.15.3:0/1284166492 shutdown_connections 2023-08-11T14:03:11.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.494+0000 7ffab83ff700 1 -- 172.21.15.3:0/1284166492 wait complete. 2023-08-11T14:03:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 Processor -- start 2023-08-11T14:03:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 -- start start 2023-08-11T14:03:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab0074960 0x7ffab0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab01087e0 0x7ffab019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01a03b0 con 0x7ffab0073380 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01a04f0 con 0x7ffab01087e0 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab83ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01a0680 con 0x7ffab0074960 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab619b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab619b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36260/0 (socket says 172.21.15.3:36260) 2023-08-11T14:03:11.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab619b700 1 -- 172.21.15.3:0/250947152 learned_addr learned my addr 172.21.15.3:0/250947152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.495+0000 7ffab599a700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab0074960 0x7ffab0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab699c700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab01087e0 0x7ffab019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 -- 172.21.15.3:0/250947152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab0074960 msgr2=0x7ffab0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab0074960 0x7ffab0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 -- 172.21.15.3:0/250947152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab01087e0 msgr2=0x7ffab019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab01087e0 0x7ffab019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 -- 172.21.15.3:0/250947152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0095f0 con 0x7ffab0073380 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab619b700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b680 tx=0x7ffaac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d070 con 0x7ffab0073380 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac022470 con 0x7ffab0073380 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac017650 con 0x7ffab0073380 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.496+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab01a0900 con 0x7ffab0073380 2023-08-11T14:03:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.497+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab01a0cd0 con 0x7ffab0073380 2023-08-11T14:03:11.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.498+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0106e90 con 0x7ffab0073380 2023-08-11T14:03:11.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.499+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ffaac0225e0 con 0x7ffab0073380 2023-08-11T14:03:11.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.501+0000 7ffaa77fe700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 0x7ffa9c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.501+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffaac0cd0e0 con 0x7ffab0073380 2023-08-11T14:03:11.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.501+0000 7ffab599a700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 0x7ffa9c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.502+0000 7ffab599a700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 0x7ffa9c0a24e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffab0074680 tx=0x7ffaa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.503+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffaac096c00 con 0x7ffab0073380 2023-08-11T14:03:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:11 smithi003 ceph-mon[97195]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 639 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:11 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3420603953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:11 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2379885659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:11 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1869428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:11 smithi103 ceph-mon[130434]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 639 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:11 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3420603953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:11 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2379885659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:11 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1869428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.760+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffab010e360 con 0x7ffab0073380 2023-08-11T14:03:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.761+0000 7ffaa77fe700 1 -- 172.21.15.3:0/250947152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7ffaac09a220 con 0x7ffab0073380 2023-08-11T14:03:11.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:11.762 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 msgr2=0x7ffa9c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 0x7ffa9c0a24e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffab0074680 tx=0x7ffaa0009260 comp rx=0 tx=0).stop 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 msgr2=0x7ffab0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00b680 tx=0x7ffaac005740 comp rx=0 tx=0).stop 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 shutdown_connections 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa9c0a0020 0x7ffa9c0a24e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffab01087e0 0x7ffab019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffab0073380 0x7ffab0198830 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 --2- 172.21.15.3:0/250947152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffab0074960 0x7ffab0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 >> 172.21.15.3:0/250947152 conn(0x7ffab00fbe50 msgr2=0x7ffab00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 shutdown_connections 2023-08-11T14:03:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.763+0000 7ffab83ff700 1 -- 172.21.15.3:0/250947152 wait complete. 2023-08-11T14:03:11.767 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:11.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:11.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:11 smithi187 ceph-mon[130973]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 639 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:11.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:11 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3420603953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:11 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2379885659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:11 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1869428160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:11.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 -- 172.21.15.3:0/242342027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554100920 msgr2=0x7f8554100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 --2- 172.21.15.3:0/242342027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554100920 0x7f8554100d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85440099c0 tx=0x7f8544009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 -- 172.21.15.3:0/242342027 shutdown_connections 2023-08-11T14:03:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 --2- 172.21.15.3:0/242342027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 0x7f8554108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 --2- 172.21.15.3:0/242342027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554100920 0x7f8554100d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 --2- 172.21.15.3:0/242342027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554101b10 0x7f8554101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 -- 172.21.15.3:0/242342027 >> 172.21.15.3:0/242342027 conn(0x7f85540fbe50 msgr2=0x7f85540fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 -- 172.21.15.3:0/242342027 shutdown_connections 2023-08-11T14:03:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.986+0000 7f8559fff700 1 -- 172.21.15.3:0/242342027 wait complete. 2023-08-11T14:03:11.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.987+0000 7f8559fff700 1 Processor -- start 2023-08-11T14:03:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.987+0000 7f8559fff700 1 -- start start 2023-08-11T14:03:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554100920 0x7f85540746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 0x7f8554073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85540739f0 con 0x7f8554101b10 2023-08-11T14:03:11.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554073b60 con 0x7f8554100920 2023-08-11T14:03:11.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8559fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85541a0670 con 0x7f85541024b0 2023-08-11T14:03:11.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45766/0 (socket says 172.21.15.3:45766) 2023-08-11T14:03:11.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 -- 172.21.15.3:0/2808224076 learned_addr learned my addr 172.21.15.3:0/2808224076 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:11.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f85537fe700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554100920 0x7f85540746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8553fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 0x7f8554073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 -- 172.21.15.3:0/2808224076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 msgr2=0x7f8554073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 0x7f8554073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 -- 172.21.15.3:0/2808224076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554100920 msgr2=0x7f85540746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:11.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.988+0000 7f8552ffd700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554100920 0x7f85540746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:11.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8552ffd700 1 -- 172.21.15.3:0/2808224076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440095f0 con 0x7f8554101b10 2023-08-11T14:03:11.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8552ffd700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9a0 tx=0x7f854800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548009790 con 0x7f8554101b10 2023-08-11T14:03:11.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85541a0950 con 0x7f8554101b10 2023-08-11T14:03:11.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8548010430 con 0x7f8554101b10 2023-08-11T14:03:11.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548018680 con 0x7f8554101b10 2023-08-11T14:03:11.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.989+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85541a0e60 con 0x7f8554101b10 2023-08-11T14:03:11.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.990+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855410c180 con 0x7f8554101b10 2023-08-11T14:03:11.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.992+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f85480105a0 con 0x7f8554101b10 2023-08-11T14:03:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.993+0000 7f8550ff9700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 0x7f853c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.994+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8548014070 con 0x7f8554101b10 2023-08-11T14:03:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.994+0000 7f85537fe700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 0x7f853c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.994+0000 7f85537fe700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 0x7f853c0a2460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8544009990 tx=0x7f8544005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:11.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:11.997+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f8548051c30 con 0x7f8554101b10 2023-08-11T14:03:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.211+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f855410c340 con 0x7f8554101b10 2023-08-11T14:03:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.212+0000 7f8550ff9700 1 -- 172.21.15.3:0/2808224076 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f8548095180 con 0x7f8554101b10 2023-08-11T14:03:12.214 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:12.218 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 msgr2=0x7f853c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 0x7f853c0a2460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8544009990 tx=0x7f8544005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 msgr2=0x7f8554072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f854800e9a0 tx=0x7f854800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 shutdown_connections 2023-08-11T14:03:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85541024b0 0x7f8554073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f853c09ffa0 0x7f853c0a2460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8554100920 0x7f85540746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 --2- 172.21.15.3:0/2808224076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8554101b10 0x7f8554072d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 >> 172.21.15.3:0/2808224076 conn(0x7f85540fbe50 msgr2=0x7f85540fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 shutdown_connections 2023-08-11T14:03:12.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.215+0000 7f8559fff700 1 -- 172.21.15.3:0/2808224076 wait complete. 2023-08-11T14:03:12.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 -- 172.21.15.3:0/2388455645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa008100920 msgr2=0x7fa008100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 --2- 172.21.15.3:0/2388455645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa008100920 0x7fa008100d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ff80099c0 tx=0x7f9ff8009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 -- 172.21.15.3:0/2388455645 shutdown_connections 2023-08-11T14:03:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 --2- 172.21.15.3:0/2388455645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0081024b0 0x7fa008108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 --2- 172.21.15.3:0/2388455645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa008100920 0x7fa008100d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 --2- 172.21.15.3:0/2388455645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008101b10 0x7fa008101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 -- 172.21.15.3:0/2388455645 >> 172.21.15.3:0/2388455645 conn(0x7fa0080fbe50 msgr2=0x7fa0080fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 -- 172.21.15.3:0/2388455645 shutdown_connections 2023-08-11T14:03:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.445+0000 7fa00cbff700 1 -- 172.21.15.3:0/2388455645 wait complete. 2023-08-11T14:03:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.446+0000 7fa00cbff700 1 Processor -- start 2023-08-11T14:03:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 -- start start 2023-08-11T14:03:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008100920 0x7fa0080746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0081024b0 0x7fa008073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080739f0 con 0x7fa008101b10 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008073b60 con 0x7fa0081024b0 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.447+0000 7fa00cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081a0680 con 0x7fa008100920 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45792/0 (socket says 172.21.15.3:45792) 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 -- 172.21.15.3:0/1406900224 learned_addr learned my addr 172.21.15.3:0/1406900224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa00659c700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008100920 0x7fa0080746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa006d9d700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0081024b0 0x7fa008073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 -- 172.21.15.3:0/1406900224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008100920 msgr2=0x7fa0080746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008100920 0x7fa0080746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 -- 172.21.15.3:0/1406900224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0081024b0 msgr2=0x7fa008073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0081024b0 0x7fa008073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 -- 172.21.15.3:0/1406900224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80095f0 con 0x7fa008101b10 2023-08-11T14:03:12.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.448+0000 7fa005d9b700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00e9a0 tx=0x7f9ffc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.449+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc009790 con 0x7fa008101b10 2023-08-11T14:03:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.449+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ffc010430 con 0x7fa008101b10 2023-08-11T14:03:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.449+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc018680 con 0x7fa008101b10 2023-08-11T14:03:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.449+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0081a0960 con 0x7fa008101b10 2023-08-11T14:03:12.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.449+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081a0e20 con 0x7fa008101b10 2023-08-11T14:03:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.450+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00810c180 con 0x7fa008101b10 2023-08-11T14:03:12.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.452+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9ffc0105a0 con 0x7fa008101b10 2023-08-11T14:03:12.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.453+0000 7f9ff77fe700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 0x7f9ff00a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.453+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9ffc014070 con 0x7fa008101b10 2023-08-11T14:03:12.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.453+0000 7fa00659c700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 0x7f9ff00a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.454+0000 7fa00659c700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 0x7f9ff00a24e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009990 tx=0x7f9ff8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.457+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9ffc051c30 con 0x7fa008101b10 2023-08-11T14:03:12.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:12 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/250947152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:12 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2808224076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:12 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/250947152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:12 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2808224076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.713+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00810c390 con 0x7fa008101b10 2023-08-11T14:03:12.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.714+0000 7f9ff77fe700 1 -- 172.21.15.3:0/1406900224 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f9ffc095100 con 0x7fa008101b10 2023-08-11T14:03:12.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:12.716 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 msgr2=0x7f9ff00a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 0x7f9ff00a24e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009990 tx=0x7f9ff8005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 msgr2=0x7fa008072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00e9a0 tx=0x7f9ffc00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 shutdown_connections 2023-08-11T14:03:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9ff00a0020 0x7f9ff00a24e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0081024b0 0x7fa008073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa008100920 0x7fa0080746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 --2- 172.21.15.3:0/1406900224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa008101b10 0x7fa008072d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 >> 172.21.15.3:0/1406900224 conn(0x7fa0080fbe50 msgr2=0x7fa0080fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 shutdown_connections 2023-08-11T14:03:12.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.717+0000 7fa00cbff700 1 -- 172.21.15.3:0/1406900224 wait complete. 2023-08-11T14:03:12.722 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:12.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-11T14:01:32.831322+0000', 'modified': '2023-08-11T14:02:01.312375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24532, 'mds_2': 24527, 'mds_3': 24511, 'mds_4': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14502': {'gid': 14502, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/3729253320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 3729253320}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 3729253320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6841/1195165407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 1195165407}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 1195165407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6833/1102333992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1102333992}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1102333992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/618362106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 618362106}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 618362106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/3236312344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 3236312344}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3236312344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.103:6835/1763153778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 1763153778}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 1763153778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.187:6837/1876247524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 1876247524}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 1876247524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24527': {'gid': 24527, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.103:6837/4115864783', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 4115864783}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 4115864783}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.187:6839/1685844974', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 1685844974}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 1685844974}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.103:6839/2494449018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2494449018}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2494449018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T14:03:12.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-08-11T14:03:12.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-08-11T14:03:12.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:12 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/250947152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:12 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2808224076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:12.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 -- 172.21.15.3:0/3053674545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 msgr2=0x7f3d3c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 --2- 172.21.15.3:0/3053674545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c101f60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0099c0 tx=0x7f3d2c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 -- 172.21.15.3:0/3053674545 shutdown_connections 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 --2- 172.21.15.3:0/3053674545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 0x7f3d3c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 --2- 172.21.15.3:0/3053674545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 --2- 172.21.15.3:0/3053674545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c101f60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 -- 172.21.15.3:0/3053674545 >> 172.21.15.3:0/3053674545 conn(0x7f3d3c0fbe60 msgr2=0x7f3d3c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 -- 172.21.15.3:0/3053674545 shutdown_connections 2023-08-11T14:03:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.948+0000 7f3d43dff700 1 -- 172.21.15.3:0/3053674545 wait complete. 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.949+0000 7f3d43dff700 1 Processor -- start 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 -- start start 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 0x7f3d3c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c0739f0 con 0x7f3d3c101b00 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c073b60 con 0x7f3d3c100910 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d43dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c1a0690 con 0x7f3d3c1024a0 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d41b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d41b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33848/0 (socket says 172.21.15.3:33848) 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45820/0 (socket says 172.21.15.3:45820) 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.950+0000 7f3d41b9b700 1 -- 172.21.15.3:0/256445335 learned_addr learned my addr 172.21.15.3:0/256445335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d4239c700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 0x7f3d3c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 -- 172.21.15.3:0/256445335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 msgr2=0x7f3d3c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 0x7f3d3c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 -- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 msgr2=0x7f3d3c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 -- 172.21.15.3:0/256445335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0095f0 con 0x7f3d3c100910 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d41b9b700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e970 tx=0x7f3d3800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d32ffd700 1 -- 172.21.15.3:0/256445335 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38009790 con 0x7f3d3c100910 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d32ffd700 1 -- 172.21.15.3:0/256445335 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3800c7e0 con 0x7f3d3c100910 2023-08-11T14:03:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c1a0970 con 0x7f3d3c100910 2023-08-11T14:03:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.951+0000 7f3d32ffd700 1 -- 172.21.15.3:0/256445335 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003c70 con 0x7f3d3c100910 2023-08-11T14:03:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.952+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c1a0e80 con 0x7f3d3c100910 2023-08-11T14:03:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.956+0000 7f3d32ffd700 1 -- 172.21.15.3:0/256445335 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f3d3800cc50 con 0x7f3d3c100910 2023-08-11T14:03:12.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.957+0000 7f3d577f6b80 1 Processor -- start 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.957+0000 7f3d577f6b80 1 -- start start 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f66860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826ebe5e0 con 0x555826f663e0 2023-08-11T14:03:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826ebbf00 con 0x555826f66da0 2023-08-11T14:03:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826eb77f0 con 0x555826f65a80 2023-08-11T14:03:12.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d41b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f66860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33964/0 (socket says 172.21.15.3:33964) 2023-08-11T14:03:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 -- 172.21.15.3:0/4255727758 learned_addr learned my addr 172.21.15.3:0/4255727758 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4139a700 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 -- 172.21.15.3:0/4255727758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 msgr2=0x555826f67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 -- 172.21.15.3:0/4255727758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 msgr2=0x555826f66860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f66860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.958+0000 7f3d4239c700 1 -- 172.21.15.3:0/4255727758 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555826d375e0 con 0x555826f65a80 2023-08-11T14:03:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d4239c700 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d34011600 tx=0x7f3d34009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=895cf572468005 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d32ffd700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 0x7f3d280a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d32ffd700 1 -- 172.21.15.3:0/256445335 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3d38014070 con 0x7f3d3c100910 2023-08-11T14:03:12.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d41b9b700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 0x7f3d280a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d27fff700 1 -- 172.21.15.3:0/4255727758 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400e040 con 0x555826f65a80 2023-08-11T14:03:12.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d27fff700 1 -- 172.21.15.3:0/4255727758 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3400f040 con 0x555826f65a80 2023-08-11T14:03:12.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.959+0000 7f3d27fff700 1 -- 172.21.15.3:0/4255727758 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34004790 con 0x555826f65a80 2023-08-11T14:03:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d41b9b700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 0x7f3d280a2410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3d380a0710 tx=0x7f3d380d1060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- 172.21.15.3:0/4255727758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 msgr2=0x555826f65ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d34011600 tx=0x7f3d34009920 comp rx=0 tx=0).stop 2023-08-11T14:03:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- 172.21.15.3:0/4255727758 shutdown_connections 2023-08-11T14:03:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f65ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/4255727758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f66860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- 172.21.15.3:0/4255727758 >> 172.21.15.3:0/4255727758 conn(0x555826e92900 msgr2=0x555826f603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- 172.21.15.3:0/4255727758 shutdown_connections 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- 172.21.15.3:0/4255727758 wait complete. 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.960+0000 7f3d577f6b80 1 -- start start 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 Processor -- start 2023-08-11T14:03:12.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f6f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f66da0 0x555826f76590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826e55a70 con 0x555826f663e0 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826e55770 con 0x555826f65a80 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826d398d0 con 0x555826f66da0 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d577f6b80 10 client.0 _ll_register_callbacks cb 0x555826e58a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45830/0 (socket says 172.21.15.3:45830) 2023-08-11T14:03:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.961+0000 7f3d4139a700 1 -- 172.21.15.3:0/3726990324 learned_addr learned my addr 172.21.15.3:0/3726990324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f6f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 -- 172.21.15.3:0/3726990324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f66da0 msgr2=0x555826f76590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f66da0 0x555826f76590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 -- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 msgr2=0x555826f6f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f6f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 -- 172.21.15.3:0/3726990324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d340112b0 con 0x555826f663e0 2023-08-11T14:03:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0058e0 tx=0x7f3d2c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201c441f3f083232 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d257fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c003d20 con 0x555826f663e0 2023-08-11T14:03:12.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555826f58a40 con 0x555826f663e0 2023-08-11T14:03:12.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.962+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3d2c005040 con 0x555826f663e0 2023-08-11T14:03:12.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d2c003e80 con 0x555826f663e0 2023-08-11T14:03:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 10 client.15129 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555826ce3d90 con 0x555826f663e0 2023-08-11T14:03:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555826f58a40 con 0x555826f663e0 2023-08-11T14:03:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3d2c00fae0 con 0x555826f663e0 2023-08-11T14:03:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 10 client.15129 resolve_mds: resolved name 'e' to daemon mds.e 2023-08-11T14:03:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 20 client.15129 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826f7abb0 0x555826f79690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 4 client.15129 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.963+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555826f34490 con 0x555826f7abb0 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.964+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826f7abb0 0x555826f79690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.964+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826f7abb0 0x555826f79690 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2cb08e7b44261b7 server_cookie=2a4e7f7ee5ef8668 in_seq=0 out_seq=0 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.964+0000 7f3d257fa700 10 client.15129 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.965+0000 7f3d257fa700 1 -- 172.21.15.3:0/3726990324 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555826f34490 con 0x555826f7abb0 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.965+0000 7f3d257fa700 10 client.15129 handle_command_reply: tid=0 2023-08-11T14:03:12.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.965+0000 7f3d577f6b80 1 client.15129 shutdown 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.965+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 msgr2=0x555826f6f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.965+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0058e0 tx=0x7f3d2c005ee0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 shutdown_connections 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826f7abb0 0x555826f79690 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f66da0 0x555826f76590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f6f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3726990324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6f550 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 >> 172.21.15.3:0/3726990324 conn(0x555826e92900 msgr2=0x555826f5ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d257fa700 0 client.15129 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:12.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 shutdown_connections 2023-08-11T14:03:12.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3726990324 wait complete. 2023-08-11T14:03:12.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.966+0000 7f3d577f6b80 20 client.15129 trim_cache size 0 max 16384 2023-08-11T14:03:12.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.984+0000 7f3d577f6b80 1 Processor -- start 2023-08-11T14:03:12.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.984+0000 7f3d577f6b80 1 -- start start 2023-08-11T14:03:12.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f76520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826f70200 con 0x555826f663e0 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826d45de0 con 0x555826f66da0 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826e51d50 con 0x555826f65a80 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4239c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45834/0 (socket says 172.21.15.3:45834) 2023-08-11T14:03:12.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 -- 172.21.15.3:0/536251251 learned_addr learned my addr 172.21.15.3:0/536251251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4239c700 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f76520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 -- 172.21.15.3:0/536251251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 msgr2=0x555826f69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 -- 172.21.15.3:0/536251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 msgr2=0x555826f76520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.985+0000 7f3d4139a700 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f76520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d4139a700 1 -- 172.21.15.3:0/536251251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555826cccc10 con 0x555826f663e0 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d4239c700 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d4139a700 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00ffb0 tx=0x7f3d2c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e06018180a65db server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d30bff700 1 -- 172.21.15.3:0/536251251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c003d20 con 0x555826f663e0 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d30bff700 1 -- 172.21.15.3:0/536251251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2c01e050 con 0x555826f663e0 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.986+0000 7f3d30bff700 1 -- 172.21.15.3:0/536251251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c021690 con 0x555826f663e0 2023-08-11T14:03:12.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- 172.21.15.3:0/536251251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 msgr2=0x555826f6a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00ffb0 tx=0x7f3d2c009fd0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- 172.21.15.3:0/536251251 shutdown_connections 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f66da0 0x555826f76520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f65a80 0x555826f69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/536251251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f663e0 0x555826f6a1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- 172.21.15.3:0/536251251 >> 172.21.15.3:0/536251251 conn(0x555826e92900 msgr2=0x555826f6d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:12.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- 172.21.15.3:0/536251251 shutdown_connections 2023-08-11T14:03:12.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- 172.21.15.3:0/536251251 wait complete. 2023-08-11T14:03:12.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 -- start start 2023-08-11T14:03:12.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.987+0000 7f3d577f6b80 1 Processor -- start 2023-08-11T14:03:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f663e0 0x555826f711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826f766a0 con 0x555826f66da0 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826f76810 con 0x555826f65a80 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555826fce420 con 0x555826f663e0 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d577f6b80 10 client.0 _ll_register_callbacks cb 0x7f3d2c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d4239c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d4239c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45844/0 (socket says 172.21.15.3:45844) 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.988+0000 7f3d4239c700 1 -- 172.21.15.3:0/3245530754 learned_addr learned my addr 172.21.15.3:0/3245530754 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f663e0 0x555826f711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 -- 172.21.15.3:0/3245530754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f663e0 msgr2=0x555826f711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f663e0 0x555826f711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 -- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 msgr2=0x555826f70cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f70cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 -- 172.21.15.3:0/3245530754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555826d375e0 con 0x555826f66da0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d4239c700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d340115a0 tx=0x7f3d34017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249b295ffb06be99 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d257fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34025040 con 0x555826f66da0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d3400f040 con 0x555826f66da0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.989+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3d34004870 con 0x555826f66da0 2023-08-11T14:03:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d3400e040 con 0x555826f66da0 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 10 client.15135 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555826ce3d90 con 0x555826f66da0 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d3402f020 con 0x555826f66da0 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3d34003c30 con 0x555826f66da0 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 10 client.15135 resolve_mds: resolved name 'e' to daemon mds.e 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 20 client.15135 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826fd3670 0x555826fcfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 4 client.15135 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555826eb07d0 con 0x555826fd3670 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.990+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826fd3670 0x555826fcfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d257fa700 10 client.15135 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d4139a700 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826fd3670 0x555826fcfaf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9987dc4066fd152c server_cookie=570b616d7c269209 in_seq=0 out_seq=0 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d257fa700 1 -- 172.21.15.3:0/3245530754 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555826eb07d0 con 0x555826fd3670 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d257fa700 10 client.15135 handle_command_reply: tid=0 2023-08-11T14:03:12.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d577f6b80 1 client.15135 shutdown 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 msgr2=0x555826f76160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.991+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d340115a0 tx=0x7f3d34017040 comp rx=0 tx=0).stop 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 shutdown_connections 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x555826fd3670 0x555826fcfaf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555826f66da0 0x555826f76160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555826f65a80 0x555826f70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 --2- 172.21.15.3:0/3245530754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555826f663e0 0x555826f711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d257fa700 0 client.15135 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 >> 172.21.15.3:0/3245530754 conn(0x555826e92900 msgr2=0x555826f69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 shutdown_connections 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 1 -- 172.21.15.3:0/3245530754 wait complete. 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d577f6b80 20 client.15135 trim_cache size 0 max 16384 2023-08-11T14:03:13.001 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24500, 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:13.002 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 72.683283063999994, 2023-08-11T14:03:13.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:13.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:13.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:13.003 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 123.525654905 2023-08-11T14:03:13.003 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 msgr2=0x7f3d280a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 0x7f3d280a2410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3d380a0710 tx=0x7f3d380d1060 comp rx=0 tx=0).stop 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 msgr2=0x7f3d3c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.992+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800e970 tx=0x7f3d3800ec80 comp rx=0 tx=0).stop 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 shutdown_connections 2023-08-11T14:03:13.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d3c1024a0 0x7f3d3c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3d2809ff50 0x7f3d280a2410 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d3c100910 0x7f3d3c0746d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 --2- 172.21.15.3:0/256445335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d3c101b00 0x7f3d3c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 >> 172.21.15.3:0/256445335 conn(0x7f3d3c0fbe60 msgr2=0x7f3d3c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 shutdown_connections 2023-08-11T14:03:13.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:12.993+0000 7f3d43dff700 1 -- 172.21.15.3:0/256445335 wait complete. 2023-08-11T14:03:13.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-11T14:03:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 -- 172.21.15.3:0/2476718483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 msgr2=0x7f58b0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2476718483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b0073800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009990 tx=0x7f58a0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 -- 172.21.15.3:0/2476718483 shutdown_connections 2023-08-11T14:03:13.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2476718483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2476718483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 0x7f58b0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2476718483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b0073800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 -- 172.21.15.3:0/2476718483 >> 172.21.15.3:0/2476718483 conn(0x7f58b00fbe60 msgr2=0x7f58b00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 -- 172.21.15.3:0/2476718483 shutdown_connections 2023-08-11T14:03:13.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.215+0000 7f58b7bff700 1 -- 172.21.15.3:0/2476718483 wait complete. 2023-08-11T14:03:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.216+0000 7f58b7bff700 1 Processor -- start 2023-08-11T14:03:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 -- start start 2023-08-11T14:03:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 0x7f58b019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b01a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b01a47e0 con 0x7f58b010cee0 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b01a4950 con 0x7f58b0073380 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b7bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b01a4b10 con 0x7f58b0074960 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b01a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33910/0 (socket says 172.21.15.3:33910) 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b599b700 1 -- 172.21.15.3:0/2242101857 learned_addr learned my addr 172.21.15.3:0/2242101857 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.217+0000 7f58b519a700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 0x7f58b019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 -- 172.21.15.3:0/2242101857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 msgr2=0x7f58b019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 0x7f58b019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 -- 172.21.15.3:0/2242101857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 msgr2=0x7f58b019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 -- 172.21.15.3:0/2242101857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00095f0 con 0x7f58b010cee0 2023-08-11T14:03:13.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b599b700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b619c700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b01a4210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58a800efc0 tx=0x7f58a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58a6ffd700 1 -- 172.21.15.3:0/2242101857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8018070 con 0x7f58b010cee0 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b01a4d90 con 0x7f58b010cee0 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58a6ffd700 1 -- 172.21.15.3:0/2242101857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a800f040 con 0x7f58b010cee0 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58a6ffd700 1 -- 172.21.15.3:0/2242101857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800cd40 con 0x7f58b010cee0 2023-08-11T14:03:13.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.218+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b01a5210 con 0x7f58b010cee0 2023-08-11T14:03:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.221+0000 7f58a6ffd700 1 -- 172.21.15.3:0/2242101857 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f58a8003a00 con 0x7f58b010cee0 2023-08-11T14:03:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 Processor -- start 2023-08-11T14:03:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58a6ffd700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 0x7f589c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 -- start start 2023-08-11T14:03:13.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58a6ffd700 1 -- 172.21.15.3:0/2242101857 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f58a80cb190 con 0x7f58b010cee0 2023-08-11T14:03:13.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a163454ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58b519a700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 0x7f589c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 0x55a163455860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163455da0 0x55a163456220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1633ad5e0 con 0x55a1634553e0 2023-08-11T14:03:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1633aaf00 con 0x55a163454a80 2023-08-11T14:03:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.223+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1633a67f0 con 0x55a163455da0 2023-08-11T14:03:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b519a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 0x55a163455860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b519a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 0x55a163455860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45868/0 (socket says 172.21.15.3:45868) 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b519a700 1 -- 172.21.15.3:0/3467385183 learned_addr learned my addr 172.21.15.3:0/3467385183 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a163454ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b619c700 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163455da0 0x55a163456220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 -- 172.21.15.3:0/3467385183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163455da0 msgr2=0x55a163456220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163455da0 0x55a163456220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 -- 172.21.15.3:0/3467385183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 msgr2=0x55a163455860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 0x55a163455860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 -- 172.21.15.3:0/3467385183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1632265e0 con 0x55a163454a80 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b519a700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 0x7f589c0a2410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005950 tx=0x7f58a000b850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.224+0000 7f58b599b700 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a163454ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58b019db40 tx=0x7f58ac0118a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d6090c3e80c1bd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f589bfff700 1 -- 172.21.15.3:0/3467385183 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004230 con 0x55a163454a80 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f589bfff700 1 -- 172.21.15.3:0/3467385183 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58ac004cf0 con 0x55a163454a80 2023-08-11T14:03:13.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f589bfff700 1 -- 172.21.15.3:0/3467385183 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac005720 con 0x55a163454a80 2023-08-11T14:03:13.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3467385183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 msgr2=0x55a163454ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a163454ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58b019db40 tx=0x7f58ac0118a0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3467385183 shutdown_connections 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163455da0 0x55a163456220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a163454ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3467385183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1634553e0 0x55a163455860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3467385183 >> 172.21.15.3:0/3467385183 conn(0x55a163381900 msgr2=0x55a16344f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.225+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3467385183 shutdown_connections 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.226+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3467385183 wait complete. 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.226+0000 7f58cb5bab80 1 -- start start 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.226+0000 7f58cb5bab80 1 Processor -- start 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a16345e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a163344a70 con 0x55a163455da0 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a163344770 con 0x55a1634553e0 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1632288d0 con 0x55a163454a80 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58cb5bab80 10 client.0 _ll_register_callbacks cb 0x55a163347a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b599b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b519a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a16345e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b599b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34036/0 (socket says 172.21.15.3:34036) 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45880/0 (socket says 172.21.15.3:45880) 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.227+0000 7f58b599b700 1 -- 172.21.15.3:0/612707351 learned_addr learned my addr 172.21.15.3:0/612707351 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 -- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 msgr2=0x55a16345e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 -- 172.21.15.3:0/612707351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 msgr2=0x55a1634655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 -- 172.21.15.3:0/612707351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac009d80 con 0x55a1634553e0 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b599b700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b619c700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58b519a700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a16345e5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d883dc7210006a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58997fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0004b70 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a163447a40 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f58a0004cd0 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a0004e90 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58cb5bab80 10 client.25187 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.228+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a1631d2d90 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a163447a40 con 0x55a1634553e0 2023-08-11T14:03:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58a0031070 con 0x55a1634553e0 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 10 client.25187 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 10 client.25187 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 20 client.25187 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a163469b70 0x55a163468620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 4 client.25187 mds_command: new command op to 24511 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a163423490 con 0x55a163469b70 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.229+0000 7f58b599b700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a163469b70 0x55a163468620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.230+0000 7f58b599b700 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a163469b70 0x55a163468620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fe77c66cd9465ce server_cookie=70eea12a417511df in_seq=0 out_seq=0 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.230+0000 7f58997fa700 10 client.25187 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58997fa700 1 -- 172.21.15.3:0/612707351 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a163423490 con 0x55a163469b70 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58997fa700 10 client.25187 handle_command_reply: tid=0 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 client.25187 shutdown 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 msgr2=0x55a16345e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a16345e5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000c00 tx=0x7f58a0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 shutdown_connections 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a163469b70 0x55a163468620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a1634655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a16345e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/612707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a16345e5a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 >> 172.21.15.3:0/612707351 conn(0x55a163381900 msgr2=0x55a163449c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58997fa700 0 client.25187 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 shutdown_connections 2023-08-11T14:03:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 1 -- 172.21.15.3:0/612707351 wait complete. 2023-08-11T14:03:13.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.231+0000 7f58cb5bab80 20 client.25187 trim_cache size 0 max 16384 2023-08-11T14:03:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.249+0000 7f58cb5bab80 1 Processor -- start 2023-08-11T14:03:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.249+0000 7f58cb5bab80 1 -- start start 2023-08-11T14:03:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a163458d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a163340d50 con 0x55a163455da0 2023-08-11T14:03:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16336fef0 con 0x55a163454a80 2023-08-11T14:03:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a163363480 con 0x55a1634553e0 2023-08-11T14:03:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a163458d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b519a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b519a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34048/0 (socket says 172.21.15.3:34048) 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b599b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33940/0 (socket says 172.21.15.3:33940) 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b519a700 1 -- 172.21.15.3:0/3978019481 learned_addr learned my addr 172.21.15.3:0/3978019481 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 -- 172.21.15.3:0/3978019481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 msgr2=0x55a1634655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 -- 172.21.15.3:0/3978019481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 msgr2=0x55a1634724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b619c700 1 -- 172.21.15.3:0/3978019481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1631bbc10 con 0x55a163455da0 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.250+0000 7f58b599b700 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58b619c700 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a163458d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c88e0 tx=0x7f58a80ca040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edeedf44036b08f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58a4bff700 1 -- 172.21.15.3:0/3978019481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8098ff0 con 0x55a163455da0 2023-08-11T14:03:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58a4bff700 1 -- 172.21.15.3:0/3978019481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a8083040 con 0x55a163455da0 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58a4bff700 1 -- 172.21.15.3:0/3978019481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8098db0 con 0x55a163455da0 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3978019481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 msgr2=0x55a163458d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.251+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a163458d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f58a80c88e0 tx=0x7f58a80ca040 comp rx=0 tx=0).stop 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3978019481 shutdown_connections 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a163458d00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a163454a80 0x55a1634724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/3978019481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3978019481 >> 172.21.15.3:0/3978019481 conn(0x55a163381900 msgr2=0x55a16345c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3978019481 shutdown_connections 2023-08-11T14:03:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 -- 172.21.15.3:0/3978019481 wait complete. 2023-08-11T14:03:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 -- start start 2023-08-11T14:03:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.252+0000 7f58cb5bab80 1 Processor -- start 2023-08-11T14:03:13.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a163459e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16345ba20 con 0x55a163455da0 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16345a880 con 0x55a1634553e0 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a16345aa40 con 0x55a163454a80 2023-08-11T14:03:13.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58cb5bab80 10 client.0 _ll_register_callbacks cb 0x7f58a000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45898/0 (socket says 172.21.15.3:45898) 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 -- 172.21.15.3:0/4344981 learned_addr learned my addr 172.21.15.3:0/4344981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b599b700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a163459e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b519a700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 -- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 msgr2=0x55a163459e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a163459e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 -- 172.21.15.3:0/4344981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 msgr2=0x55a1634655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.253+0000 7f58b619c700 1 -- 172.21.15.3:0/4344981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac01d040 con 0x55a163455da0 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58b619c700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58a8095360 tx=0x7f58a8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a196aa90d28a77 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80988f0 con 0x55a163455da0 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ac004cf0 con 0x55a163455da0 2023-08-11T14:03:13.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f58a809c720 con 0x55a163455da0 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a8098ff0 con 0x55a163455da0 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58cb5bab80 10 client.15153 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f58b01a5210 con 0x55a163455da0 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58ac004cf0 con 0x55a163455da0 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.254+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58a8097850 con 0x55a163455da0 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 10 client.15153 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-11T14:03:13.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 20 client.15153 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a1634c1d70 0x55a1634be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 4 client.15153 mds_command: new command op to 24511 tid=0[{"prefix": "status"}] 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a16339f7d0 con 0x55a1634c1d70 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58b599b700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a1634c1d70 0x55a1634be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58b599b700 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a1634c1d70 0x55a1634be230 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de959074e02b15a0 server_cookie=782ca6b8963a5e58 in_seq=0 out_seq=0 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.255+0000 7f58997fa700 10 client.15153 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58997fa700 1 -- 172.21.15.3:0/4344981 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a16339f7d0 con 0x55a1634c1d70 2023-08-11T14:03:13.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58997fa700 10 client.15153 handle_command_reply: tid=0 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 client.15153 shutdown 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 msgr2=0x55a16345a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58a8095360 tx=0x7f58a8004dc0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 shutdown_connections 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x55a1634c1d70 0x55a1634be230 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a163455da0 0x55a16345a340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a163454a80 0x55a163459e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 --2- 172.21.15.3:0/4344981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1634553e0 0x55a1634655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 >> 172.21.15.3:0/4344981 conn(0x55a163381900 msgr2=0x55a163458d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58997fa700 0 client.15153 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 shutdown_connections 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 1 -- 172.21.15.3:0/4344981 wait complete. 2023-08-11T14:03:13.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.256+0000 7f58cb5bab80 20 client.15153 trim_cache size 0 max 16384 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24511, 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:13.267 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 73.954510623999994, 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 119.25135569 2023-08-11T14:03:13.268 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:13.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 msgr2=0x7f589c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 0x7f589c0a2410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005950 tx=0x7f58a000b850 comp rx=0 tx=0).stop 2023-08-11T14:03:13.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 msgr2=0x7f58b01a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b01a4210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f58a800efc0 tx=0x7f58a800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 shutdown_connections 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58b010cee0 0x7f58b01a4210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f589c09ff50 0x7f589c0a2410 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58b0073380 0x7f58b019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 --2- 172.21.15.3:0/2242101857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58b0074960 0x7f58b019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 >> 172.21.15.3:0/2242101857 conn(0x7f58b00fbe60 msgr2=0x7f58b00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 shutdown_connections 2023-08-11T14:03:13.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.257+0000 7f58b7bff700 1 -- 172.21.15.3:0/2242101857 wait complete. 2023-08-11T14:03:13.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-08-11T14:03:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 -- 172.21.15.3:0/4271823351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 msgr2=0x7f7d30101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 --2- 172.21.15.3:0/4271823351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30101f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009920 tx=0x7f7d1c009c30 comp rx=0 tx=0).stop 2023-08-11T14:03:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 -- 172.21.15.3:0/4271823351 shutdown_connections 2023-08-11T14:03:13.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 --2- 172.21.15.3:0/4271823351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d30108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 --2- 172.21.15.3:0/4271823351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d30100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 --2- 172.21.15.3:0/4271823351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30101f60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 -- 172.21.15.3:0/4271823351 >> 172.21.15.3:0/4271823351 conn(0x7f7d300fbe60 msgr2=0x7f7d300fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 -- 172.21.15.3:0/4271823351 shutdown_connections 2023-08-11T14:03:13.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.483+0000 7f7d36fff700 1 -- 172.21.15.3:0/4271823351 wait complete. 2023-08-11T14:03:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.484+0000 7f7d36fff700 1 Processor -- start 2023-08-11T14:03:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.484+0000 7f7d36fff700 1 -- start start 2023-08-11T14:03:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d30073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300739f0 con 0x7f7d30100910 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d30073b60 con 0x7f7d30101b00 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d36fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a06c0 con 0x7f7d301024a0 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45918/0 (socket says 172.21.15.3:45918) 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1009747313 learned_addr learned my addr 172.21.15.3:0/1009747313 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d3559c700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d30073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1009747313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 msgr2=0x7f7d30073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d30073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1009747313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 msgr2=0x7f7d30072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1009747313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009520 con 0x7f7d30100910 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.485+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e9f0 tx=0x7f7d2400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d2dffb700 1 -- 172.21.15.3:0/1009747313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24004220 con 0x7f7d30100910 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d2dffb700 1 -- 172.21.15.3:0/1009747313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d24004380 con 0x7f7d30100910 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d301a09a0 con 0x7f7d30100910 2023-08-11T14:03:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d2dffb700 1 -- 172.21.15.3:0/1009747313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24003be0 con 0x7f7d30100910 2023-08-11T14:03:13.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.486+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301a0f00 con 0x7f7d30100910 2023-08-11T14:03:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.489+0000 7f7d2dffb700 1 -- 172.21.15.3:0/1009747313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7d24003d40 con 0x7f7d30100910 2023-08-11T14:03:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 Processor -- start 2023-08-11T14:03:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d2dffb700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 0x7f7d200a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 -- start start 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d2dffb700 1 -- 172.21.15.3:0/1009747313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7d24014070 con 0x7f7d30100910 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d3559c700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 0x7f7d200a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 0x55f2b0492860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b0493220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b03ea5e0 con 0x55f2b0492da0 2023-08-11T14:03:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b03e7f00 con 0x55f2b0491a80 2023-08-11T14:03:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.491+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b03e37f0 con 0x55f2b04923e0 2023-08-11T14:03:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 0x55f2b0492860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33956/0 (socket says 172.21.15.3:33956) 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d34d9b700 1 -- 172.21.15.3:0/103890421 learned_addr learned my addr 172.21.15.3:0/103890421 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d3559c700 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b0493220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d3559c700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 0x7f7d200a2490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d30074d30 tx=0x7f7d28009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 -- 172.21.15.3:0/103890421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 msgr2=0x55f2b0491ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 -- 172.21.15.3:0/103890421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 msgr2=0x55f2b0493220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b0493220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d2ffff700 1 -- 172.21.15.3:0/103890421 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b02635e0 con 0x55f2b04923e0 2023-08-11T14:03:13.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.492+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 0x55f2b0492860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005e90 tx=0x7f7d1c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b28394686efd2402 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d1affd700 1 -- 172.21.15.3:0/103890421 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01c050 con 0x55f2b04923e0 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d1affd700 1 -- 172.21.15.3:0/103890421 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c01d040 con 0x55f2b04923e0 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d1affd700 1 -- 172.21.15.3:0/103890421 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0052a0 con 0x55f2b04923e0 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d4a851b80 1 -- 172.21.15.3:0/103890421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 msgr2=0x55f2b0492860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.493+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 0x55f2b0492860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005e90 tx=0x7f7d1c005dc0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 -- 172.21.15.3:0/103890421 shutdown_connections 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b0493220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0491a80 0x55f2b0491ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/103890421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b04923e0 0x55f2b0492860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 -- 172.21.15.3:0/103890421 >> 172.21.15.3:0/103890421 conn(0x55f2b03be900 msgr2=0x55f2b048c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 -- 172.21.15.3:0/103890421 shutdown_connections 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 -- 172.21.15.3:0/103890421 wait complete. 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 -- start start 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.494+0000 7f7d4a851b80 1 Processor -- start 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b049b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b0381a70 con 0x55f2b0492da0 2023-08-11T14:03:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b0381770 con 0x55f2b04923e0 2023-08-11T14:03:13.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b02658d0 con 0x55f2b0491a80 2023-08-11T14:03:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d4a851b80 10 client.0 _ll_register_callbacks cb 0x55f2b0384a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d3559c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d3559c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45942/0 (socket says 172.21.15.3:45942) 2023-08-11T14:03:13.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d3559c700 1 -- 172.21.15.3:0/122385037 learned_addr learned my addr 172.21.15.3:0/122385037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.495+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b049b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 -- 172.21.15.3:0/122385037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 msgr2=0x55f2b049b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 -- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 msgr2=0x55f2b049b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b049b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 -- 172.21.15.3:0/122385037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301a0f00 con 0x55f2b0492da0 2023-08-11T14:03:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d3559c700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d28019e60 tx=0x7f7d28004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee84d062e90a6ad server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800e040 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2b0484a40 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7d28017d40 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d28004bb0 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d4a851b80 10 client.15165 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.496+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7d301096a0 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2b0484a40 con 0x55f2b0492da0 2023-08-11T14:03:13.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d28004630 con 0x55f2b0492da0 2023-08-11T14:03:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 10 client.15165 resolve_mds: resolved name 'h' to daemon mds.h 2023-08-11T14:03:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 20 client.15165 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04a6cc0 0x55f2b04a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 4 client.15165 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2b0460490 con 0x55f2b04a6cc0 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.497+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04a6cc0 0x55f2b04a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.498+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04a6cc0 0x55f2b04a5700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d69b770b96deb872 server_cookie=d74b18cea96a862a in_seq=0 out_seq=0 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.498+0000 7f7d0ffff700 10 client.15165 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d0ffff700 1 -- 172.21.15.3:0/122385037 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f2b0460490 con 0x55f2b04a6cc0 2023-08-11T14:03:13.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d0ffff700 10 client.15165 handle_command_reply: tid=0 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 client.15165 shutdown 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 msgr2=0x55f2b04a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d28019e60 tx=0x7f7d28004b40 comp rx=0 tx=0).stop 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 shutdown_connections 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04a6cc0 0x55f2b04a5700 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0492da0 0x55f2b04a25f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/122385037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b049b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d0ffff700 0 client.15165 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 >> 172.21.15.3:0/122385037 conn(0x55f2b03be900 msgr2=0x55f2b0486c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 shutdown_connections 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.499+0000 7f7d4a851b80 1 -- 172.21.15.3:0/122385037 wait complete. 2023-08-11T14:03:13.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.500+0000 7f7d4a851b80 20 client.15165 trim_cache size 0 max 16384 2023-08-11T14:03:13.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 Processor -- start 2023-08-11T14:03:13.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 -- start start 2023-08-11T14:03:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b04af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0492da0 0x55f2b049b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b03a04b0 con 0x55f2b0491a80 2023-08-11T14:03:13.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b048d510 con 0x55f2b04923e0 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.518+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b049a5c0 con 0x55f2b0492da0 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45948/0 (socket says 172.21.15.3:45948) 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1199650394 learned_addr learned my addr 172.21.15.3:0/1199650394 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b04af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d3559c700 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0492da0 0x55f2b049b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1199650394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0492da0 msgr2=0x55f2b049b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0492da0 0x55f2b049b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1199650394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 msgr2=0x55f2b04af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b04af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1199650394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b020fd90 con 0x55f2b0491a80 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.519+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d24098bf0 tx=0x7f7d2400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e2adc09e710760 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d1b7fe700 1 -- 172.21.15.3:0/1199650394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240c2ea0 con 0x55f2b0491a80 2023-08-11T14:03:13.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d1b7fe700 1 -- 172.21.15.3:0/1199650394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d24098cb0 con 0x55f2b0491a80 2023-08-11T14:03:13.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d1b7fe700 1 -- 172.21.15.3:0/1199650394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240953c0 con 0x55f2b0491a80 2023-08-11T14:03:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1199650394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 msgr2=0x55f2b049b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d24098bf0 tx=0x7f7d2400ef80 comp rx=0 tx=0).stop 2023-08-11T14:03:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1199650394 shutdown_connections 2023-08-11T14:03:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0492da0 0x55f2b049b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b0491a80 0x55f2b049b070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1199650394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b04923e0 0x55f2b04af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1199650394 >> 172.21.15.3:0/1199650394 conn(0x55f2b03be900 msgr2=0x55f2b0495d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1199650394 shutdown_connections 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.520+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1199650394 wait complete. 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 -- start start 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 Processor -- start 2023-08-11T14:03:13.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b04aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 0x55f2b04aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b04fa530 con 0x55f2b04923e0 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b04fa670 con 0x55f2b0492da0 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b04fa7b0 con 0x55f2b0491a80 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.521+0000 7f7d4a851b80 10 client.0 _ll_register_callbacks cb 0x7f7d28019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b04aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d3559c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 0x55f2b04aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d34d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b04aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34118/0 (socket says 172.21.15.3:34118) 2023-08-11T14:03:13.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45964/0 (socket says 172.21.15.3:45964) 2023-08-11T14:03:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d34d9b700 1 -- 172.21.15.3:0/1935514229 learned_addr learned my addr 172.21.15.3:0/1935514229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 -- 172.21.15.3:0/1935514229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 msgr2=0x55f2b04aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b04aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 -- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 msgr2=0x55f2b04aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 0x55f2b04aa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 -- 172.21.15.3:0/1935514229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301a09a0 con 0x55f2b04923e0 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d3559c700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 0x55f2b04aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00f9a0 tx=0x7f7d1c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0d2243f48d8d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.522+0000 7f7d0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01c040 con 0x55f2b04923e0 2023-08-11T14:03:13.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d1c01d040 con 0x55f2b04923e0 2023-08-11T14:03:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7d1c0050c0 con 0x55f2b04923e0 2023-08-11T14:03:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d1c005280 con 0x55f2b04923e0 2023-08-11T14:03:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 10 client.15171 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f2b01f8c10 con 0x55f2b04923e0 2023-08-11T14:03:13.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1c01d040 con 0x55f2b04923e0 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7d1c015020 con 0x55f2b04923e0 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 10 client.15171 resolve_mds: resolved name 'h' to daemon mds.h 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 20 client.15171 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04ff550 0x55f2b04fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 4 client.15171 mds_command: new command op to 24527 tid=0[{"prefix": "status"}] 2023-08-11T14:03:13.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.523+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f2b03dc7d0 con 0x55f2b04ff550 2023-08-11T14:03:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04ff550 0x55f2b04fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d34d9b700 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04ff550 0x55f2b04fba20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f7307bf1438b4c6 server_cookie=558fbe68c9fe23b7 in_seq=0 out_seq=0 2023-08-11T14:03:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d0ffff700 10 client.15171 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d0ffff700 1 -- 172.21.15.3:0/1935514229 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f2b03dc7d0 con 0x55f2b04ff550 2023-08-11T14:03:13.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d0ffff700 10 client.15171 handle_command_reply: tid=0 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.524+0000 7f7d4a851b80 1 client.15171 shutdown 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 msgr2=0x55f2b04af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00f9a0 tx=0x7f7d1c00b790 comp rx=0 tx=0).stop 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 shutdown_connections 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55f2b04ff550 0x55f2b04fba20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2b0492da0 0x55f2b04aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2b0491a80 0x55f2b04aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 --2- 172.21.15.3:0/1935514229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2b04923e0 0x55f2b04af5e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d0ffff700 0 client.15171 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 >> 172.21.15.3:0/1935514229 conn(0x55f2b03be900 msgr2=0x55f2b049ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 shutdown_connections 2023-08-11T14:03:13.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 1 -- 172.21.15.3:0/1935514229 wait complete. 2023-08-11T14:03:13.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.525+0000 7f7d4a851b80 20 client.15171 trim_cache size 0 max 16384 2023-08-11T14:03:13.538 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:13.538 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:13.538 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2023-08-11T14:03:13.538 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24527, 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 75.229042929000002, 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:13.539 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:13.540 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:13.540 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 117.299909645 2023-08-11T14:03:13.540 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 msgr2=0x7f7d200a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 0x7f7d200a2490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d30074d30 tx=0x7f7d28009920 comp rx=0 tx=0).stop 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 msgr2=0x7f7d300746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e9f0 tx=0x7f7d2400ed00 comp rx=0 tx=0).stop 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 shutdown_connections 2023-08-11T14:03:13.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d301024a0 0x7f7d30073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d30100910 0x7f7d300746d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7d2009ffd0 0x7f7d200a2490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 --2- 172.21.15.3:0/1009747313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d30101b00 0x7f7d30072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 >> 172.21.15.3:0/1009747313 conn(0x7f7d300fbe60 msgr2=0x7f7d301050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 shutdown_connections 2023-08-11T14:03:13.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.526+0000 7f7d36fff700 1 -- 172.21.15.3:0/1009747313 wait complete. 2023-08-11T14:03:13.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:13 smithi003 ceph-mon[97195]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 577 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:13.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:13 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1406900224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:13.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-11T14:03:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:13 smithi103 ceph-mon[130434]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 577 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:13.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:13 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1406900224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 -- 172.21.15.3:0/3814388103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 msgr2=0x7f529c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 --2- 172.21.15.3:0/3814388103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c108d50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f529400b0d0 tx=0x7f529400b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 -- 172.21.15.3:0/3814388103 shutdown_connections 2023-08-11T14:03:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 --2- 172.21.15.3:0/3814388103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c108d50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 --2- 172.21.15.3:0/3814388103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c100910 0x7f529c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 --2- 172.21.15.3:0/3814388103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c101b00 0x7f529c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 -- 172.21.15.3:0/3814388103 >> 172.21.15.3:0/3814388103 conn(0x7f529c0fbe60 msgr2=0x7f529c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 -- 172.21.15.3:0/3814388103 shutdown_connections 2023-08-11T14:03:13.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.749+0000 7f52a23ff700 1 -- 172.21.15.3:0/3814388103 wait complete. 2023-08-11T14:03:13.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.750+0000 7f52a23ff700 1 Processor -- start 2023-08-11T14:03:13.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.750+0000 7f52a23ff700 1 -- start start 2023-08-11T14:03:13.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 0x7f529c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c101b00 0x7f529c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a0440 con 0x7f529c1024a0 2023-08-11T14:03:13.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a0580 con 0x7f529c101b00 2023-08-11T14:03:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a23ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a0710 con 0x7f529c100910 2023-08-11T14:03:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a099c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f529bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 0x7f529c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a099c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45970/0 (socket says 172.21.15.3:45970) 2023-08-11T14:03:13.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a099c700 1 -- 172.21.15.3:0/595713310 learned_addr learned my addr 172.21.15.3:0/595713310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f529b7fe700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c101b00 0x7f529c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a099c700 1 -- 172.21.15.3:0/595713310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 msgr2=0x7f529c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.751+0000 7f52a099c700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 0x7f529c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a099c700 1 -- 172.21.15.3:0/595713310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c101b00 msgr2=0x7f529c198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a099c700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c101b00 0x7f529c198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a099c700 1 -- 172.21.15.3:0/595713310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009520 con 0x7f529c1024a0 2023-08-11T14:03:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f529bfff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 0x7f529c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a099c700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5294008ec0 tx=0x7f5294008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52997fa700 1 -- 172.21.15.3:0/595713310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400e040 con 0x7f529c1024a0 2023-08-11T14:03:13.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5294009d00 con 0x7f529c1024a0 2023-08-11T14:03:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52997fa700 1 -- 172.21.15.3:0/595713310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294005070 con 0x7f529c1024a0 2023-08-11T14:03:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52997fa700 1 -- 172.21.15.3:0/595713310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529401b6c0 con 0x7f529c1024a0 2023-08-11T14:03:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.752+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a0c80 con 0x7f529c1024a0 2023-08-11T14:03:13.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.755+0000 7f52997fa700 1 -- 172.21.15.3:0/595713310 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f52940051e0 con 0x7f529c1024a0 2023-08-11T14:03:13.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.757+0000 7f52b5de3b80 1 Processor -- start 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 -- start start 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52997fa700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 0x7f52840a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 0x5650955e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e43e0 0x5650955e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 0x5650955e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52997fa700 1 -- 172.21.15.3:0/595713310 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f52940cc650 con 0x7f529c1024a0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509553c5e0 con 0x5650955e4da0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x565095539f00 con 0x5650955e43e0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955357f0 con 0x5650955e3a80 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.758+0000 7f529b7fe700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 0x7f52840a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 0x5650955e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f529bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 0x5650955e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f529bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 0x5650955e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34144/0 (socket says 172.21.15.3:34144) 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f529bfff700 1 -- 172.21.15.3:0/2897571147 learned_addr learned my addr 172.21.15.3:0/2897571147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f529b7fe700 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e43e0 0x5650955e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 -- 172.21.15.3:0/2897571147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 msgr2=0x5650955e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 0x5650955e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 -- 172.21.15.3:0/2897571147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e43e0 msgr2=0x5650955e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e43e0 0x5650955e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 -- 172.21.15.3:0/2897571147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650953b55e0 con 0x5650955e4da0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f529b7fe700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 0x7f52840a2410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5290013570 tx=0x7f5290009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52a099c700 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 0x5650955e5220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f5294005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89aef21e52af480 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52827fc700 1 -- 172.21.15.3:0/2897571147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529409a4b0 con 0x5650955e4da0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52827fc700 1 -- 172.21.15.3:0/2897571147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294099db0 con 0x5650955e4da0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.759+0000 7f52827fc700 1 -- 172.21.15.3:0/2897571147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940cc420 con 0x5650955e4da0 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 -- 172.21.15.3:0/2897571147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 msgr2=0x5650955e5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 0x5650955e5220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5294009fd0 tx=0x7f5294005780 comp rx=0 tx=0).stop 2023-08-11T14:03:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 -- 172.21.15.3:0/2897571147 shutdown_connections 2023-08-11T14:03:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e4da0 0x5650955e5220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e3a80 0x5650955e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/2897571147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e43e0 0x5650955e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 -- 172.21.15.3:0/2897571147 >> 172.21.15.3:0/2897571147 conn(0x565095510900 msgr2=0x5650955de3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 -- 172.21.15.3:0/2897571147 shutdown_connections 2023-08-11T14:03:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.760+0000 7f52b5de3b80 1 -- 172.21.15.3:0/2897571147 wait complete. 2023-08-11T14:03:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.761+0000 7f52b5de3b80 1 -- start start 2023-08-11T14:03:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.761+0000 7f52b5de3b80 1 Processor -- start 2023-08-11T14:03:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 0x5650955ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e4da0 0x5650955f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650954d3a70 con 0x5650955e43e0 2023-08-11T14:03:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650954d3770 con 0x5650955e3a80 2023-08-11T14:03:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650953b78d0 con 0x5650955e4da0 2023-08-11T14:03:13.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52b5de3b80 10 client.0 _ll_register_callbacks cb 0x5650954d6a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 0x5650955ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f52a099c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e4da0 0x5650955f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45998/0 (socket says 172.21.15.3:45998) 2023-08-11T14:03:13.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 -- 172.21.15.3:0/3389448727 learned_addr learned my addr 172.21.15.3:0/3389448727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 -- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e4da0 msgr2=0x5650955f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e4da0 0x5650955f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 -- 172.21.15.3:0/3389448727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 msgr2=0x5650955ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 0x5650955ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529b7fe700 1 -- 172.21.15.3:0/3389448727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a0c80 con 0x5650955e43e0 2023-08-11T14:03:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.762+0000 7f529bfff700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 0x5650955ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f529b7fe700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5290009910 tx=0x7f52900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af85801f8d29e8d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f527b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290018040 con 0x5650955e43e0 2023-08-11T14:03:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650955d6a40 con 0x5650955e43e0 2023-08-11T14:03:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5290010630 con 0x5650955e43e0 2023-08-11T14:03:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52900122e0 con 0x5650955e43e0 2023-08-11T14:03:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f52b5de3b80 10 client.15183 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565095361d90 con 0x5650955e43e0 2023-08-11T14:03:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.763+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650955d6a40 con 0x5650955e43e0 2023-08-11T14:03:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5290026070 con 0x5650955e43e0 2023-08-11T14:03:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 10 client.15183 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 10 client.15183 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-11T14:03:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 20 client.15183 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x5650955f8ae0 0x5650955f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 4 client.15183 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:13.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650955b2490 con 0x5650955f8ae0 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.764+0000 7f529bfff700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x5650955f8ae0 0x5650955f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.765+0000 7f529bfff700 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x5650955f8ae0 0x5650955f7570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6770ad128f62402 server_cookie=61fd4af97096c3bb in_seq=0 out_seq=0 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.765+0000 7f527b7fe700 10 client.15183 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f527b7fe700 1 -- 172.21.15.3:0/3389448727 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5650955b2490 con 0x5650955f8ae0 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f527b7fe700 10 client.15183 handle_command_reply: tid=0 2023-08-11T14:03:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 client.15183 shutdown 2023-08-11T14:03:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 msgr2=0x5650955ed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5290009910 tx=0x7f52900047c0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 shutdown_connections 2023-08-11T14:03:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x5650955f8ae0 0x5650955f7570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e4da0 0x5650955f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e3a80 0x5650955ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/3389448727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e43e0 0x5650955ed430 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f527b7fe700 0 client.15183 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 >> 172.21.15.3:0/3389448727 conn(0x565095510900 msgr2=0x5650955d8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 shutdown_connections 2023-08-11T14:03:13.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 1 -- 172.21.15.3:0/3389448727 wait complete. 2023-08-11T14:03:13.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.766+0000 7f52b5de3b80 20 client.15183 trim_cache size 0 max 16384 2023-08-11T14:03:13.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 Processor -- start 2023-08-11T14:03:13.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 -- start start 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955b2490 con 0x5650955e3a80 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955f6160 con 0x5650955e4da0 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650953c3de0 con 0x5650955e43e0 2023-08-11T14:03:13.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.785+0000 7f529bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46006/0 (socket says 172.21.15.3:46006) 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 -- 172.21.15.3:0/517064436 learned_addr learned my addr 172.21.15.3:0/517064436 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f52a099c700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529b7fe700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 -- 172.21.15.3:0/517064436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 msgr2=0x5650955f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 -- 172.21.15.3:0/517064436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 msgr2=0x5650955de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 -- 172.21.15.3:0/517064436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56509534ac10 con 0x5650955e3a80 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529b7fe700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f529bfff700 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5650954ffd60 tx=0x7f528c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d224fd1dac3d7d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.786+0000 7f5282ffd700 1 -- 172.21.15.3:0/517064436 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c017040 con 0x5650955e3a80 2023-08-11T14:03:13.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f5282ffd700 1 -- 172.21.15.3:0/517064436 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528c018040 con 0x5650955e3a80 2023-08-11T14:03:13.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f5282ffd700 1 -- 172.21.15.3:0/517064436 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c005280 con 0x5650955e3a80 2023-08-11T14:03:13.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 -- 172.21.15.3:0/517064436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 msgr2=0x5650955de360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5650954ffd60 tx=0x7f528c009260 comp rx=0 tx=0).stop 2023-08-11T14:03:13.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 -- 172.21.15.3:0/517064436 shutdown_connections 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955de360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/517064436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 -- 172.21.15.3:0/517064436 >> 172.21.15.3:0/517064436 conn(0x565095510900 msgr2=0x5650955e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 -- 172.21.15.3:0/517064436 shutdown_connections 2023-08-11T14:03:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.787+0000 7f52b5de3b80 1 -- 172.21.15.3:0/517064436 wait complete. 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.788+0000 7f52b5de3b80 1 -- start start 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.788+0000 7f52b5de3b80 1 Processor -- start 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955e9b10 con 0x5650955e3a80 2023-08-11T14:03:13.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955e9c80 con 0x5650955e4da0 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650955e9e40 con 0x5650955e43e0 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52b5de3b80 10 client.0 _ll_register_callbacks cb 0x7f5290004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46018/0 (socket says 172.21.15.3:46018) 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 -- 172.21.15.3:0/532924902 learned_addr learned my addr 172.21.15.3:0/532924902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52a099c700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529b7fe700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 -- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 msgr2=0x5650955f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 -- 172.21.15.3:0/532924902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 msgr2=0x5650955e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955e95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f529bfff700 1 -- 172.21.15.3:0/532924902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c012ca0 con 0x5650955e3a80 2023-08-11T14:03:13.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.789+0000 7f52a099c700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f529bfff700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f528c004fb0 tx=0x7f528c0120d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e296d2f1ad8e48de server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f527b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c004950 con 0x5650955e3a80 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5294099db0 con 0x5650955e3a80 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f528c004ab0 con 0x5650955e3a80 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f528c0210f0 con 0x5650955e3a80 2023-08-11T14:03:13.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f52b5de3b80 10 client.15189 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565095388200 con 0x5650955e3a80 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.790+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294099db0 con 0x5650955e3a80 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f528c033500 con 0x5650955e3a80 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 20 client.15189 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:13.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x565095651200 0x56509564d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 4 client.15189 mds_command: new command op to 24532 tid=0[{"prefix": "status"}] 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56509552e7d0 con 0x565095651200 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.791+0000 7f529b7fe700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x565095651200 0x56509564d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f529b7fe700 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x565095651200 0x56509564d660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98adf102a49e9c06 server_cookie=71e89ddcf60775f8 in_seq=0 out_seq=0 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f527b7fe700 10 client.15189 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f527b7fe700 1 -- 172.21.15.3:0/532924902 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56509552e7d0 con 0x565095651200 2023-08-11T14:03:13.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f527b7fe700 10 client.15189 handle_command_reply: tid=0 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 client.15189 shutdown 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 msgr2=0x5650955e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f528c004fb0 tx=0x7f528c0120d0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 shutdown_connections 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x565095651200 0x56509564d660 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650955e4da0 0x5650955e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5650955e3a80 0x5650955e9090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 --2- 172.21.15.3:0/532924902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650955e43e0 0x5650955f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 >> 172.21.15.3:0/532924902 conn(0x565095510900 msgr2=0x5650955de230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f527b7fe700 0 client.15189 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 shutdown_connections 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 1 -- 172.21.15.3:0/532924902 wait complete. 2023-08-11T14:03:13.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.792+0000 7f52b5de3b80 20 client.15189 trim_cache size 0 max 16384 2023-08-11T14:03:13.808 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:13.808 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24532, 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:13.809 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 76.509592538000007, 2023-08-11T14:03:13.810 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:13.810 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:13.810 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:13.810 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 115.32730003499999 2023-08-11T14:03:13.810 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:13.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 msgr2=0x7f52840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 0x7f52840a2410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5290013570 tx=0x7f5290009f90 comp rx=0 tx=0).stop 2023-08-11T14:03:13.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 msgr2=0x7f529c19fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:13.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5294008ec0 tx=0x7f5294008ef0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 shutdown_connections 2023-08-11T14:03:13.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f529c1024a0 0x7f529c19fe70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f529c100910 0x7f529c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f528409ff50 0x7f52840a2410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 --2- 172.21.15.3:0/595713310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f529c101b00 0x7f529c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:13.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 >> 172.21.15.3:0/595713310 conn(0x7f529c0fbe60 msgr2=0x7f529c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:13.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 shutdown_connections 2023-08-11T14:03:13.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:13.793+0000 7f52a23ff700 1 -- 172.21.15.3:0/595713310 wait complete. 2023-08-11T14:03:13.815 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:13 smithi187 ceph-mon[130973]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 577 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:13.816 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:13 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1406900224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:13.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T14:03:14.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 -- 172.21.15.3:0/2147305877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c100930 msgr2=0x7fc86c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 --2- 172.21.15.3:0/2147305877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c100930 0x7fc86c100d50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099c0 tx=0x7fc868009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 -- 172.21.15.3:0/2147305877 shutdown_connections 2023-08-11T14:03:14.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 --2- 172.21.15.3:0/2147305877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c1024c0 0x7fc86c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 --2- 172.21.15.3:0/2147305877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c100930 0x7fc86c100d50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 --2- 172.21.15.3:0/2147305877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c101b20 0x7fc86c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 -- 172.21.15.3:0/2147305877 >> 172.21.15.3:0/2147305877 conn(0x7fc86c0fbea0 msgr2=0x7fc86c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 -- 172.21.15.3:0/2147305877 shutdown_connections 2023-08-11T14:03:14.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.023+0000 7fc873fff700 1 -- 172.21.15.3:0/2147305877 wait complete. 2023-08-11T14:03:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.024+0000 7fc873fff700 1 Processor -- start 2023-08-11T14:03:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.024+0000 7fc873fff700 1 -- start start 2023-08-11T14:03:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.024+0000 7fc873fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c100930 0x7fc86c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc873fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc873fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c1024c0 0x7fc86c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc873fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0739f0 con 0x7fc86c101b20 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc873fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c073b60 con 0x7fc86c100930 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc873fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1a0730 con 0x7fc86c1024c0 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46038/0 (socket says 172.21.15.3:46038) 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 -- 172.21.15.3:0/1664423150 learned_addr learned my addr 172.21.15.3:0/1664423150 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc871d9b700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c100930 0x7fc86c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87259c700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c1024c0 0x7fc86c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 -- 172.21.15.3:0/1664423150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c1024c0 msgr2=0x7fc86c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c1024c0 0x7fc86c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 -- 172.21.15.3:0/1664423150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c100930 msgr2=0x7fc86c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c100930 0x7fc86c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.025+0000 7fc87159a700 1 -- 172.21.15.3:0/1664423150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680095f0 con 0x7fc86c101b20 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc87159a700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9a0 tx=0x7fc85c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc862ffd700 1 -- 172.21.15.3:0/1664423150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c009790 con 0x7fc86c101b20 2023-08-11T14:03:14.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc862ffd700 1 -- 172.21.15.3:0/1664423150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85c010430 con 0x7fc86c101b20 2023-08-11T14:03:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c1a0a10 con 0x7fc86c101b20 2023-08-11T14:03:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc862ffd700 1 -- 172.21.15.3:0/1664423150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c018680 con 0x7fc86c101b20 2023-08-11T14:03:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.026+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1a0f70 con 0x7fc86c101b20 2023-08-11T14:03:14.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.029+0000 7fc862ffd700 1 -- 172.21.15.3:0/1664423150 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc85c0105a0 con 0x7fc86c101b20 2023-08-11T14:03:14.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.031+0000 7fc88797ab80 1 Processor -- start 2023-08-11T14:03:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 -- start start 2023-08-11T14:03:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc862ffd700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 0x7fc8580a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc862ffd700 1 -- 172.21.15.3:0/1664423150 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc85c014070 con 0x7fc86c101b20 2023-08-11T14:03:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc87259c700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 0x7fc8580a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001dd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001dd8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001d2f5e0 con 0x55f001dd73e0 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001d2cf00 con 0x55f001dd6a80 2023-08-11T14:03:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc88797ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001d287f0 con 0x55f001dd7da0 2023-08-11T14:03:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc871d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc871d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:34070/0 (socket says 172.21.15.3:34070) 2023-08-11T14:03:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc871d9b700 1 -- 172.21.15.3:0/4132200091 learned_addr learned my addr 172.21.15.3:0/4132200091 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.032+0000 7fc87159a700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001dd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc87259c700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001dd8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 -- 172.21.15.3:0/4132200091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 msgr2=0x55f001dd8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001dd8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 -- 172.21.15.3:0/4132200091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 msgr2=0x55f001dd7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001dd7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 -- 172.21.15.3:0/4132200091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f001ba85e0 con 0x55f001dd6a80 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc87259c700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001dd8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc87259c700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 0x7fc8580a2490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc86c074d30 tx=0x7fc864009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc871d9b700 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099c0 tx=0x7fc868005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbeee0820ff34022 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc857fff700 1 -- 172.21.15.3:0/4132200091 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800ba10 con 0x55f001dd6a80 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc857fff700 1 -- 172.21.15.3:0/4132200091 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc86801e040 con 0x55f001dd6a80 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.033+0000 7fc857fff700 1 -- 172.21.15.3:0/4132200091 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868003b50 con 0x55f001dd6a80 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 -- 172.21.15.3:0/4132200091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 msgr2=0x55f001dd6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8680099c0 tx=0x7fc868005ee0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 -- 172.21.15.3:0/4132200091 shutdown_connections 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001dd8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001dd6ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 --2- 172.21.15.3:0/4132200091 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001dd7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 -- 172.21.15.3:0/4132200091 >> 172.21.15.3:0/4132200091 conn(0x55f001d03900 msgr2=0x55f001dd13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 -- 172.21.15.3:0/4132200091 shutdown_connections 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.034+0000 7fc88797ab80 1 -- 172.21.15.3:0/4132200091 wait complete. 2023-08-11T14:03:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.035+0000 7fc88797ab80 1 -- start start 2023-08-11T14:03:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.035+0000 7fc88797ab80 1 Processor -- start 2023-08-11T14:03:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001de00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 0x55f001de05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001cc6a70 con 0x55f001dd7da0 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001cc6770 con 0x55f001dd6a80 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001baa8d0 con 0x55f001dd73e0 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc88797ab80 10 client.0 _ll_register_callbacks cb 0x55f001cc9a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87159a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 0x55f001de05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46048/0 (socket says 172.21.15.3:46048) 2023-08-11T14:03:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 -- 172.21.15.3:0/683077068 learned_addr learned my addr 172.21.15.3:0/683077068 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc871d9b700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001de00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 -- 172.21.15.3:0/683077068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 msgr2=0x55f001de05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 0x55f001de05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 -- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 msgr2=0x55f001de00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001de00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87259c700 1 -- 172.21.15.3:0/683077068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1096c0 con 0x55f001dd7da0 2023-08-11T14:03:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc87159a700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 0x55f001de05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.036+0000 7fc871d9b700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001de00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc87259c700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc864019ec0 tx=0x7fc864004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec38edabea177c29 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc8557fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400e040 con 0x55f001dd7da0 2023-08-11T14:03:14.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f001dc9a40 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc864003c00 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc864017e70 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc88797ab80 10 client.15201 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.037+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f001b54d90 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f001dc9a40 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc8640213f0 con 0x55f001dd7da0 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 10 client.15201 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 10 client.15201 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 20 client.15201 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001debd60 0x55f001dea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 4 client.15201 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f001da5490 con 0x55f001debd60 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.038+0000 7fc871d9b700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001debd60 0x55f001dea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.039+0000 7fc871d9b700 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001debd60 0x55f001dea780 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca298bc2b28a4d94 server_cookie=73386bc4ad7672bb in_seq=0 out_seq=0 2023-08-11T14:03:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.039+0000 7fc8557fa700 10 client.15201 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc8557fa700 1 -- 172.21.15.3:0/683077068 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f001da5490 con 0x55f001debd60 2023-08-11T14:03:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc8557fa700 10 client.15201 handle_command_reply: tid=0 2023-08-11T14:03:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 client.15201 shutdown 2023-08-11T14:03:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 msgr2=0x55f001de7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc864019ec0 tx=0x7fc864004b40 comp rx=0 tx=0).stop 2023-08-11T14:03:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 shutdown_connections 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd7da0 0x55f001de7620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001debd60 0x55f001dea780 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001de00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 --2- 172.21.15.3:0/683077068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd73e0 0x55f001de05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc8557fa700 0 client.15201 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 >> 172.21.15.3:0/683077068 conn(0x55f001d03900 msgr2=0x55f001dcbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 shutdown_connections 2023-08-11T14:03:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 1 -- 172.21.15.3:0/683077068 wait complete. 2023-08-11T14:03:14.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.040+0000 7fc88797ab80 20 client.15201 trim_cache size 0 max 16384 2023-08-11T14:03:14.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 Processor -- start 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 -- start start 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001df46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001cf1ef0 con 0x55f001dd73e0 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001ce5480 con 0x55f001dd6a80 2023-08-11T14:03:14.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc88797ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001dd2a90 con 0x55f001dd7da0 2023-08-11T14:03:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc871d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc871d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:34086/0 (socket says 172.21.15.3:34086) 2023-08-11T14:03:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc871d9b700 1 -- 172.21.15.3:0/3829742140 learned_addr learned my addr 172.21.15.3:0/3829742140 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.059+0000 7fc87159a700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc87259c700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001df46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 -- 172.21.15.3:0/3829742140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 msgr2=0x55f001df46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001df46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 -- 172.21.15.3:0/3829742140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 msgr2=0x55f001ddb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 -- 172.21.15.3:0/3829742140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f001b3dc10 con 0x55f001dd6a80 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc87259c700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001df46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc871d9b700 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc868024000 tx=0x7fc868009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166e00ec4645a286 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc860ff9700 1 -- 172.21.15.3:0/3829742140 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680037d0 con 0x55f001dd6a80 2023-08-11T14:03:14.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc860ff9700 1 -- 172.21.15.3:0/3829742140 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc868031070 con 0x55f001dd6a80 2023-08-11T14:03:14.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.060+0000 7fc860ff9700 1 -- 172.21.15.3:0/3829742140 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868005040 con 0x55f001dd6a80 2023-08-11T14:03:14.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- 172.21.15.3:0/3829742140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 msgr2=0x55f001ddac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc868024000 tx=0x7fc868009fd0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- 172.21.15.3:0/3829742140 shutdown_connections 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001df46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddac60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 --2- 172.21.15.3:0/3829742140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- 172.21.15.3:0/3829742140 >> 172.21.15.3:0/3829742140 conn(0x55f001d03900 msgr2=0x55f001ddee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- 172.21.15.3:0/3829742140 shutdown_connections 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- 172.21.15.3:0/3829742140 wait complete. 2023-08-11T14:03:14.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.061+0000 7fc88797ab80 1 -- start start 2023-08-11T14:03:14.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 Processor -- start 2023-08-11T14:03:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001ddbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001df4950 con 0x55f001dd73e0 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001df4ac0 con 0x55f001dd6a80 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f001ddc120 con 0x55f001dd7da0 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc88797ab80 10 client.0 _ll_register_callbacks cb 0x7fc864019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc87159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc87159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46056/0 (socket says 172.21.15.3:46056) 2023-08-11T14:03:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc87159a700 1 -- 172.21.15.3:0/2857156782 learned_addr learned my addr 172.21.15.3:0/2857156782 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc871d9b700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.062+0000 7fc87259c700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001ddbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 -- 172.21.15.3:0/2857156782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 msgr2=0x55f001ddbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001ddbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 -- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 msgr2=0x55f001ddd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 -- 172.21.15.3:0/2857156782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f001ba85e0 con 0x55f001dd73e0 2023-08-11T14:03:14.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc87159a700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc85c098d50 tx=0x7fc85c0c8d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f8b75889b7f45b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc8557fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c09cbf0 con 0x55f001dd73e0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc86801e040 con 0x55f001dd73e0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc85c0a0bd0 con 0x55f001dd73e0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc85c010880 con 0x55f001dd73e0 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc88797ab80 10 client.15207 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:14.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.063+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f001b7b200 con 0x55f001dd73e0 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc85c07b070 con 0x55f001dd73e0 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc85c0b5030 con 0x55f001dd73e0 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 10 client.15207 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 10 client.15207 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 20 client.15207 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001e43e80 0x55f001e402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 4 client.15207 mds_command: new command op to 24569 tid=0[{"prefix": "status"}] 2023-08-11T14:03:14.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f001d217d0 con 0x55f001e43e80 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.064+0000 7fc871d9b700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001e43e80 0x55f001e402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc871d9b700 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001e43e80 0x55f001e402c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20684b523cb1603e server_cookie=558db6c33f7e3d5e in_seq=0 out_seq=0 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc8557fa700 10 client.15207 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc8557fa700 1 -- 172.21.15.3:0/2857156782 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55f001d217d0 con 0x55f001e43e80 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc8557fa700 10 client.15207 handle_command_reply: tid=0 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 client.15207 shutdown 2023-08-11T14:03:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 msgr2=0x55f001ddd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc85c098d50 tx=0x7fc85c0c8d40 comp rx=0 tx=0).stop 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 shutdown_connections 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f001dd7da0 0x55f001ddbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f001e43e80 0x55f001e402c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f001dd6a80 0x55f001ddd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 --2- 172.21.15.3:0/2857156782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001dd73e0 0x55f001ddd650 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc8557fa700 0 client.15207 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:14.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 >> 172.21.15.3:0/2857156782 conn(0x55f001d03900 msgr2=0x55f001ddac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 shutdown_connections 2023-08-11T14:03:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 1 -- 172.21.15.3:0/2857156782 wait complete. 2023-08-11T14:03:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.065+0000 7fc88797ab80 20 client.15207 trim_cache size 0 max 16384 2023-08-11T14:03:14.076 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:14.076 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24569, 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 100.736360319, 2023-08-11T14:03:14.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:14.078 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:14.078 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:14.078 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 106.973930502 2023-08-11T14:03:14.078 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 msgr2=0x7fc8580a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 0x7fc8580a2490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc86c074d30 tx=0x7fc864009920 comp rx=0 tx=0).stop 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 msgr2=0x7fc86c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9a0 tx=0x7fc85c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 shutdown_connections 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc86c1024c0 0x7fc86c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc85809ffd0 0x7fc8580a2490 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc86c100930 0x7fc86c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 --2- 172.21.15.3:0/1664423150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101b20 0x7fc86c072d10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 >> 172.21.15.3:0/1664423150 conn(0x7fc86c0fbea0 msgr2=0x7fc86c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 shutdown_connections 2023-08-11T14:03:14.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:14.066+0000 7fc873fff700 1 -- 172.21.15.3:0/1664423150 wait complete. 2023-08-11T14:03:14.086 INFO:tasks.cephadm:Setup complete, yielding 2023-08-11T14:03:14.086 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-11T14:03:14.096 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:03:14.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph orch status' 2023-08-11T14:03:14.828 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:15.667 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:15 smithi003 ceph-mon[97195]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:15 smithi103 ceph-mon[130434]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:15 smithi187 ceph-mon[130973]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 2 B/s, 0 objects/s recovering 2023-08-11T14:03:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 -- 172.21.15.3:0/2857684873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6001009b0 msgr2=0x7fd600100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 --2- 172.21.15.3:0/2857684873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6001009b0 0x7fd600100dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0099c0 tx=0x7fd5fc009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 -- 172.21.15.3:0/2857684873 shutdown_connections 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 --2- 172.21.15.3:0/2857684873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 0x7fd600108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 --2- 172.21.15.3:0/2857684873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6001009b0 0x7fd600100dd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 --2- 172.21.15.3:0/2857684873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd600101bb0 0x7fd600102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 -- 172.21.15.3:0/2857684873 >> 172.21.15.3:0/2857684873 conn(0x7fd6000fbf10 msgr2=0x7fd6000fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.367+0000 7fd607a7f700 1 -- 172.21.15.3:0/2857684873 shutdown_connections 2023-08-11T14:03:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.368+0000 7fd607a7f700 1 -- 172.21.15.3:0/2857684873 wait complete. 2023-08-11T14:03:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 Processor -- start 2023-08-11T14:03:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 -- start start 2023-08-11T14:03:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600101bb0 0x7fd600196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 0x7fd60019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60019e230 con 0x7fd600101bb0 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60019e370 con 0x7fd600102550 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd607a7f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60019e500 con 0x7fd6001009b0 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.369+0000 7fd60581b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34218/0 (socket says 172.21.15.3:34218) 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 -- 172.21.15.3:0/4092675098 learned_addr learned my addr 172.21.15.3:0/4092675098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60601c700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 0x7fd60019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60501a700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600101bb0 0x7fd600196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 -- 172.21.15.3:0/4092675098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 msgr2=0x7fd60019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 0x7fd60019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 -- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600101bb0 msgr2=0x7fd600196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600101bb0 0x7fd600196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 -- 172.21.15.3:0/4092675098 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0095f0 con 0x7fd6001009b0 2023-08-11T14:03:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.370+0000 7fd60581b700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0098e0 tx=0x7fd5fc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.372+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc01d070 con 0x7fd6001009b0 2023-08-11T14:03:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.372+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5fc005030 con 0x7fd6001009b0 2023-08-11T14:03:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.372+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc004a90 con 0x7fd6001009b0 2023-08-11T14:03:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.372+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60019e780 con 0x7fd6001009b0 2023-08-11T14:03:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.373+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60019eb90 con 0x7fd6001009b0 2023-08-11T14:03:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.374+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd600107fd0 con 0x7fd6001009b0 2023-08-11T14:03:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.375+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd5fc0051a0 con 0x7fd6001009b0 2023-08-11T14:03:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.377+0000 7fd5f6ffd700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 0x7fd5ec0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.377+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd5fc0cc2d0 con 0x7fd6001009b0 2023-08-11T14:03:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.377+0000 7fd60501a700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 0x7fd5ec0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.378+0000 7fd60501a700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 0x7fd5ec0a2490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009c60 tx=0x7fd5f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.379+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd5fc095d70 con 0x7fd6001009b0 2023-08-11T14:03:16.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.609+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd60002cd30 con 0x7fd5ec09ffd0 2023-08-11T14:03:16.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.610+0000 7fd5f6ffd700 1 -- 172.21.15.3:0/4092675098 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd60002cd30 con 0x7fd5ec09ffd0 2023-08-11T14:03:16.611 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2023-08-11T14:03:16.612 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2023-08-11T14:03:16.612 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2023-08-11T14:03:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.612+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 msgr2=0x7fd5ec0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.612+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 0x7fd5ec0a2490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009c60 tx=0x7fd5f0009260 comp rx=0 tx=0).stop 2023-08-11T14:03:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.612+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 msgr2=0x7fd6001966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.612+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0098e0 tx=0x7fd5fc00f690 comp rx=0 tx=0).stop 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 shutdown_connections 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd5ec09ffd0 0x7fd5ec0a2490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600102550 0x7fd60019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6001009b0 0x7fd6001966a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 --2- 172.21.15.3:0/4092675098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd600101bb0 0x7fd600196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 >> 172.21.15.3:0/4092675098 conn(0x7fd6000fbf10 msgr2=0x7fd600105110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 shutdown_connections 2023-08-11T14:03:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:16.613+0000 7fd607a7f700 1 -- 172.21.15.3:0/4092675098 wait complete. 2023-08-11T14:03:17.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph orch ps' 2023-08-11T14:03:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:17 smithi103 ceph-mon[130434]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 86 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2023-08-11T14:03:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:17 smithi003 ceph-mon[97195]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 86 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2023-08-11T14:03:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:17 smithi187 ceph-mon[130973]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 86 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2023-08-11T14:03:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:18.429 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:18 smithi003 ceph-mon[97195]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:18.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:18 smithi103 ceph-mon[130434]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:18.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:18 smithi187 ceph-mon[130973]: from='client.25252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 -- 172.21.15.3:0/3127605656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70108880 msgr2=0x7f2d7010ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/3127605656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70108880 0x7f2d7010ad10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b0a0 tx=0x7f2d6800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 -- 172.21.15.3:0/3127605656 shutdown_connections 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/3127605656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70108880 0x7f2d7010ad10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/3127605656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70074950 0x7f2d70072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/3127605656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70073380 0x7f2d70073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 -- 172.21.15.3:0/3127605656 >> 172.21.15.3:0/3127605656 conn(0x7f2d700fbf10 msgr2=0x7f2d700fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 -- 172.21.15.3:0/3127605656 shutdown_connections 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.469+0000 7f2d74f6c700 1 -- 172.21.15.3:0/3127605656 wait complete. 2023-08-11T14:03:19.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 Processor -- start 2023-08-11T14:03:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 -- start start 2023-08-11T14:03:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70074950 0x7f2d70198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70108880 0x7f2d7019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:19.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701a0570 con 0x7f2d70073380 2023-08-11T14:03:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.470+0000 7f2d74f6c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701a06b0 con 0x7f2d70108880 2023-08-11T14:03:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d74f6c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701a0840 con 0x7f2d70074950 2023-08-11T14:03:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46092/0 (socket says 172.21.15.3:46092) 2023-08-11T14:03:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 -- 172.21.15.3:0/432407967 learned_addr learned my addr 172.21.15.3:0/432407967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6dd9b700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70074950 0x7f2d70198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6ed9d700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70108880 0x7f2d7019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 -- 172.21.15.3:0/432407967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70074950 msgr2=0x7f2d70198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70074950 0x7f2d70198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 -- 172.21.15.3:0/432407967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70108880 msgr2=0x7f2d7019ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70108880 0x7f2d7019ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 -- 172.21.15.3:0/432407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68009d00 con 0x7f2d70073380 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.471+0000 7f2d6e59c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9a0 tx=0x7f2d6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.472+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60009790 con 0x7f2d70073380 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.472+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d6000c810 con 0x7f2d70073380 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.472+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600054c0 con 0x7f2d70073380 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.472+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d701a0b20 con 0x7f2d70073380 2023-08-11T14:03:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.472+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d701a0fd0 con 0x7f2d70073380 2023-08-11T14:03:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.473+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70106f30 con 0x7f2d70073380 2023-08-11T14:03:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.475+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2d6000cc80 con 0x7f2d70073380 2023-08-11T14:03:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.476+0000 7f2d5f7fe700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 0x7f2d580a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.476+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2d60014070 con 0x7f2d70073380 2023-08-11T14:03:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.476+0000 7f2d6dd9b700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 0x7f2d580a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.477+0000 7f2d6dd9b700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 0x7f2d580a2410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2d70074680 tx=0x7f2d64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:19.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.480+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2d600951f0 con 0x7f2d70073380 2023-08-11T14:03:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:19 smithi003 ceph-mon[97195]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-08-11T14:03:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:19 smithi103 ceph-mon[130434]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-08-11T14:03:19.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.698+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d7002cee0 con 0x7f2d5809ff50 2023-08-11T14:03:19.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.711+0000 7f2d5f7fe700 1 -- 172.21.15.3:0/432407967 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f2d7002cee0 con 0x7f2d5809ff50 2023-08-11T14:03:19.713 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-08-11T14:03:19.713 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (115s) - 114s 16.2M - 18.0.0-5437-g92684807 a2667c7aa6ee 498c6e8d7749 2023-08-11T14:03:19.713 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi103 running (113s) - 112s 33.4M - 18.0.0-5437-g92684807 a2667c7aa6ee ece46451161c 2023-08-11T14:03:19.713 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi187 running (2m) - 2m 21.3M - 18.0.0-5437-g92684807 a2667c7aa6ee ddd25877ae6e 2023-08-11T14:03:19.713 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (2m) - 2m 13.8M - 18.0.0-5437-g92684807 a2667c7aa6ee 4ce7b2a37b2b 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi103 running (2m) - 2m 21.3M - 18.0.0-5437-g92684807 a2667c7aa6ee cb56ab98f7e5 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi187 running (2m) - 2m 18.4M - 18.0.0-5437-g92684807 a2667c7aa6ee 3197c4da4a40 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (118s) - 118s 16.2M - 18.0.0-5437-g92684807 a2667c7aa6ee 80e7fc13fac6 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi103 running (2m) - 2m 19.0M - 18.0.0-5437-g92684807 a2667c7aa6ee eb8c24be5d42 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi187 running (2m) - 2m 19.0M - 18.0.0-5437-g92684807 a2667c7aa6ee debec6b5e694 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (2m) - 2m 13.8M - 18.0.0-5437-g92684807 a2667c7aa6ee 4b5d51e45ded 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi103 running (2m) - 2m 17.4M - 18.0.0-5437-g92684807 a2667c7aa6ee 9f2be860fd40 2023-08-11T14:03:19.714 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi187 running (2m) - 2m 16.5M - 18.0.0-5437-g92684807 a2667c7aa6ee 05379e5872a5 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (9m) - 9m 527M - 18.0.0-5437-g92684807 a2667c7aa6ee af89c8d4f916 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi103 *:8443,8765 running (7m) - 7m 451M - 18.0.0-5437-g92684807 a2667c7aa6ee 10a2895cfcbc 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi187 *:8443,8765 running (7m) - 7m 452M - 18.0.0-5437-g92684807 a2667c7aa6ee b2fbdd1461f7 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (9m) - 9m 94.9M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee da8973229e7e 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi103 running (7m) - 7m 64.7M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee e2ad3ee33b28 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi187 running (7m) - 7m 66.2M 2048M 18.0.0-5437-g92684807 a2667c7aa6ee b9de1bdbe0e4 2023-08-11T14:03:19.715 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (6m) - 6m 71.2M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee 2bff5f937eef 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi103 running (6m) - 6m 77.4M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 56d39bff036a 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi187 running (6m) - 6m 78.7M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 638e36e879d5 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (5m) - 5m 66.8M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee e115eee8c851 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi103 running (5m) - 5m 64.1M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 79985bcda936 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi187 running (4m) - 4m 69.9M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee a98a1a9e0b97 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (4m) - 4m 62.9M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee ddcab6b233f5 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi103 running (4m) - 4m 67.5M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 0a22399d041a 2023-08-11T14:03:19.716 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi187 running (3m) - 3m 63.4M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee c59583865c61 2023-08-11T14:03:19.717 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (3m) - 3m 62.0M 4318M 18.0.0-5437-g92684807 a2667c7aa6ee dee62872d305 2023-08-11T14:03:19.717 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi103 running (3m) - 3m 63.6M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee 93b4b73a48ab 2023-08-11T14:03:19.717 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi187 running (2m) - 2m 56.3M 4292M 18.0.0-5437-g92684807 a2667c7aa6ee b1a8d1378e2a 2023-08-11T14:03:19.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.713+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 msgr2=0x7f2d580a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:19.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.713+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 0x7f2d580a2410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2d70074680 tx=0x7f2d64009260 comp rx=0 tx=0).stop 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.713+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 msgr2=0x7f2d701989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.713+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9a0 tx=0x7f2d6000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 shutdown_connections 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d5809ff50 0x7f2d580a2410 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d70108880 0x7f2d7019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70073380 0x7f2d701989e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 --2- 172.21.15.3:0/432407967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d70074950 0x7f2d70198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:19.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 >> 172.21.15.3:0/432407967 conn(0x7f2d700fbf10 msgr2=0x7f2d700fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:19.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 shutdown_connections 2023-08-11T14:03:19.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:19.714+0000 7f2d74f6c700 1 -- 172.21.15.3:0/432407967 wait complete. 2023-08-11T14:03:19.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:19 smithi187 ceph-mon[130973]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-08-11T14:03:20.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph orch ls' 2023-08-11T14:03:20.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:20 smithi003 ceph-mon[97195]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:20.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:20 smithi003 ceph-mon[97195]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-08-11T14:03:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:20 smithi103 ceph-mon[130434]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:20 smithi103 ceph-mon[130434]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-08-11T14:03:20.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:20 smithi187 ceph-mon[130973]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:20.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:20 smithi187 ceph-mon[130973]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-08-11T14:03:20.897 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:22.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 -- 172.21.15.3:0/683781684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 msgr2=0x7fbbd4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:22.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 --2- 172.21.15.3:0/683781684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4100dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40099c0 tx=0x7fbbc4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 -- 172.21.15.3:0/683781684 shutdown_connections 2023-08-11T14:03:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 --2- 172.21.15.3:0/683781684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4102550 0x7fbbd4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 --2- 172.21.15.3:0/683781684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4100dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 --2- 172.21.15.3:0/683781684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 -- 172.21.15.3:0/683781684 >> 172.21.15.3:0/683781684 conn(0x7fbbd40fbf10 msgr2=0x7fbbd40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.423+0000 7fbbda129700 1 -- 172.21.15.3:0/683781684 shutdown_connections 2023-08-11T14:03:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.424+0000 7fbbda129700 1 -- 172.21.15.3:0/683781684 wait complete. 2023-08-11T14:03:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 Processor -- start 2023-08-11T14:03:22.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 -- start start 2023-08-11T14:03:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:22.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4102550 0x7fbbd419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419e200 con 0x7fbbd41009b0 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419e340 con 0x7fbbd4102550 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbda129700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419e4d0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.425+0000 7fbbd2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44206/0 (socket says 172.21.15.3:44206) 2023-08-11T14:03:22.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 -- 172.21.15.3:0/249645982 learned_addr learned my addr 172.21.15.3:0/249645982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd3fff700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4102550 0x7fbbd419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 -- 172.21.15.3:0/249645982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4102550 msgr2=0x7fbbd419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4102550 0x7fbbd419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 -- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 msgr2=0x7fbbd4196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 -- 172.21.15.3:0/249645982 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc40095f0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd2ffd700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e9d0 tx=0x7fbbc800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8009790 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.426+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd419e7b0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.427+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc800ee70 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.427+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8010ea0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.427+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd419ed00 con 0x7fbbd4101bb0 2023-08-11T14:03:22.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.428+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd410c1b0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.430+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbbc800cbd0 con 0x7fbbd4101bb0 2023-08-11T14:03:22.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.431+0000 7fbbd0ff9700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 0x7fbbbc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.431+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbbc8014070 con 0x7fbbd4101bb0 2023-08-11T14:03:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.431+0000 7fbbd37fe700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 0x7fbbbc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.432+0000 7fbbd37fe700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 0x7fbbbc0a2410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009990 tx=0x7fbbc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:22.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.433+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbbc8095220 con 0x7fbbd4101bb0 2023-08-11T14:03:22.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.651+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd419ef20 con 0x7fbbbc09ff50 2023-08-11T14:03:22.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.656+0000 7fbbd0ff9700 1 -- 172.21.15.3:0/249645982 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fbbd419ef20 con 0x7fbbbc09ff50 2023-08-11T14:03:22.704 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-08-11T14:03:22.704 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 - 2m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-08-11T14:03:22.704 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 - 7m smithi003=x;smithi103=y;smithi187=z;count:3 2023-08-11T14:03:22.705 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 - 7m smithi003:172.21.15.3=a;smithi103:172.21.15.103=b;smithi187:172.21.15.187=c;count:3 2023-08-11T14:03:22.705 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 - - 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 msgr2=0x7fbbbc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 0x7fbbbc0a2410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009990 tx=0x7fbbc4005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 msgr2=0x7fbbd4196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800e9d0 tx=0x7fbbc800ece0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 shutdown_connections 2023-08-11T14:03:22.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbbbc09ff50 0x7fbbbc0a2410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.658+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd4102550 0x7fbbd419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.659+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd41009b0 0x7fbbd4196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.659+0000 7fbbda129700 1 --2- 172.21.15.3:0/249645982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbd4101bb0 0x7fbbd4196bb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.659+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 >> 172.21.15.3:0/249645982 conn(0x7fbbd40fbf10 msgr2=0x7fbbd40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.659+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 shutdown_connections 2023-08-11T14:03:22.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:22.659+0000 7fbbda129700 1 -- 172.21.15.3:0/249645982 wait complete. 2023-08-11T14:03:23.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph orch host ls' 2023-08-11T14:03:23.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:23 smithi003 ceph-mon[97195]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:23 smithi103 ceph-mon[130434]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:23.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:23 smithi187 ceph-mon[130973]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:23.932 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:24.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:24 smithi003 ceph-mon[97195]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:24.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:24 smithi103 ceph-mon[130434]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:24.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:24 smithi187 ceph-mon[130973]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:25.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 -- 172.21.15.3:0/1941597141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798074950 msgr2=0x7ff798072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:25.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/1941597141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798074950 0x7ff798072db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff788009990 tx=0x7ff788009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 -- 172.21.15.3:0/1941597141 shutdown_connections 2023-08-11T14:03:25.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/1941597141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798108880 0x7ff79810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/1941597141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798074950 0x7ff798072db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/1941597141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798073380 0x7ff798073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 -- 172.21.15.3:0/1941597141 >> 172.21.15.3:0/1941597141 conn(0x7ff7980fbf10 msgr2=0x7ff7980fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:25.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 -- 172.21.15.3:0/1941597141 shutdown_connections 2023-08-11T14:03:25.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.616+0000 7ff79cfa9700 1 -- 172.21.15.3:0/1941597141 wait complete. 2023-08-11T14:03:25.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.618+0000 7ff79cfa9700 1 Processor -- start 2023-08-11T14:03:25.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.618+0000 7ff79cfa9700 1 -- start start 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.618+0000 7ff79cfa9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.618+0000 7ff79cfa9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798074950 0x7ff798198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.618+0000 7ff79cfa9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798108880 0x7ff79819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79cfa9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a0480 con 0x7ff798074950 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79cfa9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a05c0 con 0x7ff798073380 2023-08-11T14:03:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79cfa9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7981a0750 con 0x7ff798108880 2023-08-11T14:03:25.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:25.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57350/0 (socket says 172.21.15.3:57350) 2023-08-11T14:03:25.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff795d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798074950 0x7ff798198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:25.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 -- 172.21.15.3:0/4265820450 learned_addr learned my addr 172.21.15.3:0/4265820450 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff796d9d700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798108880 0x7ff79819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 -- 172.21.15.3:0/4265820450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798108880 msgr2=0x7ff79819feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798108880 0x7ff79819feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 -- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798074950 msgr2=0x7ff798198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798074950 0x7ff798198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.619+0000 7ff79659c700 1 -- 172.21.15.3:0/4265820450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880095f0 con 0x7ff798073380 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff79659c700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff788004710 tx=0x7ff788009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78801d070 con 0x7ff798073380 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff788021410 con 0x7ff798073380 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7981a09d0 con 0x7ff798073380 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788017650 con 0x7ff798073380 2023-08-11T14:03:25.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.620+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981a0de0 con 0x7ff798073380 2023-08-11T14:03:25.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.621+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798106f30 con 0x7ff798073380 2023-08-11T14:03:25.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.623+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff788021580 con 0x7ff798073380 2023-08-11T14:03:25.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.624+0000 7ff7877fe700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 0x7ff7800a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:25.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.625+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff7880ccff0 con 0x7ff798073380 2023-08-11T14:03:25.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.625+0000 7ff795d9b700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 0x7ff7800a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:25.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.625+0000 7ff795d9b700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 0x7ff7800a2410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff798074680 tx=0x7ff78c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:25.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.627+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff788096ad0 con 0x7ff798073380 2023-08-11T14:03:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:25 smithi103 ceph-mon[130434]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:25 smithi003 ceph-mon[97195]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:25.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:25 smithi187 ceph-mon[130973]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:25.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.841+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff79802cee0 con 0x7ff78009ff50 2023-08-11T14:03:25.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.842+0000 7ff7877fe700 1 -- 172.21.15.3:0/4265820450 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ff79802cee0 con 0x7ff78009ff50 2023-08-11T14:03:25.843 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2023-08-11T14:03:25.843 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2023-08-11T14:03:25.844 INFO:teuthology.orchestra.run.smithi003.stdout:smithi103 172.21.15.103 2023-08-11T14:03:25.844 INFO:teuthology.orchestra.run.smithi003.stdout:smithi187 172.21.15.187 2023-08-11T14:03:25.844 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2023-08-11T14:03:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 msgr2=0x7ff7800a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:25.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 0x7ff7800a2410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff798074680 tx=0x7ff78c009260 comp rx=0 tx=0).stop 2023-08-11T14:03:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 msgr2=0x7ff7981988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff788004710 tx=0x7ff788009580 comp rx=0 tx=0).stop 2023-08-11T14:03:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 shutdown_connections 2023-08-11T14:03:25.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff798108880 0x7ff79819feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff78009ff50 0x7ff7800a2410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff798073380 0x7ff7981988f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 --2- 172.21.15.3:0/4265820450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff798074950 0x7ff798198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 >> 172.21.15.3:0/4265820450 conn(0x7ff7980fbf10 msgr2=0x7ff7980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 shutdown_connections 2023-08-11T14:03:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:25.845+0000 7ff79cfa9700 1 -- 172.21.15.3:0/4265820450 wait complete. 2023-08-11T14:03:26.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph orch device ls' 2023-08-11T14:03:27.097 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:27.608 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:27 smithi003 ceph-mon[97195]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:27.608 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:27 smithi003 ceph-mon[97195]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:27 smithi103 ceph-mon[130434]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:27.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:27 smithi103 ceph-mon[130434]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:27.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:27 smithi187 ceph-mon[130973]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:27.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:27 smithi187 ceph-mon[130973]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 -- 172.21.15.3:0/3009512834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c101bb0 msgr2=0x7f796c102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 --2- 172.21.15.3:0/3009512834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c101bb0 0x7f796c102010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79600099c0 tx=0x7f7960009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 -- 172.21.15.3:0/3009512834 shutdown_connections 2023-08-11T14:03:28.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 --2- 172.21.15.3:0/3009512834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c102550 0x7f796c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 --2- 172.21.15.3:0/3009512834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 --2- 172.21.15.3:0/3009512834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c101bb0 0x7f796c102010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 -- 172.21.15.3:0/3009512834 >> 172.21.15.3:0/3009512834 conn(0x7f796c0fbf10 msgr2=0x7f796c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.569+0000 7f7970fa2700 1 -- 172.21.15.3:0/3009512834 shutdown_connections 2023-08-11T14:03:28.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.570+0000 7f7970fa2700 1 -- 172.21.15.3:0/3009512834 wait complete. 2023-08-11T14:03:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.570+0000 7f7970fa2700 1 Processor -- start 2023-08-11T14:03:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 -- start start 2023-08-11T14:03:28.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 0x7f796c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c19e210 con 0x7f796c102550 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c19e350 con 0x7f796c1009b0 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7970fa2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c19e4e0 con 0x7f796c101bb0 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f796ad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 0x7f796c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:28.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7969d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7969d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44240/0 (socket says 172.21.15.3:44240) 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f7969d9b700 1 -- 172.21.15.3:0/538111548 learned_addr learned my addr 172.21.15.3:0/538111548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.571+0000 7f796a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 -- 172.21.15.3:0/538111548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 msgr2=0x7f796c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 -- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 msgr2=0x7f796c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:28.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 0x7f796c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 -- 172.21.15.3:0/538111548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c009520 con 0x7f796c1009b0 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f7969d9b700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796ad9d700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 0x7f796c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f796a59c700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c196680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e9c0 tx=0x7f795c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00cb70 con 0x7f796c1009b0 2023-08-11T14:03:28.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600095f0 con 0x7f796c1009b0 2023-08-11T14:03:28.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f795c00ee60 con 0x7f796c1009b0 2023-08-11T14:03:28.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.572+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c018680 con 0x7f796c1009b0 2023-08-11T14:03:28.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.573+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c19eb80 con 0x7f796c1009b0 2023-08-11T14:03:28.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.574+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c108000 con 0x7f796c1009b0 2023-08-11T14:03:28.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.576+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f795c010430 con 0x7f796c1009b0 2023-08-11T14:03:28.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.577+0000 7f795b7fe700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 0x7f79540a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:28.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.577+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f795c014070 con 0x7f796c1009b0 2023-08-11T14:03:28.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.577+0000 7f7969d9b700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 0x7f79540a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:28.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.578+0000 7f7969d9b700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 0x7f79540a2410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7960000c00 tx=0x7f7960005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:28.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.579+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f795c095350 con 0x7f796c1009b0 2023-08-11T14:03:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.793+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f796c02cd20 con 0x7f795409ff50 2023-08-11T14:03:28.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.794+0000 7f795b7fe700 1 -- 172.21.15.3:0/538111548 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f796c02cd20 con 0x7f795409ff50 2023-08-11T14:03:28.795 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-08-11T14:03:28.795 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 3m ago LVM detected, locked 2023-08-11T14:03:28.795 INFO:teuthology.orchestra.run.smithi003.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago LVM detected, locked 2023-08-11T14:03:28.796 INFO:teuthology.orchestra.run.smithi003.stdout:smithi187 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MW400BGN 372G No 2m ago LVM detected, locked 2023-08-11T14:03:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 msgr2=0x7f79540a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:28.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 0x7f79540a2410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7960000c00 tx=0x7f7960005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 msgr2=0x7f796c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c196680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e9c0 tx=0x7f795c00ecd0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 shutdown_connections 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f796c102550 0x7f796c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f795409ff50 0x7f79540a2410 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c1009b0 0x7f796c196680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 --2- 172.21.15.3:0/538111548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c101bb0 0x7f796c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:28.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 >> 172.21.15.3:0/538111548 conn(0x7f796c0fbf10 msgr2=0x7f796c105140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:28.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 shutdown_connections 2023-08-11T14:03:28.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:28.797+0000 7f7970fa2700 1 -- 172.21.15.3:0/538111548 wait complete. 2023-08-11T14:03:29.295 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-08-11T14:03:29.329 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:03:29.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph fs dump' 2023-08-11T14:03:29.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:29 smithi003 ceph-mon[97195]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:29 smithi103 ceph-mon[130434]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:29 smithi187 ceph-mon[130973]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:30.063 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:30.594 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:30 smithi003 ceph-mon[97195]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:30 smithi103 ceph-mon[130434]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:30 smithi187 ceph-mon[130973]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-08-11T14:03:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 -- 172.21.15.3:0/849578231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 msgr2=0x7f3b34100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 --2- 172.21.15.3:0/849578231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b34100dd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b240099c0 tx=0x7f3b24009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 -- 172.21.15.3:0/849578231 shutdown_connections 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 --2- 172.21.15.3:0/849578231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34102550 0x7f3b34108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 --2- 172.21.15.3:0/849578231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b34100dd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 --2- 172.21.15.3:0/849578231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34101bb0 0x7f3b34102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 -- 172.21.15.3:0/849578231 >> 172.21.15.3:0/849578231 conn(0x7f3b340fbf10 msgr2=0x7f3b340fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 -- 172.21.15.3:0/849578231 shutdown_connections 2023-08-11T14:03:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.514+0000 7f3b38df6700 1 -- 172.21.15.3:0/849578231 wait complete. 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.515+0000 7f3b38df6700 1 Processor -- start 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.515+0000 7f3b38df6700 1 -- start start 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34101bb0 0x7f3b34198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34102550 0x7f3b3419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a0470 con 0x7f3b341009b0 2023-08-11T14:03:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a05b0 con 0x7f3b34101bb0 2023-08-11T14:03:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b38df6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341a0740 con 0x7f3b34102550 2023-08-11T14:03:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52494/0 (socket says 172.21.15.3:52494) 2023-08-11T14:03:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 -- 172.21.15.3:0/925585719 learned_addr learned my addr 172.21.15.3:0/925585719 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b31d9b700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34101bb0 0x7f3b34198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b32d9d700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34102550 0x7f3b3419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 -- 172.21.15.3:0/925585719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34102550 msgr2=0x7f3b3419fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34102550 0x7f3b3419fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.516+0000 7f3b3259c700 1 -- 172.21.15.3:0/925585719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34101bb0 msgr2=0x7f3b34198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b3259c700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34101bb0 0x7f3b34198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b3259c700 1 -- 172.21.15.3:0/925585719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b240095f0 con 0x7f3b341009b0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b3259c700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098e0 tx=0x7f3b2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2401d070 con 0x7f3b341009b0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b24005030 con 0x7f3b341009b0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b341a09c0 con 0x7f3b341009b0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24004a90 con 0x7f3b341009b0 2023-08-11T14:03:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.517+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b341a0dd0 con 0x7f3b341009b0 2023-08-11T14:03:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.519+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34192ff0 con 0x7f3b341009b0 2023-08-11T14:03:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.520+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f3b240051a0 con 0x7f3b341009b0 2023-08-11T14:03:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.522+0000 7f3b237fe700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 0x7f3b1c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.522+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3b240cc2f0 con 0x7f3b341009b0 2023-08-11T14:03:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.522+0000 7f3b31d9b700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 0x7f3b1c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.523+0000 7f3b31d9b700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 0x7f3b1c0a2410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009c60 tx=0x7f3b28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.526+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3b24095d90 con 0x7f3b341009b0 2023-08-11T14:03:31.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:31 smithi003 ceph-mon[97195]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:31 smithi103 ceph-mon[130434]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.777+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b3410c200 con 0x7f3b341009b0 2023-08-11T14:03:31.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.777+0000 7f3b237fe700 1 -- 172.21.15.3:0/925585719 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2971 (secure 0 0 0) 0x7f3b240993b0 con 0x7f3b341009b0 2023-08-11T14:03:31.778 INFO:teuthology.orchestra.run.smithi003.stdout:e27 2023-08-11T14:03:31.779 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-08-11T14:03:31.779 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T14:03:31.779 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2023-08-11T14:03:31.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 27 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-08-11T14:01:32.831322+0000 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2023-08-11T14:02:01.312375+0000 2023-08-11T14:03:31.780 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2023-08-11T14:03:31.781 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 5 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:in 0,1,2,3,4 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=24569,1=24532,2=24527,3=24511,4=24500} 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2023-08-11T14:03:31.782 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3,4] 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.187:6834/3236312344,v1:172.21.15.187:6835/3236312344] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.783 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{1:24532} state up:active seq 11 addr [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{1:24464} state up:standby-replay seq 1 addr [v2:172.21.15.103:6832/618362106,v1:172.21.15.103:6833/618362106] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{2:24527} state up:active seq 12 addr [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{2:24454} state up:standby-replay seq 1 addr [v2:172.21.15.187:6832/1102333992,v1:172.21.15.187:6833/1102333992] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{3:24511} state up:active seq 13 addr [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{3:14514} state up:standby-replay seq 1 addr [v2:172.21.15.3:6840/1195165407,v1:172.21.15.3:6841/1195165407] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{4:24500} state up:active seq 14 addr [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.784 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{4:14502} state up:standby-replay seq 1 addr [v2:172.21.15.3:6838/3729253320,v1:172.21.15.3:6839/3729253320] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:14433} state up:standby seq 1 addr [v2:172.21.15.3:6834/113676080,v1:172.21.15.3:6835/113676080] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.785 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:14445} state up:standby seq 1 addr [v2:172.21.15.3:6836/3880902111,v1:172.21.15.3:6837/3880902111] compat {c=[1],r=[1],i=[fff]}] 2023-08-11T14:03:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 msgr2=0x7f3b1c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 0x7f3b1c0a2410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009c60 tx=0x7f3b28009260 comp rx=0 tx=0).stop 2023-08-11T14:03:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 msgr2=0x7f3b341988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098e0 tx=0x7f3b2400f690 comp rx=0 tx=0).stop 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 shutdown_connections 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b34102550 0x7f3b3419fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3b341009b0 0x7f3b341988e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3b1c09ff50 0x7f3b1c0a2410 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 --2- 172.21.15.3:0/925585719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b34101bb0 0x7f3b34198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 >> 172.21.15.3:0/925585719 conn(0x7f3b340fbf10 msgr2=0x7f3b34107510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 shutdown_connections 2023-08-11T14:03:31.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:31.780+0000 7f3b38df6700 1 -- 172.21.15.3:0/925585719 wait complete. 2023-08-11T14:03:31.788 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:31.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:31 smithi187 ceph-mon[130973]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:32.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- bash -c 'ceph osd dump' 2023-08-11T14:03:32.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:32 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/925585719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T14:03:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:32 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/925585719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T14:03:32.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:32 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/925585719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-08-11T14:03:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:33.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:33 smithi003 ceph-mon[97195]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:33 smithi103 ceph-mon[130434]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:33.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:33 smithi187 ceph-mon[130973]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:34.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 -- 172.21.15.3:0/2885288634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0073380 msgr2=0x7f5dc0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2885288634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0073380 0x7f5dc0073800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009990 tx=0x7f5db0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 -- 172.21.15.3:0/2885288634 shutdown_connections 2023-08-11T14:03:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2885288634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0108880 0x7f5dc010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2885288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0074950 0x7f5dc0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2885288634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0073380 0x7f5dc0073800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 -- 172.21.15.3:0/2885288634 >> 172.21.15.3:0/2885288634 conn(0x7f5dc00fbf10 msgr2=0x7f5dc00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 -- 172.21.15.3:0/2885288634 shutdown_connections 2023-08-11T14:03:34.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.547+0000 7f5dc791b700 1 -- 172.21.15.3:0/2885288634 wait complete. 2023-08-11T14:03:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 Processor -- start 2023-08-11T14:03:34.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 -- start start 2023-08-11T14:03:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0073380 0x7f5dc0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0108880 0x7f5dc019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:34.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01a0490 con 0x7f5dc0074950 2023-08-11T14:03:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01a05d0 con 0x7f5dc0108880 2023-08-11T14:03:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.548+0000 7f5dc791b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01a0760 con 0x7f5dc0073380 2023-08-11T14:03:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37362/0 (socket says 172.21.15.3:37362) 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 -- 172.21.15.3:0/2888878879 learned_addr learned my addr 172.21.15.3:0/2888878879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc56b7700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0073380 0x7f5dc0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc5eb8700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0108880 0x7f5dc019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 -- 172.21.15.3:0/2888878879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0073380 msgr2=0x7f5dc0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0073380 0x7f5dc0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 -- 172.21.15.3:0/2888878879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0108880 msgr2=0x7f5dc019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0108880 0x7f5dc019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 -- 172.21.15.3:0/2888878879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00095f0 con 0x7f5dc0074950 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.549+0000 7f5dc4eb6700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5db0004740 tx=0x7f5db0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.550+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db001d070 con 0x7f5dc0074950 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.550+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db0021410 con 0x7f5dc0074950 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.550+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017650 con 0x7f5dc0074950 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.550+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc01a09e0 con 0x7f5dc0074950 2023-08-11T14:03:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.550+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01a0da0 con 0x7f5dc0074950 2023-08-11T14:03:34.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.551+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc0106f30 con 0x7f5dc0074950 2023-08-11T14:03:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.553+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5db0021580 con 0x7f5dc0074950 2023-08-11T14:03:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.554+0000 7f5db67fc700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 0x7f5dac0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.555+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5db00cc900 con 0x7f5dc0074950 2023-08-11T14:03:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.555+0000 7f5dc56b7700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 0x7f5dac0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:34.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.555+0000 7f5dc56b7700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 0x7f5dac0a2490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0074d30 tx=0x7f5dbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.558+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f5db00963a0 con 0x7f5dc0074950 2023-08-11T14:03:34.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.766+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f5dc010e420 con 0x7f5dc0074950 2023-08-11T14:03:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.767+0000 7f5db67fc700 1 -- 172.21.15.3:0/2888878879 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5584 (secure 0 0 0) 0x7f5db00999c0 con 0x7f5dc0074950 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 91 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-08-11T13:54:00.622002+0000 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2023-08-11T14:03:04.331784+0000 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-08-11T14:03:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 27 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release reef 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2023-08-11T14:03:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/2824703498,v1:172.21.15.3:6803/2824703498] [v2:172.21.15.3:6804/2824703498,v1:172.21.15.3:6805/2824703498] exists,up f155a1aa-25d5-4785-9e30-63ab3dc892ba 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/1127248620,v1:172.21.15.103:6801/1127248620] [v2:172.21.15.103:6802/1127248620,v1:172.21.15.103:6803/1127248620] exists,up 86cb5b3f-ad6a-47d8-8241-18b6a6889961 2023-08-11T14:03:34.771 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6800/2156234450,v1:172.21.15.187:6801/2156234450] [v2:172.21.15.187:6802/2156234450,v1:172.21.15.187:6803/2156234450] exists,up 3d3838c3-0422-4ead-946c-f9dd572bff9d 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/2193340429,v1:172.21.15.3:6811/2193340429] [v2:172.21.15.3:6812/2193340429,v1:172.21.15.3:6813/2193340429] exists,up 58d0f37b-9009-4887-b9fb-daa5c9d047e4 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/811611617,v1:172.21.15.103:6809/811611617] [v2:172.21.15.103:6810/811611617,v1:172.21.15.103:6811/811611617] exists,up 1c77ed74-b7d5-4c96-8507-8cd0a6a5b379 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 37 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6808/916876636,v1:172.21.15.187:6809/916876636] [v2:172.21.15.187:6810/916876636,v1:172.21.15.187:6811/916876636] exists,up 03c1c84c-d569-4383-b6b6-c0a38ea0ac4e 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 45 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/3251538985,v1:172.21.15.3:6819/3251538985] [v2:172.21.15.3:6820/3251538985,v1:172.21.15.3:6821/3251538985] exists,up 46bc04c3-30e0-461f-a65c-a9acecf575a7 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/3619083157,v1:172.21.15.103:6817/3619083157] [v2:172.21.15.103:6818/3619083157,v1:172.21.15.103:6819/3619083157] exists,up 1080578d-40d4-418d-9db4-999d831dd105 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 58 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6816/4219051644,v1:172.21.15.187:6817/4219051644] [v2:172.21.15.187:6818/4219051644,v1:172.21.15.187:6819/4219051644] exists,up 295943c9-dfde-47cd-96f0-570e5e989919 2023-08-11T14:03:34.772 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/3936829119,v1:172.21.15.3:6827/3936829119] [v2:172.21.15.3:6828/3936829119,v1:172.21.15.3:6829/3936829119] exists,up 9cd878b1-5843-4397-a987-2437787cd241 2023-08-11T14:03:34.773 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 71 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/1382085877,v1:172.21.15.103:6825/1382085877] [v2:172.21.15.103:6826/1382085877,v1:172.21.15.103:6827/1382085877] exists,up 51595dfa-2865-4bde-985a-b538b1d68644 2023-08-11T14:03:34.773 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 78 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6824/3190542621,v1:172.21.15.187:6825/3190542621] [v2:172.21.15.187:6826/3190542621,v1:172.21.15.187:6827/3190542621] exists,up b3738e6b-7935-4522-89c7-91cd190ab5e7 2023-08-11T14:03:34.773 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.f [8,11] 2023-08-11T14:03:34.773 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.13 [2,11] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.1 [4,9] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.11 [4,8,5,9] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.17 [4,6] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.38 [3,9] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.e [5,8] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.14 [9,10] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.16 [9,8] 2023-08-11T14:03:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.17 [9,8] 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2366653083 expires 2023-08-12T13:55:02.078980+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/1168721553 expires 2023-08-12T13:55:02.078980+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/1168721553 expires 2023-08-12T13:55:02.078980+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1070970995 expires 2023-08-12T13:55:02.078980+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/586553398 expires 2023-08-12T13:54:34.603107+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1776709742 expires 2023-08-12T13:55:02.078980+0000 2023-08-11T14:03:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/586553398 expires 2023-08-12T13:54:34.603107+0000 2023-08-11T14:03:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3848927136 expires 2023-08-12T13:54:34.603107+0000 2023-08-11T14:03:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3010670797 expires 2023-08-12T13:54:34.603107+0000 2023-08-11T14:03:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1680525538 expires 2023-08-12T13:54:34.603107+0000 2023-08-11T14:03:34.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.769+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 msgr2=0x7f5dac0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.769+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 0x7f5dac0a2490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0074d30 tx=0x7f5dbc009260 comp rx=0 tx=0).stop 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 msgr2=0x7f5dc0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5db0004740 tx=0x7f5db0009580 comp rx=0 tx=0).stop 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 shutdown_connections 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f5dac09ffd0 0x7f5dac0a2490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dc0108880 0x7f5dc019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dc0073380 0x7f5dc0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 --2- 172.21.15.3:0/2888878879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dc0074950 0x7f5dc0198e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 >> 172.21.15.3:0/2888878879 conn(0x7f5dc00fbf10 msgr2=0x7f5dc00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 shutdown_connections 2023-08-11T14:03:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:34.770+0000 7f5dc791b700 1 -- 172.21.15.3:0/2888878879 wait complete. 2023-08-11T14:03:35.298 INFO:teuthology.run_tasks:Running task fs.ready... 2023-08-11T14:03:35.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:35.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 -- 172.21.15.3:0/921651542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d81024b0 msgr2=0x7fa8d8108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/921651542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d81024b0 0x7fa8d8108d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000b0d0 tx=0x7fa8d000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 -- 172.21.15.3:0/921651542 shutdown_connections 2023-08-11T14:03:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/921651542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d81024b0 0x7fa8d8108d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/921651542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d8100920 0x7fa8d8100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/921651542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8101b10 0x7fa8d8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 -- 172.21.15.3:0/921651542 >> 172.21.15.3:0/921651542 conn(0x7fa8d80fbe50 msgr2=0x7fa8d80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 -- 172.21.15.3:0/921651542 shutdown_connections 2023-08-11T14:03:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.521+0000 7fa8df1ff700 1 -- 172.21.15.3:0/921651542 wait complete. 2023-08-11T14:03:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.522+0000 7fa8df1ff700 1 Processor -- start 2023-08-11T14:03:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.522+0000 7fa8df1ff700 1 -- start start 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.522+0000 7fa8df1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8100920 0x7fa8d8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8df1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8df1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d81024b0 0x7fa8d819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8df1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a04c0 con 0x7fa8d8101b10 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8df1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a0600 con 0x7fa8d81024b0 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8df1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81a0790 con 0x7fa8d8100920 2023-08-11T14:03:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37372/0 (socket says 172.21.15.3:37372) 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 -- 172.21.15.3:0/3792423254 learned_addr learned my addr 172.21.15.3:0/3792423254 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8dcf9b700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8100920 0x7fa8d8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8dd79c700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d81024b0 0x7fa8d819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 -- 172.21.15.3:0/3792423254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8100920 msgr2=0x7fa8d8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8100920 0x7fa8d8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 -- 172.21.15.3:0/3792423254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d81024b0 msgr2=0x7fa8d819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d81024b0 0x7fa8d819fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.523+0000 7fa8d7fff700 1 -- 172.21.15.3:0/3792423254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0009d00 con 0x7fa8d8101b10 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8d7fff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9a0 tx=0x7fa8c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4009790 con 0x7fa8d8101b10 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c4010430 con 0x7fa8d8101b10 2023-08-11T14:03:35.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4018680 con 0x7fa8d8101b10 2023-08-11T14:03:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d81a0a70 con 0x7fa8d8101b10 2023-08-11T14:03:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.524+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d81a0f30 con 0x7fa8d8101b10 2023-08-11T14:03:35.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.525+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d810c190 con 0x7fa8d8101b10 2023-08-11T14:03:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.527+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa8c40105a0 con 0x7fa8d8101b10 2023-08-11T14:03:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.529+0000 7fa8d5ffb700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 0x7fa8c80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.529+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa8c4014070 con 0x7fa8d8101b10 2023-08-11T14:03:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.530+0000 7fa8dcf9b700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 0x7fa8c80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.530+0000 7fa8dcf9b700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 0x7fa8c80a24e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8101970 tx=0x7fa8cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:35.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.533+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fa8c4051c30 con 0x7fa8d8101b10 2023-08-11T14:03:35.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:35 smithi003 ceph-mon[97195]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:35.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:35 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2888878879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T14:03:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:35 smithi103 ceph-mon[130434]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:35 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2888878879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T14:03:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.746+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d81a1150 con 0x7fa8d8101b10 2023-08-11T14:03:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.747+0000 7fa8d5ffb700 1 -- 172.21.15.3:0/3792423254 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7fa8c4095040 con 0x7fa8d8101b10 2023-08-11T14:03:35.749 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:35.750 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 msgr2=0x7fa8c80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 0x7fa8c80a24e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8101970 tx=0x7fa8cc009260 comp rx=0 tx=0).stop 2023-08-11T14:03:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 msgr2=0x7fa8d8198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e9a0 tx=0x7fa8c400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 shutdown_connections 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa8c80a0020 0x7fa8c80a24e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d81024b0 0x7fa8d819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d8100920 0x7fa8d8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 --2- 172.21.15.3:0/3792423254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d8101b10 0x7fa8d8198e80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 >> 172.21.15.3:0/3792423254 conn(0x7fa8d80fbe50 msgr2=0x7fa8d8107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 shutdown_connections 2023-08-11T14:03:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.750+0000 7fa8df1ff700 1 -- 172.21.15.3:0/3792423254 wait complete. 2023-08-11T14:03:35.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:35.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:35 smithi187 ceph-mon[130973]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:35 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2888878879' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-08-11T14:03:35.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 -- 172.21.15.3:0/1748677406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac100910 msgr2=0x7fbeac100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/1748677406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac100910 0x7fbeac100d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbea00099c0 tx=0x7fbea0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 -- 172.21.15.3:0/1748677406 shutdown_connections 2023-08-11T14:03:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/1748677406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac1024a0 0x7fbeac108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/1748677406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac100910 0x7fbeac100d30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/1748677406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac101b00 0x7fbeac101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.979+0000 7fbeb2dff700 1 -- 172.21.15.3:0/1748677406 >> 172.21.15.3:0/1748677406 conn(0x7fbeac0fbe60 msgr2=0x7fbeac0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.980+0000 7fbeb2dff700 1 -- 172.21.15.3:0/1748677406 shutdown_connections 2023-08-11T14:03:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.980+0000 7fbeb2dff700 1 -- 172.21.15.3:0/1748677406 wait complete. 2023-08-11T14:03:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.980+0000 7fbeb2dff700 1 Processor -- start 2023-08-11T14:03:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 -- start start 2023-08-11T14:03:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac101b00 0x7fbeac198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 0x7fbeac19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1a03d0 con 0x7fbeac1024a0 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1a0510 con 0x7fbeac100910 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb2dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac1a06a0 con 0x7fbeac101b00 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb139c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 0x7fbeac19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb0b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb0b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42278/0 (socket says 172.21.15.3:42278) 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb139c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 0x7fbeac19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37396/0 (socket says 172.21.15.3:37396) 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeb0b9b700 1 -- 172.21.15.3:0/423763854 learned_addr learned my addr 172.21.15.3:0/423763854 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.981+0000 7fbeabfff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac101b00 0x7fbeac198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 -- 172.21.15.3:0/423763854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac101b00 msgr2=0x7fbeac198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac101b00 0x7fbeac198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 -- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 msgr2=0x7fbeac19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 0x7fbeac19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 -- 172.21.15.3:0/423763854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00095f0 con 0x7fbeac100910 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb0b9b700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbea00098e0 tx=0x7fbea000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea001d070 con 0x7fbeac100910 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac1a0920 con 0x7fbeac100910 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbea0005050 con 0x7fbeac100910 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004b40 con 0x7fbeac100910 2023-08-11T14:03:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.982+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac1a0d50 con 0x7fbeac100910 2023-08-11T14:03:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.983+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac10c170 con 0x7fbeac100910 2023-08-11T14:03:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.986+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbea0004cc0 con 0x7fbeac100910 2023-08-11T14:03:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.987+0000 7fbea9ffb700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 0x7fbe9c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.987+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbea00cc7c0 con 0x7fbeac100910 2023-08-11T14:03:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.987+0000 7fbeabfff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 0x7fbe9c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.988+0000 7fbeabfff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 0x7fbe9c0a2410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009c60 tx=0x7fbe98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:35.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:35.989+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbea0096260 con 0x7fbeac100910 2023-08-11T14:03:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.198+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbeac1a0f70 con 0x7fbeac100910 2023-08-11T14:03:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.199+0000 7fbea9ffb700 1 -- 172.21.15.3:0/423763854 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbea0099880 con 0x7fbeac100910 2023-08-11T14:03:36.200 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:36.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 msgr2=0x7fbe9c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 0x7fbe9c0a2410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009c60 tx=0x7fbe98009260 comp rx=0 tx=0).stop 2023-08-11T14:03:36.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 msgr2=0x7fbeac198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbea00098e0 tx=0x7fbea000f690 comp rx=0 tx=0).stop 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 shutdown_connections 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbeac1024a0 0x7fbeac19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe9c09ff50 0x7fbe9c0a2410 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeac100910 0x7fbeac198850 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 --2- 172.21.15.3:0/423763854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac101b00 0x7fbeac198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 >> 172.21.15.3:0/423763854 conn(0x7fbeac0fbe60 msgr2=0x7fbeac0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:36.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 shutdown_connections 2023-08-11T14:03:36.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.201+0000 7fbeb2dff700 1 -- 172.21.15.3:0/423763854 wait complete. 2023-08-11T14:03:36.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 -- 172.21.15.3:0/1716261021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde20101b00 msgr2=0x7fde20101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 --2- 172.21.15.3:0/1716261021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde20101b00 0x7fde20101f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde08009920 tx=0x7fde08009c30 comp rx=0 tx=0).stop 2023-08-11T14:03:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 -- 172.21.15.3:0/1716261021 shutdown_connections 2023-08-11T14:03:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 --2- 172.21.15.3:0/1716261021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde201024a0 0x7fde20108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 --2- 172.21.15.3:0/1716261021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 0x7fde20100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 --2- 172.21.15.3:0/1716261021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde20101b00 0x7fde20101f60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 -- 172.21.15.3:0/1716261021 >> 172.21.15.3:0/1716261021 conn(0x7fde200fbe60 msgr2=0x7fde200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 -- 172.21.15.3:0/1716261021 shutdown_connections 2023-08-11T14:03:36.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.429+0000 7fde25fff700 1 -- 172.21.15.3:0/1716261021 wait complete. 2023-08-11T14:03:36.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.430+0000 7fde25fff700 1 Processor -- start 2023-08-11T14:03:36.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 -- start start 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 0x7fde200746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde201024a0 0x7fde20073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200739f0 con 0x7fde20101b00 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde20073b60 con 0x7fde201024a0 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.431+0000 7fde25fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201a0760 con 0x7fde20100910 2023-08-11T14:03:36.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37422/0 (socket says 172.21.15.3:37422) 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 -- 172.21.15.3:0/3171115496 learned_addr learned my addr 172.21.15.3:0/3171115496 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde1f7fe700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 0x7fde200746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde1ffff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde201024a0 0x7fde20073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 -- 172.21.15.3:0/3171115496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 msgr2=0x7fde200746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 0x7fde200746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 -- 172.21.15.3:0/3171115496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde201024a0 msgr2=0x7fde20073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde201024a0 0x7fde20073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 -- 172.21.15.3:0/3171115496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009d00 con 0x7fde20101b00 2023-08-11T14:03:36.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde17fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde08009230 tx=0x7fde0800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.432+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0801c070 con 0x7fde20101b00 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.433+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde08009520 con 0x7fde20101b00 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.433+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0800bba0 con 0x7fde20101b00 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.433+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08003a80 con 0x7fde20101b00 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.433+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201a0c70 con 0x7fde20101b00 2023-08-11T14:03:36.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.434+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde20107fd0 con 0x7fde20101b00 2023-08-11T14:03:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.436+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fde08020030 con 0x7fde20101b00 2023-08-11T14:03:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.437+0000 7fde1d7fa700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 0x7fde0c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.437+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fde080ccc80 con 0x7fde20101b00 2023-08-11T14:03:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.437+0000 7fde1f7fe700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 0x7fde0c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.438+0000 7fde1f7fe700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 0x7fde0c0a2410 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde20101960 tx=0x7fde10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:36.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.439+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fde08096720 con 0x7fde20101b00 2023-08-11T14:03:36.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:36 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3792423254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:36.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:36 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/423763854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.649+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde2010c170 con 0x7fde20101b00 2023-08-11T14:03:36.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.649+0000 7fde1d7fa700 1 -- 172.21.15.3:0/3171115496 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde08005040 con 0x7fde20101b00 2023-08-11T14:03:36.652 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:36.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 msgr2=0x7fde0c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 0x7fde0c0a2410 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde20101960 tx=0x7fde10009260 comp rx=0 tx=0).stop 2023-08-11T14:03:36.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 msgr2=0x7fde20072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde08009230 tx=0x7fde0800bf80 comp rx=0 tx=0).stop 2023-08-11T14:03:36.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 shutdown_connections 2023-08-11T14:03:36.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fde0c09ff50 0x7fde0c0a2410 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde201024a0 0x7fde20073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20100910 0x7fde200746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 --2- 172.21.15.3:0/3171115496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde20101b00 0x7fde20072d10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 >> 172.21.15.3:0/3171115496 conn(0x7fde200fbe60 msgr2=0x7fde20105110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:36.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 shutdown_connections 2023-08-11T14:03:36.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.652+0000 7fde25fff700 1 -- 172.21.15.3:0/3171115496 wait complete. 2023-08-11T14:03:36.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:36.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:36 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3792423254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:36.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:36 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/423763854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:36.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:36 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3792423254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:36.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:36 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/423763854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:36.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 -- 172.21.15.3:0/3614740275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 msgr2=0x7f6f0c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 --2- 172.21.15.3:0/3614740275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c101f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f000099c0 tx=0x7f6f00009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 -- 172.21.15.3:0/3614740275 shutdown_connections 2023-08-11T14:03:36.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 --2- 172.21.15.3:0/3614740275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c1024a0 0x7f6f0c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 --2- 172.21.15.3:0/3614740275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c100910 0x7f6f0c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 --2- 172.21.15.3:0/3614740275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c101f60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 -- 172.21.15.3:0/3614740275 >> 172.21.15.3:0/3614740275 conn(0x7f6f0c0fbe60 msgr2=0x7f6f0c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 -- 172.21.15.3:0/3614740275 shutdown_connections 2023-08-11T14:03:36.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.878+0000 7f6f111ff700 1 -- 172.21.15.3:0/3614740275 wait complete. 2023-08-11T14:03:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.879+0000 7f6f111ff700 1 Processor -- start 2023-08-11T14:03:36.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 -- start start 2023-08-11T14:03:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c1024a0 0x7f6f0c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19e140 con 0x7f6f0c101b00 2023-08-11T14:03:36.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19e280 con 0x7f6f0c100910 2023-08-11T14:03:36.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f111ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19e410 con 0x7f6f0c1024a0 2023-08-11T14:03:36.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f0ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.880+0000 7f6f0a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42304/0 (socket says 172.21.15.3:42304) 2023-08-11T14:03:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0ad9d700 1 -- 172.21.15.3:0/2357712492 learned_addr learned my addr 172.21.15.3:0/2357712492 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:36.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0b59e700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c1024a0 0x7f6f0c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 -- 172.21.15.3:0/2357712492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c1024a0 msgr2=0x7f6f0c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c1024a0 0x7f6f0c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 -- 172.21.15.3:0/2357712492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 msgr2=0x7f6f0c1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:36.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 -- 172.21.15.3:0/2357712492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000095f0 con 0x7f6f0c101b00 2023-08-11T14:03:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0ad9d700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6f0a59c700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c196b00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f000058e0 tx=0x7f6f0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.881+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0001d070 con 0x7f6f0c101b00 2023-08-11T14:03:36.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.882+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c19e690 con 0x7f6f0c101b00 2023-08-11T14:03:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.882+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f00005510 con 0x7f6f0c101b00 2023-08-11T14:03:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.882+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004b30 con 0x7f6f0c101b00 2023-08-11T14:03:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.882+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c19eb10 con 0x7f6f0c101b00 2023-08-11T14:03:36.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.883+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c107890 con 0x7f6f0c101b00 2023-08-11T14:03:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.885+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6f0000fb80 con 0x7f6f0c101b00 2023-08-11T14:03:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.886+0000 7f6ef3fff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 0x7f6ef40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.886+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6f000cc560 con 0x7f6f0c101b00 2023-08-11T14:03:36.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.886+0000 7f6f0ad9d700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 0x7f6ef40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.887+0000 7f6f0ad9d700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 0x7f6ef40a2410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005950 tx=0x7f6efc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:36.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:36.888+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6f00096120 con 0x7f6f0c101b00 2023-08-11T14:03:37.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.102+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f0c10c120 con 0x7f6f0c101b00 2023-08-11T14:03:37.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.103+0000 7f6ef3fff700 1 -- 172.21.15.3:0/2357712492 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f00026020 con 0x7f6f0c101b00 2023-08-11T14:03:37.104 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:37.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 msgr2=0x7f6ef40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 0x7f6ef40a2410 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005950 tx=0x7f6efc009260 comp rx=0 tx=0).stop 2023-08-11T14:03:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 msgr2=0x7f6f0c196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c196b00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f000058e0 tx=0x7f6f0000f690 comp rx=0 tx=0).stop 2023-08-11T14:03:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 shutdown_connections 2023-08-11T14:03:37.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f0c1024a0 0x7f6f0c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6ef409ff50 0x7f6ef40a2410 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f0c100910 0x7f6f0c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 --2- 172.21.15.3:0/2357712492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f0c101b00 0x7f6f0c196b00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 >> 172.21.15.3:0/2357712492 conn(0x7f6f0c0fbe60 msgr2=0x7f6f0c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 shutdown_connections 2023-08-11T14:03:37.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.105+0000 7f6f111ff700 1 -- 172.21.15.3:0/2357712492 wait complete. 2023-08-11T14:03:37.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 -- 172.21.15.3:0/4174225427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4560100910 msgr2=0x7f4560100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 --2- 172.21.15.3:0/4174225427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4560100910 0x7f4560100d30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f45500099c0 tx=0x7f4550009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 -- 172.21.15.3:0/4174225427 shutdown_connections 2023-08-11T14:03:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 --2- 172.21.15.3:0/4174225427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45601024a0 0x7f4560108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 --2- 172.21.15.3:0/4174225427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4560100910 0x7f4560100d30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 --2- 172.21.15.3:0/4174225427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 0x7f4560101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 -- 172.21.15.3:0/4174225427 >> 172.21.15.3:0/4174225427 conn(0x7f45600fbe60 msgr2=0x7f45600fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 -- 172.21.15.3:0/4174225427 shutdown_connections 2023-08-11T14:03:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.331+0000 7f45667ff700 1 -- 172.21.15.3:0/4174225427 wait complete. 2023-08-11T14:03:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.332+0000 7f45667ff700 1 Processor -- start 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.332+0000 7f45667ff700 1 -- start start 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.332+0000 7f45667ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 0x7f4560196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.332+0000 7f45667ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 0x7f4560196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f45667ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f45667ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019e200 con 0x7f45601024a0 2023-08-11T14:03:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f45667ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019e340 con 0x7f4560100910 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f45667ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019e4d0 con 0x7f4560101b00 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f455f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 0x7f4560196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f455ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 0x7f4560196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37458/0 (socket says 172.21.15.3:37458) 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 -- 172.21.15.3:0/1471489290 learned_addr learned my addr 172.21.15.3:0/1471489290 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 -- 172.21.15.3:0/1471489290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 msgr2=0x7f4560196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 0x7f4560196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 -- 172.21.15.3:0/1471489290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 msgr2=0x7f4560196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 0x7f4560196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 -- 172.21.15.3:0/1471489290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500095f0 con 0x7f45601024a0 2023-08-11T14:03:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f455ffff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 0x7f4560196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.333+0000 7f4564d9c700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f455800ef30 tx=0x7f455800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.334+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558018070 con 0x7f45601024a0 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.334+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f455800f040 con 0x7f45601024a0 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.334+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800cd00 con 0x7f45601024a0 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.334+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456019e750 con 0x7f45601024a0 2023-08-11T14:03:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.334+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456019ec10 con 0x7f45601024a0 2023-08-11T14:03:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.335+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4560107ef0 con 0x7f45601024a0 2023-08-11T14:03:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.337+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4558003a00 con 0x7f45601024a0 2023-08-11T14:03:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.338+0000 7f455d7fa700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 0x7f45480a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.338+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f45580cb110 con 0x7f45601024a0 2023-08-11T14:03:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.338+0000 7f455ffff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 0x7f45480a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.339+0000 7f455ffff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 0x7f45480a2410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4550009990 tx=0x7f4550005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:37.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.343+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4558094bb0 con 0x7f45601024a0 2023-08-11T14:03:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:37 smithi003 ceph-mon[97195]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:37 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3171115496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:37 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2357712492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.560+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f456002cd30 con 0x7f45601024a0 2023-08-11T14:03:37.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.561+0000 7f455d7fa700 1 -- 172.21.15.3:0/1471489290 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f45580981d0 con 0x7f45601024a0 2023-08-11T14:03:37.562 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:37.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 msgr2=0x7f45480a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 0x7f45480a2410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4550009990 tx=0x7f4550005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 msgr2=0x7f456019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f455800ef30 tx=0x7f455800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 shutdown_connections 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45601024a0 0x7f456019dc30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f454809ff50 0x7f45480a2410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4560100910 0x7f4560196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 --2- 172.21.15.3:0/1471489290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4560101b00 0x7f4560196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 >> 172.21.15.3:0/1471489290 conn(0x7f45600fbe60 msgr2=0x7f4560105030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:37.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 shutdown_connections 2023-08-11T14:03:37.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.563+0000 7f45667ff700 1 -- 172.21.15.3:0/1471489290 wait complete. 2023-08-11T14:03:37.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:37 smithi103 ceph-mon[130434]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:37 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3171115496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:37 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2357712492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.786+0000 7f2d3afff700 1 -- 172.21.15.3:0/851805769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d34074960 msgr2=0x7f2d34072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.786+0000 7f2d3afff700 1 --2- 172.21.15.3:0/851805769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d34074960 0x7f2d34072db0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009990 tx=0x7f2d28009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.786+0000 7f2d3afff700 1 -- 172.21.15.3:0/851805769 shutdown_connections 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.786+0000 7f2d3afff700 1 --2- 172.21.15.3:0/851805769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d341087e0 0x7f2d3410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.786+0000 7f2d3afff700 1 --2- 172.21.15.3:0/851805769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d34074960 0x7f2d34072db0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.787+0000 7f2d3afff700 1 --2- 172.21.15.3:0/851805769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34073380 0x7f2d34073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.787+0000 7f2d3afff700 1 -- 172.21.15.3:0/851805769 >> 172.21.15.3:0/851805769 conn(0x7f2d340fbe50 msgr2=0x7f2d340fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.787+0000 7f2d3afff700 1 -- 172.21.15.3:0/851805769 shutdown_connections 2023-08-11T14:03:37.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.787+0000 7f2d3afff700 1 -- 172.21.15.3:0/851805769 wait complete. 2023-08-11T14:03:37.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 Processor -- start 2023-08-11T14:03:37.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 -- start start 2023-08-11T14:03:37.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34074960 0x7f2d34198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 0x7f2d3419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d341a03b0 con 0x7f2d341087e0 2023-08-11T14:03:37.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d341a04f0 con 0x7f2d34073380 2023-08-11T14:03:37.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3afff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d341a0680 con 0x7f2d34074960 2023-08-11T14:03:37.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d38d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d3959c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 0x7f2d3419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d38d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42344/0 (socket says 172.21.15.3:42344) 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d38d9b700 1 -- 172.21.15.3:0/426819280 learned_addr learned my addr 172.21.15.3:0/426819280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.788+0000 7f2d33fff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34074960 0x7f2d34198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 -- 172.21.15.3:0/426819280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34074960 msgr2=0x7f2d34198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34074960 0x7f2d34198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 -- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 msgr2=0x7f2d3419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 0x7f2d3419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 -- 172.21.15.3:0/426819280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280095f0 con 0x7f2d34073380 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d3959c700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 0x7f2d3419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d38d9b700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005910 tx=0x7f2d2800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2801d070 con 0x7f2d34073380 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d341a0900 con 0x7f2d34073380 2023-08-11T14:03:37.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d28022470 con 0x7f2d34073380 2023-08-11T14:03:37.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.789+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28017650 con 0x7f2d34073380 2023-08-11T14:03:37.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.790+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d341a0d80 con 0x7f2d34073380 2023-08-11T14:03:37.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:37 smithi187 ceph-mon[130973]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:37.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:37 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3171115496' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:37 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2357712492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.790+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d34106e90 con 0x7f2d34073380 2023-08-11T14:03:37.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.793+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2d2800fad0 con 0x7f2d34073380 2023-08-11T14:03:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.794+0000 7f2d31ffb700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 0x7f2d240a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.794+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2d280cca60 con 0x7f2d34073380 2023-08-11T14:03:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.794+0000 7f2d33fff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 0x7f2d240a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:37.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.795+0000 7f2d33fff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 0x7f2d240a2460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d34074680 tx=0x7f2d2000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:37.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:37.796+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f2d28096500 con 0x7f2d34073380 2023-08-11T14:03:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.056+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d3410e360 con 0x7f2d34073380 2023-08-11T14:03:38.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.057+0000 7f2d31ffb700 1 -- 172.21.15.3:0/426819280 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f2d28099b20 con 0x7f2d34073380 2023-08-11T14:03:38.059 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:38.059 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 msgr2=0x7f2d240a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 0x7f2d240a2460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d34074680 tx=0x7f2d2000b410 comp rx=0 tx=0).stop 2023-08-11T14:03:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 msgr2=0x7f2d34198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005910 tx=0x7f2d2800f690 comp rx=0 tx=0).stop 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 shutdown_connections 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d341087e0 0x7f2d3419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2d2409ffa0 0x7f2d240a2460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d34073380 0x7f2d34198830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 --2- 172.21.15.3:0/426819280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d34074960 0x7f2d34198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 >> 172.21.15.3:0/426819280 conn(0x7f2d340fbe50 msgr2=0x7f2d340fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:38.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 shutdown_connections 2023-08-11T14:03:38.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.060+0000 7f2d3afff700 1 -- 172.21.15.3:0/426819280 wait complete. 2023-08-11T14:03:38.063 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:38.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1192060303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c4101b10 msgr2=0x7f42c4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1192060303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c4101b10 0x7f42c4101f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42b80099c0 tx=0x7f42b8009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1192060303 shutdown_connections 2023-08-11T14:03:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1192060303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c41024b0 0x7f42c4108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1192060303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1192060303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c4101b10 0x7f42c4101f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1192060303 >> 172.21.15.3:0/1192060303 conn(0x7f42c40fbe50 msgr2=0x7f42c40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1192060303 shutdown_connections 2023-08-11T14:03:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.282+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1192060303 wait complete. 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.283+0000 7f42ca3ff700 1 Processor -- start 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 -- start start 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c4101b10 0x7f42c4198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c41024b0 0x7f42c419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41a0310 con 0x7f42c4100920 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41a0450 con 0x7f42c41024b0 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42ca3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41a05e0 con 0x7f42c4101b10 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37486/0 (socket says 172.21.15.3:37486) 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42c3fff700 1 -- 172.21.15.3:0/1076651120 learned_addr learned my addr 172.21.15.3:0/1076651120 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42c899c700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c41024b0 0x7f42c419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.284+0000 7f42c37fe700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c4101b10 0x7f42c4198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 -- 172.21.15.3:0/1076651120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c4101b10 msgr2=0x7f42c4198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c4101b10 0x7f42c4198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 -- 172.21.15.3:0/1076651120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c41024b0 msgr2=0x7f42c419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c41024b0 0x7f42c419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 -- 172.21.15.3:0/1076651120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4009520 con 0x7f42c4100920 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c3fff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f42b400ea50 tx=0x7f42b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400cbd0 con 0x7f42c4100920 2023-08-11T14:03:38.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42b400cd30 con 0x7f42c4100920 2023-08-11T14:03:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4018680 con 0x7f42c4100920 2023-08-11T14:03:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80095f0 con 0x7f42c4100920 2023-08-11T14:03:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.285+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c41a0bf0 con 0x7f42c4100920 2023-08-11T14:03:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.286+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c410c130 con 0x7f42c4100920 2023-08-11T14:03:38.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.288+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f42b4010430 con 0x7f42c4100920 2023-08-11T14:03:38.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.290+0000 7f42c17fa700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 0x7f42ac0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.290+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f42b4014070 con 0x7f42c4100920 2023-08-11T14:03:38.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.290+0000 7f42c37fe700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 0x7f42ac0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.291+0000 7f42c37fe700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 0x7f42ac0a24e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42b800b580 tx=0x7f42b8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:38.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.292+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f42b40959d0 con 0x7f42c4100920 2023-08-11T14:03:38.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.508+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c41a0e10 con 0x7f42c4100920 2023-08-11T14:03:38.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.510+0000 7f42c17fa700 1 -- 172.21.15.3:0/1076651120 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f42b4098ff0 con 0x7f42c4100920 2023-08-11T14:03:38.512 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:38.512 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 msgr2=0x7f42ac0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 0x7f42ac0a24e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42b800b580 tx=0x7f42b8005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 msgr2=0x7f42c4198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f42b400ea50 tx=0x7f42b400ed60 comp rx=0 tx=0).stop 2023-08-11T14:03:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 shutdown_connections 2023-08-11T14:03:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f42ac0a0020 0x7f42ac0a24e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42c41024b0 0x7f42c419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42c4100920 0x7f42c4198790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 --2- 172.21.15.3:0/1076651120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c4101b10 0x7f42c4198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 >> 172.21.15.3:0/1076651120 conn(0x7f42c40fbe50 msgr2=0x7f42c40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:38.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 shutdown_connections 2023-08-11T14:03:38.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.512+0000 7f42ca3ff700 1 -- 172.21.15.3:0/1076651120 wait complete. 2023-08-11T14:03:38.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:38.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:38 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1471489290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:38 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/426819280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:38.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 -- 172.21.15.3:0/1717813938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd90101b00 msgr2=0x7efd90101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 --2- 172.21.15.3:0/1717813938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd90101b00 0x7efd90101f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd800099c0 tx=0x7efd80009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 -- 172.21.15.3:0/1717813938 shutdown_connections 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 --2- 172.21.15.3:0/1717813938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd901024a0 0x7efd90108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 --2- 172.21.15.3:0/1717813938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90100910 0x7efd90100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 --2- 172.21.15.3:0/1717813938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd90101b00 0x7efd90101f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 -- 172.21.15.3:0/1717813938 >> 172.21.15.3:0/1717813938 conn(0x7efd900fbe60 msgr2=0x7efd900fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 -- 172.21.15.3:0/1717813938 shutdown_connections 2023-08-11T14:03:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.740+0000 7efd979ff700 1 -- 172.21.15.3:0/1717813938 wait complete. 2023-08-11T14:03:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 Processor -- start 2023-08-11T14:03:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 -- start start 2023-08-11T14:03:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd90100910 0x7efd90106690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 0x7efd90106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901039c0 con 0x7efd901024a0 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90103b30 con 0x7efd90101b00 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd979ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90103cf0 con 0x7efd90100910 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.741+0000 7efd95f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37494/0 (socket says 172.21.15.3:37494) 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 -- 172.21.15.3:0/1736684981 learned_addr learned my addr 172.21.15.3:0/1736684981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd9579b700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd90100910 0x7efd90106690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd94f9a700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 0x7efd90106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 -- 172.21.15.3:0/1736684981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd90100910 msgr2=0x7efd90106690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd90100910 0x7efd90106690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 -- 172.21.15.3:0/1736684981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 msgr2=0x7efd90106bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 0x7efd90106bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 -- 172.21.15.3:0/1736684981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800095f0 con 0x7efd901024a0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd94f9a700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 0x7efd90106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd95f9c700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d980 tx=0x7efd8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88004a30 con 0x7efd901024a0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8800b5c0 con 0x7efd901024a0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd901a4df0 con 0x7efd901024a0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.742+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88009b50 con 0x7efd901024a0 2023-08-11T14:03:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.743+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd901a5220 con 0x7efd901024a0 2023-08-11T14:03:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.744+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9010c170 con 0x7efd901024a0 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.745+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efd88009cb0 con 0x7efd901024a0 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.747+0000 7efd867fc700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 0x7efd7c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.747+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efd880cb440 con 0x7efd901024a0 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.747+0000 7efd9579b700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 0x7efd7c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.747+0000 7efd9579b700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 0x7efd7c0a2410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd90101960 tx=0x7efd8c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:38.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.750+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7efd88094e50 con 0x7efd901024a0 2023-08-11T14:03:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:38 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1471489290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:38 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/426819280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:38.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:38 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1471489290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:38.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:38 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/426819280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:38.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.961+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd901a5410 con 0x7efd901024a0 2023-08-11T14:03:38.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.961+0000 7efd867fc700 1 -- 172.21.15.3:0/1736684981 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd88098470 con 0x7efd901024a0 2023-08-11T14:03:38.963 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 msgr2=0x7efd7c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 0x7efd7c0a2410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efd90101960 tx=0x7efd8c00b040 comp rx=0 tx=0).stop 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 msgr2=0x7efd90103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d980 tx=0x7efd8800dc90 comp rx=0 tx=0).stop 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 shutdown_connections 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd901024a0 0x7efd90103220 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd90100910 0x7efd90106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efd7c09ff50 0x7efd7c0a2410 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 --2- 172.21.15.3:0/1736684981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd90101b00 0x7efd90106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 >> 172.21.15.3:0/1736684981 conn(0x7efd900fbe60 msgr2=0x7efd900fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 shutdown_connections 2023-08-11T14:03:38.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:38.964+0000 7efd979ff700 1 -- 172.21.15.3:0/1736684981 wait complete. 2023-08-11T14:03:38.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:39.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 -- 172.21.15.3:0/370363650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 msgr2=0x7f03b0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 --2- 172.21.15.3:0/370363650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b0100d30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f03a00099c0 tx=0x7f03a0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 -- 172.21.15.3:0/370363650 shutdown_connections 2023-08-11T14:03:39.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 --2- 172.21.15.3:0/370363650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b01024a0 0x7f03b0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 --2- 172.21.15.3:0/370363650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b0100d30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 --2- 172.21.15.3:0/370363650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b0101b00 0x7f03b0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 -- 172.21.15.3:0/370363650 >> 172.21.15.3:0/370363650 conn(0x7f03b00fbe60 msgr2=0x7f03b00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:39.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 -- 172.21.15.3:0/370363650 shutdown_connections 2023-08-11T14:03:39.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.190+0000 7f03b61ff700 1 -- 172.21.15.3:0/370363650 wait complete. 2023-08-11T14:03:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.191+0000 7f03b61ff700 1 Processor -- start 2023-08-11T14:03:39.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.191+0000 7f03b61ff700 1 -- start start 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.191+0000 7f03b61ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.191+0000 7f03b61ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b0101b00 0x7f03b0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03b61ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 0x7f03b0073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03b61ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00739f0 con 0x7f03b0100910 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03b61ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0073b60 con 0x7f03b01024a0 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03b61ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b01a0670 con 0x7f03b0101b00 2023-08-11T14:03:39.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37514/0 (socket says 172.21.15.3:37514) 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 -- 172.21.15.3:0/2812241307 learned_addr learned my addr 172.21.15.3:0/2812241307 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03aeffd700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b0101b00 0x7f03b0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03affff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 0x7f03b0073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 -- 172.21.15.3:0/2812241307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b0101b00 msgr2=0x7f03b0072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b0101b00 0x7f03b0072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 -- 172.21.15.3:0/2812241307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 msgr2=0x7f03b0073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 0x7f03b0073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 -- 172.21.15.3:0/2812241307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a4009520 con 0x7f03b0100910 2023-08-11T14:03:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03affff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 0x7f03b0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:03:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.192+0000 7f03af7fe700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03a00058e0 tx=0x7f03a000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.193+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a001d070 con 0x7f03b0100910 2023-08-11T14:03:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.193+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a00095f0 con 0x7f03b0100910 2023-08-11T14:03:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.193+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03a0005510 con 0x7f03b0100910 2023-08-11T14:03:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.193+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0004c70 con 0x7f03b0100910 2023-08-11T14:03:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.193+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b01a0b80 con 0x7f03b0100910 2023-08-11T14:03:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.194+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b010c120 con 0x7f03b0100910 2023-08-11T14:03:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.196+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f03a000fc60 con 0x7f03b0100910 2023-08-11T14:03:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.197+0000 7f03acff9700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 0x7f03980a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.197+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f03a00cc5d0 con 0x7f03b0100910 2023-08-11T14:03:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.197+0000 7f03aeffd700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 0x7f03980a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.198+0000 7f03aeffd700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 0x7f03980a2410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d80 tx=0x7f03a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.201+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f03a0096070 con 0x7f03b0100910 2023-08-11T14:03:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.415+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03b0107890 con 0x7f03b0100910 2023-08-11T14:03:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.416+0000 7f03acff9700 1 -- 172.21.15.3:0/2812241307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03a0099690 con 0x7f03b0100910 2023-08-11T14:03:39.418 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 msgr2=0x7f03980a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 0x7f03980a2410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009d80 tx=0x7f03a4009260 comp rx=0 tx=0).stop 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 msgr2=0x7f03b00746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03a00058e0 tx=0x7f03a000f740 comp rx=0 tx=0).stop 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 shutdown_connections 2023-08-11T14:03:39.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f039809ff50 0x7f03980a2410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03b01024a0 0x7f03b0073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03b0100910 0x7f03b00746d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 --2- 172.21.15.3:0/2812241307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b0101b00 0x7f03b0072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 >> 172.21.15.3:0/2812241307 conn(0x7f03b00fbe60 msgr2=0x7f03b00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 shutdown_connections 2023-08-11T14:03:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.418+0000 7f03b61ff700 1 -- 172.21.15.3:0/2812241307 wait complete. 2023-08-11T14:03:39.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:39.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.645+0000 7f53f13ff700 1 -- 172.21.15.3:0/804466980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec100910 msgr2=0x7f53ec100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.645+0000 7f53f13ff700 1 --2- 172.21.15.3:0/804466980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec100910 0x7f53ec100d30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0099c0 tx=0x7f53dc009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 -- 172.21.15.3:0/804466980 shutdown_connections 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 --2- 172.21.15.3:0/804466980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec1024a0 0x7f53ec108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 --2- 172.21.15.3:0/804466980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec100910 0x7f53ec100d30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 --2- 172.21.15.3:0/804466980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec101b00 0x7f53ec101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 -- 172.21.15.3:0/804466980 >> 172.21.15.3:0/804466980 conn(0x7f53ec0fbe60 msgr2=0x7f53ec0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:39.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 -- 172.21.15.3:0/804466980 shutdown_connections 2023-08-11T14:03:39.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.646+0000 7f53f13ff700 1 -- 172.21.15.3:0/804466980 wait complete. 2023-08-11T14:03:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 Processor -- start 2023-08-11T14:03:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 -- start start 2023-08-11T14:03:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 0x7f53ec196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec1024a0 0x7f53ec19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec19e1b0 con 0x7f53ec101b00 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec19e2f0 con 0x7f53ec1024a0 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53f13ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec19e480 con 0x7f53ec100910 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53ea7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53ea7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37534/0 (socket says 172.21.15.3:37534) 2023-08-11T14:03:39.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53ea7fc700 1 -- 172.21.15.3:0/181534138 learned_addr learned my addr 172.21.15.3:0/181534138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.647+0000 7f53eb7fe700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec1024a0 0x7f53ec19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53eaffd700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 0x7f53ec196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 -- 172.21.15.3:0/181534138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 msgr2=0x7f53ec196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 0x7f53ec196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 -- 172.21.15.3:0/181534138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec1024a0 msgr2=0x7f53ec19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec1024a0 0x7f53ec19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 -- 172.21.15.3:0/181534138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0095f0 con 0x7f53ec101b00 2023-08-11T14:03:39.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53eaffd700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 0x7f53ec196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53ea7fc700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0009790 con 0x7f53ec101b00 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec19e760 con 0x7f53ec101b00 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e000cbd0 con 0x7f53ec101b00 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0018680 con 0x7f53ec101b00 2023-08-11T14:03:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.648+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec19ecc0 con 0x7f53ec101b00 2023-08-11T14:03:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.649+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec10c170 con 0x7f53ec101b00 2023-08-11T14:03:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.651+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f53e000cd40 con 0x7f53ec101b00 2023-08-11T14:03:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.654+0000 7f53d3fff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 0x7f53d40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.654+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f53e0014070 con 0x7f53ec101b00 2023-08-11T14:03:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.654+0000 7f53eaffd700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 0x7f53d40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.655+0000 7f53eaffd700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 0x7f53d40a2410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009990 tx=0x7f53dc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:39.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.658+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f53e0051c30 con 0x7f53ec101b00 2023-08-11T14:03:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:39 smithi103 ceph-mon[130434]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:39 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1076651120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:39.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:39 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1736684981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:39 smithi003 ceph-mon[97195]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1076651120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:39 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1736684981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:39.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:39 smithi187 ceph-mon[130973]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:39.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:39 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1076651120' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:39.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:39 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1736684981' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:39.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.871+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53ec107840 con 0x7f53ec101b00 2023-08-11T14:03:39.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.872+0000 7f53d3fff700 1 -- 172.21.15.3:0/181534138 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53e0095270 con 0x7f53ec101b00 2023-08-11T14:03:39.874 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 msgr2=0x7f53d40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 0x7f53d40a2410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009990 tx=0x7f53dc01a040 comp rx=0 tx=0).stop 2023-08-11T14:03:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 msgr2=0x7f53ec196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 shutdown_connections 2023-08-11T14:03:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f53d409ff50 0x7f53d40a2410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53ec1024a0 0x7f53ec19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53ec100910 0x7f53ec196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 --2- 172.21.15.3:0/181534138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec101b00 0x7f53ec196b70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 >> 172.21.15.3:0/181534138 conn(0x7f53ec0fbe60 msgr2=0x7f53ec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 shutdown_connections 2023-08-11T14:03:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:39.874+0000 7f53f13ff700 1 -- 172.21.15.3:0/181534138 wait complete. 2023-08-11T14:03:39.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:40.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 -- 172.21.15.3:0/1180862428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94101b00 msgr2=0x7fcd94101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/1180862428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94101b00 0x7fcd94101f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd840099c0 tx=0x7fcd84009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 -- 172.21.15.3:0/1180862428 shutdown_connections 2023-08-11T14:03:40.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/1180862428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd941024a0 0x7fcd94108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/1180862428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd94100910 0x7fcd94100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/1180862428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94101b00 0x7fcd94101f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 -- 172.21.15.3:0/1180862428 >> 172.21.15.3:0/1180862428 conn(0x7fcd940fbe60 msgr2=0x7fcd940fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:40.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 -- 172.21.15.3:0/1180862428 shutdown_connections 2023-08-11T14:03:40.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.100+0000 7fcd9bfff700 1 -- 172.21.15.3:0/1180862428 wait complete. 2023-08-11T14:03:40.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.101+0000 7fcd9bfff700 1 Processor -- start 2023-08-11T14:03:40.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.101+0000 7fcd9bfff700 1 -- start start 2023-08-11T14:03:40.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94100910 0x7fcd94196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941024a0 0x7fcd9419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e1e0 con 0x7fcd94101b00 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e320 con 0x7fcd94100910 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9bfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e4b0 con 0x7fcd941024a0 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37548/0 (socket says 172.21.15.3:37548) 2023-08-11T14:03:40.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 -- 172.21.15.3:0/4146973190 learned_addr learned my addr 172.21.15.3:0/4146973190 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd99d9b700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94100910 0x7fcd94196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9a59c700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941024a0 0x7fcd9419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 -- 172.21.15.3:0/4146973190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941024a0 msgr2=0x7fcd9419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941024a0 0x7fcd9419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 -- 172.21.15.3:0/4146973190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94100910 msgr2=0x7fcd94196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94100910 0x7fcd94196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.102+0000 7fcd9959a700 1 -- 172.21.15.3:0/4146973190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840095f0 con 0x7fcd94101b00 2023-08-11T14:03:40.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd9959a700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009fd0 tx=0x7fcd8400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8401d070 con 0x7fcd94101b00 2023-08-11T14:03:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9419e730 con 0x7fcd94101b00 2023-08-11T14:03:40.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd84005050 con 0x7fcd94101b00 2023-08-11T14:03:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84004b30 con 0x7fcd94101b00 2023-08-11T14:03:40.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.103+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9419ebb0 con 0x7fcd94101b00 2023-08-11T14:03:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.104+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd941078c0 con 0x7fcd94101b00 2023-08-11T14:03:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.106+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcd84004c90 con 0x7fcd94101b00 2023-08-11T14:03:40.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.108+0000 7fcd8affd700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 0x7fcd800a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.108+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcd840cc730 con 0x7fcd94101b00 2023-08-11T14:03:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.108+0000 7fcd99d9b700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 0x7fcd800a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.108+0000 7fcd99d9b700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 0x7fcd800a2410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcd94101960 tx=0x7fcd90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:40.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.110+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fcd840961d0 con 0x7fcd94101b00 2023-08-11T14:03:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.323+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcd9410c170 con 0x7fcd94101b00 2023-08-11T14:03:40.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.324+0000 7fcd8affd700 1 -- 172.21.15.3:0/4146973190 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd840997f0 con 0x7fcd94101b00 2023-08-11T14:03:40.326 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:40.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.326+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 msgr2=0x7fcd800a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.326+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 0x7fcd800a2410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcd94101960 tx=0x7fcd90009260 comp rx=0 tx=0).stop 2023-08-11T14:03:40.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 msgr2=0x7fcd94196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009fd0 tx=0x7fcd8400f690 comp rx=0 tx=0).stop 2023-08-11T14:03:40.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 shutdown_connections 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd941024a0 0x7fcd9419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd8009ff50 0x7fcd800a2410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd94100910 0x7fcd94196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 --2- 172.21.15.3:0/4146973190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd94101b00 0x7fcd94196ba0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 >> 172.21.15.3:0/4146973190 conn(0x7fcd940fbe60 msgr2=0x7fcd940fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:40.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 shutdown_connections 2023-08-11T14:03:40.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.327+0000 7fcd9bfff700 1 -- 172.21.15.3:0/4146973190 wait complete. 2023-08-11T14:03:40.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.552+0000 7f793c9ff700 1 -- 172.21.15.3:0/3834082106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7938073380 msgr2=0x7f7938073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.552+0000 7f793c9ff700 1 --2- 172.21.15.3:0/3834082106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7938073380 0x7f7938073800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f792c009990 tx=0x7f792c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 -- 172.21.15.3:0/3834082106 shutdown_connections 2023-08-11T14:03:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 --2- 172.21.15.3:0/3834082106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79381087f0 0x7f793810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 --2- 172.21.15.3:0/3834082106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938074960 0x7f7938072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 --2- 172.21.15.3:0/3834082106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7938073380 0x7f7938073800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 -- 172.21.15.3:0/3834082106 >> 172.21.15.3:0/3834082106 conn(0x7f79380fbe60 msgr2=0x7f79380fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:40.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 -- 172.21.15.3:0/3834082106 shutdown_connections 2023-08-11T14:03:40.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.553+0000 7f793c9ff700 1 -- 172.21.15.3:0/3834082106 wait complete. 2023-08-11T14:03:40.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 Processor -- start 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 -- start start 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938073380 0x7f7938198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7938074960 0x7f7938198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381a03c0 con 0x7f79381087f0 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381a0500 con 0x7f7938073380 2023-08-11T14:03:40.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793c9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79381a0690 con 0x7f7938074960 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f7936d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f793659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938073380 0x7f7938198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f7936d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37572/0 (socket says 172.21.15.3:37572) 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.554+0000 7f7936d9d700 1 -- 172.21.15.3:0/2011454276 learned_addr learned my addr 172.21.15.3:0/2011454276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7935d9b700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7938074960 0x7f7938198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 -- 172.21.15.3:0/2011454276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7938074960 msgr2=0x7f7938198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7938074960 0x7f7938198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 -- 172.21.15.3:0/2011454276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938073380 msgr2=0x7f7938198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938073380 0x7f7938198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 -- 172.21.15.3:0/2011454276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009520 con 0x7f79381087f0 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f7936d9d700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f793000ef30 tx=0x7f793000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930018070 con 0x7f79381087f0 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793000f040 con 0x7f79381087f0 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000cd00 con 0x7f79381087f0 2023-08-11T14:03:40.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792c0095f0 con 0x7f79381087f0 2023-08-11T14:03:40.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.555+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79381a0c40 con 0x7f79381087f0 2023-08-11T14:03:40.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.556+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7938106ea0 con 0x7f79381087f0 2023-08-11T14:03:40.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.558+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7930003a00 con 0x7f79381087f0 2023-08-11T14:03:40.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.560+0000 7f79277fe700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 0x7f79200a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:40.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.560+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f79300cba50 con 0x7f79381087f0 2023-08-11T14:03:40.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.560+0000 7f793659c700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 0x7f79200a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:40.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.561+0000 7f793659c700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 0x7f79200a24e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7928009df0 tx=0x7f7928009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:40.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.562+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f79300954f0 con 0x7f79381087f0 2023-08-11T14:03:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:40 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2812241307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:40 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/181534138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:40 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/4146973190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:40 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2812241307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:40 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/181534138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:40 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/4146973190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:40 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2812241307' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:40 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/181534138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:40 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/4146973190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:40.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.819+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793810e370 con 0x7f79381087f0 2023-08-11T14:03:40.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.820+0000 7f79277fe700 1 -- 172.21.15.3:0/2011454276 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f7930098b10 con 0x7f79381087f0 2023-08-11T14:03:40.822 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:40.822 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:40.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 msgr2=0x7f79200a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 0x7f79200a24e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7928009df0 tx=0x7f7928009260 comp rx=0 tx=0).stop 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 msgr2=0x7f793819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f793000ef30 tx=0x7f793000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 shutdown_connections 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79381087f0 0x7f793819fdf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f79200a0020 0x7f79200a24e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7938073380 0x7f7938198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 --2- 172.21.15.3:0/2011454276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7938074960 0x7f7938198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:40.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 >> 172.21.15.3:0/2011454276 conn(0x7f79380fbe60 msgr2=0x7f79380fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:40.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 shutdown_connections 2023-08-11T14:03:40.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:40.823+0000 7f793c9ff700 1 -- 172.21.15.3:0/2011454276 wait complete. 2023-08-11T14:03:40.827 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:40.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 -- 172.21.15.3:0/1981058454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c100920 msgr2=0x7f1e8c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 --2- 172.21.15.3:0/1981058454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c100920 0x7f1e8c100d40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0099c0 tx=0x7f1e7c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 -- 172.21.15.3:0/1981058454 shutdown_connections 2023-08-11T14:03:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 --2- 172.21.15.3:0/1981058454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 --2- 172.21.15.3:0/1981058454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c100920 0x7f1e8c100d40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 --2- 172.21.15.3:0/1981058454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 0x7f1e8c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 -- 172.21.15.3:0/1981058454 >> 172.21.15.3:0/1981058454 conn(0x7f1e8c0fbe50 msgr2=0x7f1e8c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 -- 172.21.15.3:0/1981058454 shutdown_connections 2023-08-11T14:03:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.048+0000 7f1e91dff700 1 -- 172.21.15.3:0/1981058454 wait complete. 2023-08-11T14:03:41.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 Processor -- start 2023-08-11T14:03:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 -- start start 2023-08-11T14:03:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c100920 0x7f1e8c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 0x7f1e8c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19e190 con 0x7f1e8c1024b0 2023-08-11T14:03:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19e2d0 con 0x7f1e8c100920 2023-08-11T14:03:41.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.049+0000 7f1e91dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19e460 con 0x7f1e8c101b10 2023-08-11T14:03:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 0x7f1e8c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37582/0 (socket says 172.21.15.3:37582) 2023-08-11T14:03:41.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 -- 172.21.15.3:0/2643255147 learned_addr learned my addr 172.21.15.3:0/2643255147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8b7fe700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c100920 0x7f1e8c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 -- 172.21.15.3:0/2643255147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 msgr2=0x7f1e8c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 0x7f1e8c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 -- 172.21.15.3:0/2643255147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c100920 msgr2=0x7f1e8c196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c100920 0x7f1e8c196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 -- 172.21.15.3:0/2643255147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0095f0 con 0x7f1e8c1024b0 2023-08-11T14:03:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.050+0000 7f1e8bfff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ef30 tx=0x7f1e8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.051+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84018070 con 0x7f1e8c1024b0 2023-08-11T14:03:41.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.051+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c19e6e0 con 0x7f1e8c1024b0 2023-08-11T14:03:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.051+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8400f040 con 0x7f1e8c1024b0 2023-08-11T14:03:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.051+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400cd00 con 0x7f1e8c1024b0 2023-08-11T14:03:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.051+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c19eba0 con 0x7f1e8c1024b0 2023-08-11T14:03:41.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.052+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c10c180 con 0x7f1e8c1024b0 2023-08-11T14:03:41.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.054+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1e84003a00 con 0x7f1e8c1024b0 2023-08-11T14:03:41.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.055+0000 7f1e88ff9700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 0x7f1e740a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.055+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1e840cb110 con 0x7f1e8c1024b0 2023-08-11T14:03:41.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.055+0000 7f1e8b7fe700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 0x7f1e740a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.056+0000 7f1e8b7fe700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 0x7f1e740a2460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009990 tx=0x7f1e7c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.059+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1e84094bb0 con 0x7f1e8c1024b0 2023-08-11T14:03:41.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.270+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e8c10c390 con 0x7f1e8c1024b0 2023-08-11T14:03:41.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.272+0000 7f1e88ff9700 1 -- 172.21.15.3:0/2643255147 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f1e840981d0 con 0x7f1e8c1024b0 2023-08-11T14:03:41.274 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:41.274 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 msgr2=0x7f1e740a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 0x7f1e740a2460 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009990 tx=0x7f1e7c005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 msgr2=0x7f1e8c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ef30 tx=0x7f1e8400c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 shutdown_connections 2023-08-11T14:03:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e8c1024b0 0x7f1e8c19dbc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1e7409ffa0 0x7f1e740a2460 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e8c100920 0x7f1e8c196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 --2- 172.21.15.3:0/2643255147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e8c101b10 0x7f1e8c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.274+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 >> 172.21.15.3:0/2643255147 conn(0x7f1e8c0fbe50 msgr2=0x7f1e8c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.275+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 shutdown_connections 2023-08-11T14:03:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.275+0000 7f1e91dff700 1 -- 172.21.15.3:0/2643255147 wait complete. 2023-08-11T14:03:41.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-08-11T14:01:32.831322+0000', 'modified': '2023-08-11T14:02:01.312375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24532, 'mds_2': 24527, 'mds_3': 24511, 'mds_4': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14502': {'gid': 14502, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/3729253320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 3729253320}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 3729253320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14514': {'gid': 14514, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6841/1195165407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 1195165407}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 1195165407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6833/1102333992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1102333992}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1102333992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/618362106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 618362106}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 618362106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/3236312344', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 3236312344}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3236312344}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.103:6835/1763153778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 1763153778}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 1763153778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.187:6837/1876247524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 1876247524}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 1876247524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24527': {'gid': 24527, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.103:6837/4115864783', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 4115864783}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 4115864783}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.187:6839/1685844974', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 1685844974}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 1685844974}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.103:6839/2494449018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2494449018}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2494449018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-11T14:03:41.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-08-11T14:03:41.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-08-11T14:03:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 -- 172.21.15.3:0/4048780853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 msgr2=0x7f9140072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 --2- 172.21.15.3:0/4048780853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f9140072db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9134009990 tx=0x7f9134009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 -- 172.21.15.3:0/4048780853 shutdown_connections 2023-08-11T14:03:41.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 --2- 172.21.15.3:0/4048780853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914010cee0 0x7f914010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 --2- 172.21.15.3:0/4048780853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f9140072db0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 --2- 172.21.15.3:0/4048780853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9140073380 0x7f9140073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 -- 172.21.15.3:0/4048780853 >> 172.21.15.3:0/4048780853 conn(0x7f91400fbe60 msgr2=0x7f91400fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 -- 172.21.15.3:0/4048780853 shutdown_connections 2023-08-11T14:03:41.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.504+0000 7f91471ff700 1 -- 172.21.15.3:0/4048780853 wait complete. 2023-08-11T14:03:41.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.505+0000 7f91471ff700 1 Processor -- start 2023-08-11T14:03:41.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 -- start start 2023-08-11T14:03:41.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 0x7f914019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f914019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914010cee0 0x7f91401a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401a47b0 con 0x7f9140073380 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401a4920 con 0x7f914010cee0 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f91471ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401a4ae0 con 0x7f9140074960 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 0x7f914019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 0x7f914019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37606/0 (socket says 172.21.15.3:37606) 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f9144f9b700 1 -- 172.21.15.3:0/3423232033 learned_addr learned my addr 172.21.15.3:0/3423232033 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f914019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.506+0000 7f914579c700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914010cee0 0x7f91401a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 -- 172.21.15.3:0/3423232033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914010cee0 msgr2=0x7f91401a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914010cee0 0x7f91401a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 -- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 msgr2=0x7f914019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 0x7f914019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 -- 172.21.15.3:0/3423232033 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340095f0 con 0x7f9140074960 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f914019d1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e9c0 tx=0x7f912c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913dffb700 1 -- 172.21.15.3:0/3423232033 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c004220 con 0x7f9140074960 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91401a4dc0 con 0x7f9140074960 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913dffb700 1 -- 172.21.15.3:0/3423232033 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f912c004380 con 0x7f9140074960 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f913dffb700 1 -- 172.21.15.3:0/3423232033 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c003be0 con 0x7f9140074960 2023-08-11T14:03:41.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.507+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91401a5240 con 0x7f9140074960 2023-08-11T14:03:41.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.510+0000 7f913dffb700 1 -- 172.21.15.3:0/3423232033 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f912c003d40 con 0x7f9140074960 2023-08-11T14:03:41.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.512+0000 7f915ac0fb80 1 Processor -- start 2023-08-11T14:03:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.512+0000 7f915ac0fb80 1 -- start start 2023-08-11T14:03:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded03c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03cda0 0x564ded03d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f913dffb700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 0x7f91300a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564decf945e0 con 0x564ded03ba80 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564decf91f00 con 0x564ded03c3e0 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564decf8d7f0 con 0x564ded03cda0 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f913dffb700 1 -- 172.21.15.3:0/3423232033 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f912c014070 con 0x7f9140074960 2023-08-11T14:03:41.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f914579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 0x7f91300a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f914579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37612/0 (socket says 172.21.15.3:37612) 2023-08-11T14:03:41.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f914579c700 1 -- 172.21.15.3:0/3075461390 learned_addr learned my addr 172.21.15.3:0/3075461390 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.513+0000 7f9144f9b700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded03c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03cda0 0x564ded03d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 -- 172.21.15.3:0/3075461390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03cda0 msgr2=0x564ded03d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03cda0 0x564ded03d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 -- 172.21.15.3:0/3075461390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 msgr2=0x564ded03c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded03c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 -- 172.21.15.3:0/3075461390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dece0d5e0 con 0x564ded03ba80 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f9144f9b700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded03c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f913ffff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 0x7f91300a2410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f912c009f90 tx=0x7f912c00cf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f914579c700 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9138009fd0 tx=0x7f9138009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c2dd9badbc606f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f912affd700 1 -- 172.21.15.3:0/3075461390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e040 con 0x564ded03ba80 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f912affd700 1 -- 172.21.15.3:0/3075461390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913800f040 con 0x564ded03ba80 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.514+0000 7f912affd700 1 -- 172.21.15.3:0/3075461390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138013e70 con 0x564ded03ba80 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 -- 172.21.15.3:0/3075461390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 msgr2=0x564ded03bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9138009fd0 tx=0x7f9138009920 comp rx=0 tx=0).stop 2023-08-11T14:03:41.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 -- 172.21.15.3:0/3075461390 shutdown_connections 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03cda0 0x564ded03d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03ba80 0x564ded03bea0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/3075461390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded03c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 -- 172.21.15.3:0/3075461390 >> 172.21.15.3:0/3075461390 conn(0x564decf68900 msgr2=0x564ded0363d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 -- 172.21.15.3:0/3075461390 shutdown_connections 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.515+0000 7f915ac0fb80 1 -- 172.21.15.3:0/3075461390 wait complete. 2023-08-11T14:03:41.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 -- start start 2023-08-11T14:03:41.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 Processor -- start 2023-08-11T14:03:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded045140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded04c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564decf2ba70 con 0x564ded03cda0 2023-08-11T14:03:41.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564decf2b770 con 0x564ded03c3e0 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.516+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dece0f8d0 con 0x564ded03ba80 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f915ac0fb80 10 client.0 _ll_register_callbacks cb 0x564decf2ea60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f914579c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded045140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42502/0 (socket says 172.21.15.3:42502) 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 -- 172.21.15.3:0/906390313 learned_addr learned my addr 172.21.15.3:0/906390313 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f914579c700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded04c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 -- 172.21.15.3:0/906390313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 msgr2=0x564ded045140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded045140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 -- 172.21.15.3:0/906390313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 msgr2=0x564ded04c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded04c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f9144f9b700 1 -- 172.21.15.3:0/906390313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914010fc80 con 0x564ded03c3e0 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.517+0000 7f914579c700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded04c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:41.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f9144f9b700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f9134005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cb74229540827 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f911ffff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913401d030 con 0x564ded03c3e0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ded02ea40 con 0x564ded03c3e0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9134004700 con 0x564ded03c3e0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91340048c0 con 0x564ded03c3e0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f915ac0fb80 10 client.25376 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f913800f710 con 0x564ded03c3e0 2023-08-11T14:03:41.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.518+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ded02ea40 con 0x564ded03c3e0 2023-08-11T14:03:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f913400fa60 con 0x564ded03c3e0 2023-08-11T14:03:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 10 client.25376 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 10 client.25376 resolve_mds: resolved name 'e' to daemon mds.e 2023-08-11T14:03:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 20 client.25376 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:41.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded050d20 0x564ded04f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 4 client.25376 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ded00a490 con 0x564ded050d20 2023-08-11T14:03:41.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.519+0000 7f914579c700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded050d20 0x564ded04f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.520+0000 7f911ffff700 10 client.25376 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:41.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.520+0000 7f914579c700 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded050d20 0x564ded04f7c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=507e460d625a8191 server_cookie=89258a045a7c4960 in_seq=0 out_seq=0 2023-08-11T14:03:41.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.520+0000 7f911ffff700 1 -- 172.21.15.3:0/906390313 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564ded00a490 con 0x564ded050d20 2023-08-11T14:03:41.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.520+0000 7f911ffff700 10 client.25376 handle_command_reply: tid=0 2023-08-11T14:03:41.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.520+0000 7f915ac0fb80 1 client.25376 shutdown 2023-08-11T14:03:41.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 msgr2=0x564ded045680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9134000c00 tx=0x7f9134005c70 comp rx=0 tx=0).stop 2023-08-11T14:03:41.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 shutdown_connections 2023-08-11T14:03:41.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded050d20 0x564ded04f7c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded04c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f911ffff700 0 client.25376 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:41.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded045140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/906390313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded045680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 >> 172.21.15.3:0/906390313 conn(0x564decf68900 msgr2=0x564ded030c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 shutdown_connections 2023-08-11T14:03:41.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 1 -- 172.21.15.3:0/906390313 wait complete. 2023-08-11T14:03:41.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.521+0000 7f915ac0fb80 20 client.25376 trim_cache size 0 max 16384 2023-08-11T14:03:41.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.539+0000 7f915ac0fb80 1 Processor -- start 2023-08-11T14:03:41.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.539+0000 7f915ac0fb80 1 -- start start 2023-08-11T14:03:41.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded03fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded040240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded032ac0 con 0x564ded03cda0 2023-08-11T14:03:41.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded051b30 con 0x564ded03c3e0 2023-08-11T14:03:41.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded037cd0 con 0x564ded03ba80 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f914579c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded03fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f914579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded040240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f914579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33128/0 (socket says 172.21.15.3:33128) 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.540+0000 7f914579c700 1 -- 172.21.15.3:0/1954956378 learned_addr learned my addr 172.21.15.3:0/1954956378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 -- 172.21.15.3:0/1954956378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 msgr2=0x564ded03fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded03fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 -- 172.21.15.3:0/1954956378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 msgr2=0x564ded059680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 -- 172.21.15.3:0/1954956378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564decdb9d90 con 0x564ded03c3e0 2023-08-11T14:03:41.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f914579c700 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f9144f9b700 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded040240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f913400b680 tx=0x7f91340094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea2783e9e80774b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f912b7fe700 1 -- 172.21.15.3:0/1954956378 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913401d070 con 0x564ded03c3e0 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f912b7fe700 1 -- 172.21.15.3:0/1954956378 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913401e040 con 0x564ded03c3e0 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.541+0000 7f912b7fe700 1 -- 172.21.15.3:0/1954956378 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400f860 con 0x564ded03c3e0 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- 172.21.15.3:0/1954956378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 msgr2=0x564ded040240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded040240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f913400b680 tx=0x7f91340094b0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- 172.21.15.3:0/1954956378 shutdown_connections 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03cda0 0x564ded059680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded03fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/1954956378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03c3e0 0x564ded040240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- 172.21.15.3:0/1954956378 >> 172.21.15.3:0/1954956378 conn(0x564decf68900 msgr2=0x564ded043e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- 172.21.15.3:0/1954956378 shutdown_connections 2023-08-11T14:03:41.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- 172.21.15.3:0/1954956378 wait complete. 2023-08-11T14:03:41.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 -- start start 2023-08-11T14:03:41.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.542+0000 7f915ac0fb80 1 Processor -- start 2023-08-11T14:03:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded053cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03cda0 0x564ded0590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded059600 con 0x564ded03c3e0 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded059770 con 0x564ded03cda0 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ded059930 con 0x564ded03ba80 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f915ac0fb80 10 client.0 _ll_register_callbacks cb 0x7f913400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33132/0 (socket says 172.21.15.3:33132) 2023-08-11T14:03:41.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 -- 172.21.15.3:0/783218487 learned_addr learned my addr 172.21.15.3:0/783218487 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f914579c700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded053cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f914579c700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03cda0 0x564ded0590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 -- 172.21.15.3:0/783218487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 msgr2=0x564ded053cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded053cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 -- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03cda0 msgr2=0x564ded0590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03cda0 0x564ded0590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f9144f9b700 1 -- 172.21.15.3:0/783218487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dece0d5e0 con 0x564ded03c3e0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.543+0000 7f914579c700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded053cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f9144f9b700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f913400fe70 tx=0x7f9134005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f14c3cafc32d165 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f911ffff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913401d030 con 0x564ded03c3e0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9134000e40 con 0x564ded03c3e0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f913800f040 con 0x564ded03c3e0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9134024bc0 con 0x564ded03c3e0 2023-08-11T14:03:41.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f915ac0fb80 10 client.15345 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:41.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.544+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f913800f710 con 0x564ded03c3e0 2023-08-11T14:03:41.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913800f040 con 0x564ded03c3e0 2023-08-11T14:03:41.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9134004830 con 0x564ded03c3e0 2023-08-11T14:03:41.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:41.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 10 client.15345 resolve_mds: resolved name 'e' to daemon mds.e 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 20 client.15345 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded0a9070 0x564ded0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 4 client.15345 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564decf867d0 con 0x564ded0a9070 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.545+0000 7f914579c700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded0a9070 0x564ded0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f911ffff700 10 client.15345 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f914579c700 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded0a9070 0x564ded0a5530 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c78000e1ab7702cf server_cookie=1c8c7ebf257900cb in_seq=0 out_seq=0 2023-08-11T14:03:41.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f911ffff700 1 -- 172.21.15.3:0/783218487 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564decf867d0 con 0x564ded0a9070 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f911ffff700 10 client.15345 handle_command_reply: tid=0 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 client.15345 shutdown 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 msgr2=0x564ded0541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f913400fe70 tx=0x7f9134005c70 comp rx=0 tx=0).stop 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 shutdown_connections 2023-08-11T14:03:41.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x564ded0a9070 0x564ded0a5530 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ded03cda0 0x564ded0590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564ded03ba80 0x564ded053cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 --2- 172.21.15.3:0/783218487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ded03c3e0 0x564ded0541f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f911ffff700 0 client.15345 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:03:41.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 >> 172.21.15.3:0/783218487 conn(0x564decf68900 msgr2=0x564ded04f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 shutdown_connections 2023-08-11T14:03:41.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.546+0000 7f915ac0fb80 1 -- 172.21.15.3:0/783218487 wait complete. 2023-08-11T14:03:41.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f915ac0fb80 20 client.15345 trim_cache size 0 max 16384 2023-08-11T14:03:41.558 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24500, 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 101.2380125, 2023-08-11T14:03:41.559 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:41.560 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:41.560 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:41.560 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 152.08038380799999 2023-08-11T14:03:41.560 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:41.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 msgr2=0x7f91300a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 0x7f91300a2410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f912c009f90 tx=0x7f912c00cf10 comp rx=0 tx=0).stop 2023-08-11T14:03:41.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 msgr2=0x7f914019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f914019d1b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e9c0 tx=0x7f912c00ecd0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 shutdown_connections 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f913009ff50 0x7f91300a2410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914010cee0 0x7f91401a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9140073380 0x7f914019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 --2- 172.21.15.3:0/3423232033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9140074960 0x7f914019d1b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 >> 172.21.15.3:0/3423232033 conn(0x7f91400fbe60 msgr2=0x7f91400fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 shutdown_connections 2023-08-11T14:03:41.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.547+0000 7f91471ff700 1 -- 172.21.15.3:0/3423232033 wait complete. 2023-08-11T14:03:41.564 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:41 smithi003 ceph-mon[97195]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:41.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2011454276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:41 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2643255147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-11T14:03:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:41 smithi103 ceph-mon[130434]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2011454276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:41 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2643255147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 -- 172.21.15.3:0/1678729851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 msgr2=0x7fe810101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 --2- 172.21.15.3:0/1678729851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810101de0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80099c0 tx=0x7fe7f8009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 -- 172.21.15.3:0/1678729851 shutdown_connections 2023-08-11T14:03:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 --2- 172.21.15.3:0/1678729851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810102320 0x7fe810108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 --2- 172.21.15.3:0/1678729851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810100790 0x7fe810100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 --2- 172.21.15.3:0/1678729851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810101de0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 -- 172.21.15.3:0/1678729851 >> 172.21.15.3:0/1678729851 conn(0x7fe8100fbd00 msgr2=0x7fe8100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 -- 172.21.15.3:0/1678729851 shutdown_connections 2023-08-11T14:03:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.770+0000 7fe814b9d700 1 -- 172.21.15.3:0/1678729851 wait complete. 2023-08-11T14:03:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.771+0000 7fe814b9d700 1 Processor -- start 2023-08-11T14:03:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 -- start start 2023-08-11T14:03:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810100790 0x7fe810196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 0x7fe81019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019df90 con 0x7fe810101980 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019e0d0 con 0x7fe810100790 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe814b9d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019e260 con 0x7fe810102320 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe80ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 0x7fe81019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33148/0 (socket says 172.21.15.3:33148) 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe807fff700 1 -- 172.21.15.3:0/3503869935 learned_addr learned my addr 172.21.15.3:0/3503869935 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.772+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810100790 0x7fe810196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 -- 172.21.15.3:0/3503869935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810100790 msgr2=0x7fe810196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810100790 0x7fe810196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 -- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 msgr2=0x7fe810196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 -- 172.21.15.3:0/3503869935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009c40 con 0x7fe810102320 2023-08-11T14:03:41.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe807fff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80ffff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 0x7fe81019d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe80800efb0 tx=0x7fe80800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80d7fa700 1 -- 172.21.15.3:0/3503869935 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808003a00 con 0x7fe810102320 2023-08-11T14:03:41.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80d7fa700 1 -- 172.21.15.3:0/3503869935 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80800f040 con 0x7fe810102320 2023-08-11T14:03:41.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f80095f0 con 0x7fe810102320 2023-08-11T14:03:41.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe80d7fa700 1 -- 172.21.15.3:0/3503869935 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808004540 con 0x7fe810102320 2023-08-11T14:03:41.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.773+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81019e870 con 0x7fe810102320 2023-08-11T14:03:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.778+0000 7fe80d7fa700 1 -- 172.21.15.3:0/3503869935 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe808003bf0 con 0x7fe810102320 2023-08-11T14:03:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 Processor -- start 2023-08-11T14:03:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 -- start start 2023-08-11T14:03:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a11532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a10aa5e0 con 0x5609a1151ac0 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a10a7f00 con 0x5609a1152e40 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.779+0000 7fe829720b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a10a37f0 con 0x5609a1152420 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:53636/0 (socket says 172.21.15.3:53636) 2023-08-11T14:03:41.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 -- 172.21.15.3:0/4003247802 learned_addr learned my addr 172.21.15.3:0/4003247802 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe807fff700 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a11532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80d7fa700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 0x7fe7fc0a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 -- 172.21.15.3:0/4003247802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 msgr2=0x5609a11532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a11532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 -- 172.21.15.3:0/4003247802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 msgr2=0x5609a1151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80f7fe700 1 -- 172.21.15.3:0/4003247802 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a0f235e0 con 0x5609a1152420 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80d7fa700 1 -- 172.21.15.3:0/3503869935 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe8080cb1a0 con 0x7fe810102320 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.780+0000 7fe80ffff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 0x7fe7fc0a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.781+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe800009d30 tx=0x7fe800011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e176c8dff5242882 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.781+0000 7fe80ffff700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 0x7fe7fc0a24b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8080a0150 tx=0x7fe808003b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.781+0000 7fe805ffb700 1 -- 172.21.15.3:0/4003247802 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800004220 con 0x5609a1152420 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.781+0000 7fe805ffb700 1 -- 172.21.15.3:0/4003247802 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe800004380 con 0x5609a1152420 2023-08-11T14:03:41.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.781+0000 7fe805ffb700 1 -- 172.21.15.3:0/4003247802 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800019810 con 0x5609a1152420 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- 172.21.15.3:0/4003247802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 msgr2=0x5609a1152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe800009d30 tx=0x7fe800011720 comp rx=0 tx=0).stop 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- 172.21.15.3:0/4003247802 shutdown_connections 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a11532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 --2- 172.21.15.3:0/4003247802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152420 0x5609a1152900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- 172.21.15.3:0/4003247802 >> 172.21.15.3:0/4003247802 conn(0x5609a107e900 msgr2=0x5609a114c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- 172.21.15.3:0/4003247802 shutdown_connections 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- 172.21.15.3:0/4003247802 wait complete. 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.782+0000 7fe829720b80 1 -- start start 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 Processor -- start 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a11627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a1041a70 con 0x5609a1151ac0 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a1041770 con 0x5609a1152420 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a0f258d0 con 0x5609a1152e40 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe829720b80 10 client.0 _ll_register_callbacks cb 0x5609a1044a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43436/0 (socket says 172.21.15.3:43436) 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33174/0 (socket says 172.21.15.3:33174) 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.783+0000 7fe80f7fe700 1 -- 172.21.15.3:0/3327691030 learned_addr learned my addr 172.21.15.3:0/3327691030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe807fff700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a11627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 -- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 msgr2=0x5609a11627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a11627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 -- 172.21.15.3:0/3327691030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 msgr2=0x5609a115b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 -- 172.21.15.3:0/3327691030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe800009960 con 0x5609a1152420 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe807fff700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8000097c0 tx=0x7fe800005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9c03c2ef09b6ab server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.784+0000 7fe7eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800005040 con 0x5609a1152420 2023-08-11T14:03:41.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a1144a40 con 0x5609a1152420 2023-08-11T14:03:41.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe8000051a0 con 0x5609a1152420 2023-08-11T14:03:41.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:41 smithi187 ceph-mon[130973]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-08-11T14:03:41.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2011454276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:41 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2643255147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:41.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe800005360 con 0x5609a1152420 2023-08-11T14:03:41.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe829720b80 10 client.25394 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609a0ecfd90 con 0x5609a1152420 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609a1144a40 con 0x5609a1152420 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe800021040 con 0x5609a1152420 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.785+0000 7fe829720b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe829720b80 10 client.25394 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-11T14:03:41.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe829720b80 20 client.25394 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a1166d80 0x5609a1165860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe829720b80 4 client.25394 mds_command: new command op to 24511 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609a1120490 con 0x5609a1166d80 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.786+0000 7fe807fff700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a1166d80 0x5609a1165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.787+0000 7fe807fff700 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a1166d80 0x5609a1165860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f191c06f69b693a server_cookie=8859e7a96b8b0ae7 in_seq=0 out_seq=0 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.787+0000 7fe7eeffd700 10 client.25394 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:41.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.787+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3327691030 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5609a1120490 con 0x5609a1166d80 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.787+0000 7fe7eeffd700 10 client.25394 handle_command_reply: tid=0 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.787+0000 7fe829720b80 1 client.25394 shutdown 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 msgr2=0x5609a115b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8000097c0 tx=0x7fe800005ce0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 shutdown_connections 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a1166d80 0x5609a1165860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a11627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a115b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe7eeffd700 0 client.25394 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 --2- 172.21.15.3:0/3327691030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a115b770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 >> 172.21.15.3:0/3327691030 conn(0x5609a107e900 msgr2=0x5609a1146c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 shutdown_connections 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 1 -- 172.21.15.3:0/3327691030 wait complete. 2023-08-11T14:03:41.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.788+0000 7fe829720b80 20 client.25394 trim_cache size 0 max 16384 2023-08-11T14:03:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.806+0000 7fe829720b80 1 Processor -- start 2023-08-11T14:03:41.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.806+0000 7fe829720b80 1 -- start start 2023-08-11T14:03:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.806+0000 7fe829720b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 0x5609a1155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.806+0000 7fe829720b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe829720b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a1165860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe829720b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a1060480 con 0x5609a1152420 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe829720b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a106dc30 con 0x5609a1152e40 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe829720b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a1144a20 con 0x5609a1151ac0 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33188/0 (socket says 172.21.15.3:33188) 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe807fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 0x5609a1155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 -- 172.21.15.3:0/1335425623 learned_addr learned my addr 172.21.15.3:0/1335425623 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe807fff700 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a1165860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 -- 172.21.15.3:0/1335425623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 msgr2=0x5609a1155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 0x5609a1155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 -- 172.21.15.3:0/1335425623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 msgr2=0x5609a1165860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a1165860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe80f7fe700 1 -- 172.21.15.3:0/1335425623 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a0eb8c10 con 0x5609a1152420 2023-08-11T14:03:41.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.807+0000 7fe807fff700 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 0x5609a1155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe800011760 tx=0x7fe800011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418e74f5d908b94e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe8067fc700 1 -- 172.21.15.3:0/1335425623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800019430 con 0x5609a1152420 2023-08-11T14:03:41.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe8067fc700 1 -- 172.21.15.3:0/1335425623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8000197f0 con 0x5609a1152420 2023-08-11T14:03:41.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe8067fc700 1 -- 172.21.15.3:0/1335425623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800005760 con 0x5609a1152420 2023-08-11T14:03:41.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 -- 172.21.15.3:0/1335425623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 msgr2=0x5609a1156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe800011760 tx=0x7fe800011b20 comp rx=0 tx=0).stop 2023-08-11T14:03:41.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 -- 172.21.15.3:0/1335425623 shutdown_connections 2023-08-11T14:03:41.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152e40 0x5609a1165860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1151ac0 0x5609a1155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 --2- 172.21.15.3:0/1335425623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1152420 0x5609a1156160 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.808+0000 7fe829720b80 1 -- 172.21.15.3:0/1335425623 >> 172.21.15.3:0/1335425623 conn(0x5609a107e900 msgr2=0x5609a114c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.809+0000 7fe829720b80 1 -- 172.21.15.3:0/1335425623 shutdown_connections 2023-08-11T14:03:41.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.809+0000 7fe829720b80 1 -- 172.21.15.3:0/1335425623 wait complete. 2023-08-11T14:03:41.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.809+0000 7fe829720b80 1 -- start start 2023-08-11T14:03:41.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.809+0000 7fe829720b80 1 Processor -- start 2023-08-11T14:03:41.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a1157570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a1157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a11be610 con 0x5609a1151ac0 2023-08-11T14:03:41.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a11be750 con 0x5609a1152420 2023-08-11T14:03:41.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a11be910 con 0x5609a1152e40 2023-08-11T14:03:41.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe829720b80 10 client.0 _ll_register_callbacks cb 0x7fe8080097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:41.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe807fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33204/0 (socket says 172.21.15.3:33204) 2023-08-11T14:03:41.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe807fff700 1 -- 172.21.15.3:0/3485811214 learned_addr learned my addr 172.21.15.3:0/3485811214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:41.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a1157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.810+0000 7fe807fff700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a1157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 -- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 msgr2=0x5609a1157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a1157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 -- 172.21.15.3:0/3485811214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 msgr2=0x5609a1157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:41.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a1157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 -- 172.21.15.3:0/3485811214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f8032040 con 0x5609a1151ac0 2023-08-11T14:03:41.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a1157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:41.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe807fff700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009990 tx=0x7fe8000042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff194005a916c10 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe7eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f802f070 con 0x5609a1151ac0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609a109c7d0 con 0x5609a1151ac0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe7f80050b0 con 0x5609a1151ac0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f8005270 con 0x5609a1151ac0 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe829720b80 10 client.15363 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:41.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.811+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609a0ecfd90 con 0x5609a1151ac0 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609a109c7d0 con 0x5609a1151ac0 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7f802a450 con 0x5609a1151ac0 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 10 client.15363 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 20 client.15363 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a11c3bf0 0x5609a11c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 4 client.15363 mds_command: new command op to 24511 tid=0[{"prefix": "status"}] 2023-08-11T14:03:41.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5609a105d000 con 0x5609a11c3bf0 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.812+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a11c3bf0 0x5609a11c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe80f7fe700 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a11c3bf0 0x5609a11c0070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a50dd0a7c7c6bd69 server_cookie=1f8fb40fa8ae9ea7 in_seq=0 out_seq=0 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe7eeffd700 10 client.15363 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe7eeffd700 1 -- 172.21.15.3:0/3485811214 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5609a105d000 con 0x5609a11c3bf0 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe7eeffd700 10 client.15363 handle_command_reply: tid=0 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 client.15363 shutdown 2023-08-11T14:03:41.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 msgr2=0x5609a1157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009990 tx=0x7fe8000042c0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 shutdown_connections 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x5609a11c3bf0 0x5609a11c0070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609a1152e40 0x5609a1157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609a1151ac0 0x5609a1157030 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 --2- 172.21.15.3:0/3485811214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609a1152420 0x5609a1157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe7eeffd700 0 client.15363 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:03:41.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 >> 172.21.15.3:0/3485811214 conn(0x5609a107e900 msgr2=0x5609a1159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.813+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 shutdown_connections 2023-08-11T14:03:41.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe829720b80 1 -- 172.21.15.3:0/3485811214 wait complete. 2023-08-11T14:03:41.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe829720b80 20 client.15363 trim_cache size 0 max 16384 2023-08-11T14:03:41.832 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24511, 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 102.51179852200001, 2023-08-11T14:03:41.833 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:41.834 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:41.834 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:41.834 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 147.808639916 2023-08-11T14:03:41.834 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 msgr2=0x7fe7fc0a24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 0x7fe7fc0a24b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8080a0150 tx=0x7fe808003b60 comp rx=0 tx=0).stop 2023-08-11T14:03:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 msgr2=0x7fe81019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 0x7fe81019d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe80800efb0 tx=0x7fe80800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 shutdown_connections 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe810102320 0x7fe81019d9c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe7fc09fff0 0x7fe7fc0a24b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810100790 0x7fe810196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 --2- 172.21.15.3:0/3503869935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe810101980 0x7fe810196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 >> 172.21.15.3:0/3503869935 conn(0x7fe8100fbd00 msgr2=0x7fe810104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:41.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 shutdown_connections 2023-08-11T14:03:41.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:41.814+0000 7fe814b9d700 1 -- 172.21.15.3:0/3503869935 wait complete. 2023-08-11T14:03:41.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-08-11T14:03:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 -- 172.21.15.3:0/3260731247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 msgr2=0x7f2188100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 --2- 172.21.15.3:0/3260731247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188100d30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f21840099c0 tx=0x7f2184009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 -- 172.21.15.3:0/3260731247 shutdown_connections 2023-08-11T14:03:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 --2- 172.21.15.3:0/3260731247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f2188108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 --2- 172.21.15.3:0/3260731247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188100d30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 --2- 172.21.15.3:0/3260731247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 0x7f2188101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 -- 172.21.15.3:0/3260731247 >> 172.21.15.3:0/3260731247 conn(0x7f21880fbe60 msgr2=0x7f21880fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 -- 172.21.15.3:0/3260731247 shutdown_connections 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.041+0000 7f218f5ff700 1 -- 172.21.15.3:0/3260731247 wait complete. 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.042+0000 7f218f5ff700 1 Processor -- start 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.042+0000 7f218f5ff700 1 -- start start 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 0x7f2188198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881a0410 con 0x7f21881024a0 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881a0550 con 0x7f2188100910 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218f5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881a06e0 con 0x7f2188101b00 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33226/0 (socket says 172.21.15.3:33226) 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 -- 172.21.15.3:0/2082279245 learned_addr learned my addr 172.21.15.3:0/2082279245 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218cb9a700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 0x7f2188198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218d39b700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 -- 172.21.15.3:0/2082279245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 msgr2=0x7f2188198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 0x7f2188198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 -- 172.21.15.3:0/2082279245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 msgr2=0x7f2188198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.043+0000 7f218db9c700 1 -- 172.21.15.3:0/2082279245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840095f0 con 0x7f21881024a0 2023-08-11T14:03:42.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f218db9c700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f218000ef30 tx=0x7f218000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f217e7fc700 1 -- 172.21.15.3:0/2082279245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180018070 con 0x7f21881024a0 2023-08-11T14:03:42.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21881a0960 con 0x7f21881024a0 2023-08-11T14:03:42.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f217e7fc700 1 -- 172.21.15.3:0/2082279245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218000f040 con 0x7f21881024a0 2023-08-11T14:03:42.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f217e7fc700 1 -- 172.21.15.3:0/2082279245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218000cd00 con 0x7f21881024a0 2023-08-11T14:03:42.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.044+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21881a0e70 con 0x7f21881024a0 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.047+0000 7f217e7fc700 1 -- 172.21.15.3:0/2082279245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2180003a00 con 0x7f21881024a0 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 Processor -- start 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f217e7fc700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 0x7f21740a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 -- start start 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f217e7fc700 1 -- 172.21.15.3:0/2082279245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f21800cb0a0 con 0x7f21881024a0 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f218cb9a700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 0x7f21740a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14a3e0 0x55807f14a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f0a25e0 con 0x55807f14a3e0 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f09ff00 con 0x55807f14ada0 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f09b7f0 con 0x55807f149a80 2023-08-11T14:03:42.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f218db9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f218db9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43474/0 (socket says 172.21.15.3:43474) 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f218db9c700 1 -- 172.21.15.3:0/539893328 learned_addr learned my addr 172.21.15.3:0/539893328 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.049+0000 7f218cb9a700 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14a3e0 0x55807f14a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218d39b700 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 -- 172.21.15.3:0/539893328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 msgr2=0x55807f149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 -- 172.21.15.3:0/539893328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14a3e0 msgr2=0x55807f14a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14a3e0 0x55807f14a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 -- 172.21.15.3:0/539893328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55807ef1b5e0 con 0x55807f14ada0 2023-08-11T14:03:42.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218db9c700 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21800090a0 tx=0x7f21800986d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9fd2b42f06bbb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f218cb9a700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 0x7f21740a2490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2178013570 tx=0x7f2178013930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f21737fe700 1 -- 172.21.15.3:0/539893328 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180004ad0 con 0x55807f14ada0 2023-08-11T14:03:42.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f21737fe700 1 -- 172.21.15.3:0/539893328 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218001b320 con 0x55807f14ada0 2023-08-11T14:03:42.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.050+0000 7f21737fe700 1 -- 172.21.15.3:0/539893328 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180098a60 con 0x55807f14ada0 2023-08-11T14:03:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 -- 172.21.15.3:0/539893328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 msgr2=0x55807f14b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f21800090a0 tx=0x7f21800986d0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 -- 172.21.15.3:0/539893328 shutdown_connections 2023-08-11T14:03:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14ada0 0x55807f14b220 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/539893328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14a3e0 0x55807f14a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 -- 172.21.15.3:0/539893328 >> 172.21.15.3:0/539893328 conn(0x55807f076900 msgr2=0x55807f1443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 -- 172.21.15.3:0/539893328 shutdown_connections 2023-08-11T14:03:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.051+0000 7f21a2e59b80 1 -- 172.21.15.3:0/539893328 wait complete. 2023-08-11T14:03:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 -- start start 2023-08-11T14:03:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 Processor -- start 2023-08-11T14:03:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14a3e0 0x55807f153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f039a70 con 0x55807f14ada0 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f039770 con 0x55807f149a80 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807ef1d8d0 con 0x55807f14a3e0 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.052+0000 7f21a2e59b80 10 client.0 _ll_register_callbacks cb 0x55807f03ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218db9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43476/0 (socket says 172.21.15.3:43476) 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 -- 172.21.15.3:0/545333841 learned_addr learned my addr 172.21.15.3:0/545333841 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218cb9a700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14a3e0 0x55807f153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 -- 172.21.15.3:0/545333841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14a3e0 msgr2=0x55807f153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14a3e0 0x55807f153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 -- 172.21.15.3:0/545333841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 msgr2=0x55807f15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 -- 172.21.15.3:0/545333841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21881a0e70 con 0x55807f149a80 2023-08-11T14:03:42.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218db9c700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f218d39b700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21840099c0 tx=0x7f218400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df242f2d049d6108 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.053+0000 7f2170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184003be0 con 0x55807f149a80 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2184003d40 con 0x55807f149a80 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55807f13ca40 con 0x55807f149a80 2023-08-11T14:03:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2184005040 con 0x55807f149a80 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 10 client.25412 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55807eec7d90 con 0x55807f149a80 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55807f13ca40 con 0x55807f149a80 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f218400f980 con 0x55807f149a80 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 10 client.25412 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 10 client.25412 resolve_mds: resolved name 'h' to daemon mds.h 2023-08-11T14:03:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 20 client.25412 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.054+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f15ecb0 0x55807f15d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.055+0000 7f21a2e59b80 4 client.25412 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.055+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55807f118490 con 0x55807f15ecb0 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.055+0000 7f218cb9a700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f15ecb0 0x55807f15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.055+0000 7f2170ff9700 10 client.25412 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.055+0000 7f218cb9a700 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f15ecb0 0x55807f15d770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cac561311dcd8ba5 server_cookie=19ebe28f331f4839 in_seq=0 out_seq=0 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.056+0000 7f2170ff9700 1 -- 172.21.15.3:0/545333841 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55807f118490 con 0x55807f15ecb0 2023-08-11T14:03:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.056+0000 7f2170ff9700 10 client.25412 handle_command_reply: tid=0 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.056+0000 7f21a2e59b80 1 client.25412 shutdown 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 msgr2=0x55807f1530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f21840099c0 tx=0x7f218400bea0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 shutdown_connections 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f15ecb0 0x55807f15d770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f149a80 0x55807f1530f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/545333841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14a3e0 0x55807f153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f2170ff9700 0 client.25412 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 >> 172.21.15.3:0/545333841 conn(0x55807f076900 msgr2=0x55807f13ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 shutdown_connections 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 1 -- 172.21.15.3:0/545333841 wait complete. 2023-08-11T14:03:42.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.057+0000 7f21a2e59b80 20 client.25412 trim_cache size 0 max 16384 2023-08-11T14:03:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 Processor -- start 2023-08-11T14:03:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 -- start start 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14ada0 0x55807f14e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f064f30 con 0x55807f149a80 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f058480 con 0x55807f14a3e0 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.075+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f152640 con 0x55807f14ada0 2023-08-11T14:03:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33246/0 (socket says 172.21.15.3:33246) 2023-08-11T14:03:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 -- 172.21.15.3:0/305011413 learned_addr learned my addr 172.21.15.3:0/305011413 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218cb9a700 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218db9c700 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14ada0 0x55807f14e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 -- 172.21.15.3:0/305011413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14ada0 msgr2=0x55807f14e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14ada0 0x55807f14e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 -- 172.21.15.3:0/305011413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 msgr2=0x55807f15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 -- 172.21.15.3:0/305011413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55807eeb0c10 con 0x55807f149a80 2023-08-11T14:03:42.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.076+0000 7f218d39b700 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21840098e0 tx=0x7f2184009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a8de73bae48e8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f2173fff700 1 -- 172.21.15.3:0/305011413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21840039e0 con 0x55807f149a80 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f2173fff700 1 -- 172.21.15.3:0/305011413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218401e050 con 0x55807f149a80 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f2173fff700 1 -- 172.21.15.3:0/305011413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400f680 con 0x55807f149a80 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 -- 172.21.15.3:0/305011413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 msgr2=0x55807f14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21840098e0 tx=0x7f2184009290 comp rx=0 tx=0).stop 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 -- 172.21.15.3:0/305011413 shutdown_connections 2023-08-11T14:03:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f14ada0 0x55807f14e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f149a80 0x55807f14dc60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/305011413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 -- 172.21.15.3:0/305011413 >> 172.21.15.3:0/305011413 conn(0x55807f076900 msgr2=0x55807f151150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 -- 172.21.15.3:0/305011413 shutdown_connections 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.077+0000 7f21a2e59b80 1 -- 172.21.15.3:0/305011413 wait complete. 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 -- start start 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 Processor -- start 2023-08-11T14:03:42.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f154df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f15a360 con 0x55807f14ada0 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f15a4d0 con 0x55807f14a3e0 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807f15a690 con 0x55807f149a80 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.078+0000 7f21a2e59b80 10 client.0 _ll_register_callbacks cb 0x7f2184009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218cb9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33260/0 (socket says 172.21.15.3:33260) 2023-08-11T14:03:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 -- 172.21.15.3:0/3134027798 learned_addr learned my addr 172.21.15.3:0/3134027798 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218d39b700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 -- 172.21.15.3:0/3134027798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 msgr2=0x55807f1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 -- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 msgr2=0x55807f154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 -- 172.21.15.3:0/3134027798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55807eec7d90 con 0x55807f14ada0 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f218db9c700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21800090a0 tx=0x7f2180098260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11984a0a0412a55 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f2170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:42.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21800a02f0 con 0x55807f14ada0 2023-08-11T14:03:42.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2180098800 con 0x55807f14ada0 2023-08-11T14:03:42.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.079+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f218001b320 con 0x55807f14ada0 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2180098f00 con 0x55807f14ada0 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 10 client.15381 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55807eeee200 con 0x55807f14ada0 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21800c8020 con 0x55807f14ada0 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f218009c120 con 0x55807f14ada0 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 10 client.15381 resolve_mds: resolved name 'h' to daemon mds.h 2023-08-11T14:03:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 20 client.15381 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f1b6f30 0x55807f1b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 4 client.15381 mds_command: new command op to 24527 tid=0[{"prefix": "status"}] 2023-08-11T14:03:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55807f0947d0 con 0x55807f1b6f30 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.080+0000 7f218d39b700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f1b6f30 0x55807f1b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.081+0000 7f218d39b700 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f1b6f30 0x55807f1b33c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dae0144def2d398c server_cookie=3db1e8395f434fb4 in_seq=0 out_seq=0 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.081+0000 7f2170ff9700 10 client.15381 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.081+0000 7f2170ff9700 1 -- 172.21.15.3:0/3134027798 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55807f0947d0 con 0x55807f1b6f30 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.081+0000 7f2170ff9700 10 client.15381 handle_command_reply: tid=0 2023-08-11T14:03:42.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.081+0000 7f21a2e59b80 1 client.15381 shutdown 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 msgr2=0x55807f159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21800090a0 tx=0x7f2180098260 comp rx=0 tx=0).stop 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 shutdown_connections 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x55807f1b6f30 0x55807f1b33c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55807f14ada0 0x55807f159e20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55807f149a80 0x55807f1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f2170ff9700 0 client.15381 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:03:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 --2- 172.21.15.3:0/3134027798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55807f14a3e0 0x55807f154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 >> 172.21.15.3:0/3134027798 conn(0x55807f076900 msgr2=0x55807f14dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 shutdown_connections 2023-08-11T14:03:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 1 -- 172.21.15.3:0/3134027798 wait complete. 2023-08-11T14:03:42.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f21a2e59b80 20 client.15381 trim_cache size 0 max 16384 2023-08-11T14:03:42.096 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:42.096 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:42.096 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2023-08-11T14:03:42.096 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24527, 2023-08-11T14:03:42.097 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:42.097 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:42.097 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:42.097 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 103.785980552, 2023-08-11T14:03:42.098 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:42.098 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:42.098 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:42.098 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 145.85684706500001 2023-08-11T14:03:42.098 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 msgr2=0x7f21740a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 0x7f21740a2490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2178013570 tx=0x7f2178013930 comp rx=0 tx=0).stop 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 msgr2=0x7f218819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.082+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f218000ef30 tx=0x7f218000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 shutdown_connections 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21881024a0 0x7f218819fe40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f217409ffd0 0x7f21740a2490 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2188100910 0x7f2188198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 --2- 172.21.15.3:0/2082279245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2188101b00 0x7f2188198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 >> 172.21.15.3:0/2082279245 conn(0x7f21880fbe60 msgr2=0x7f21880fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 shutdown_connections 2023-08-11T14:03:42.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.083+0000 7f218f5ff700 1 -- 172.21.15.3:0/2082279245 wait complete. 2023-08-11T14:03:42.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-08-11T14:03:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 -- 172.21.15.3:0/2458988688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 msgr2=0x7fcac0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 --2- 172.21.15.3:0/2458988688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac0108d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b0d0 tx=0x7fcab800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 -- 172.21.15.3:0/2458988688 shutdown_connections 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 --2- 172.21.15.3:0/2458988688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac0108d50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 --2- 172.21.15.3:0/2458988688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0100910 0x7fcac0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 --2- 172.21.15.3:0/2458988688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0101b00 0x7fcac0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 -- 172.21.15.3:0/2458988688 >> 172.21.15.3:0/2458988688 conn(0x7fcac00fbe60 msgr2=0x7fcac00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 -- 172.21.15.3:0/2458988688 shutdown_connections 2023-08-11T14:03:42.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.305+0000 7fcac61ff700 1 -- 172.21.15.3:0/2458988688 wait complete. 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 Processor -- start 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 -- start start 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0100910 0x7fcac0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0101b00 0x7fcac0198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0490 con 0x7fcac01024a0 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a05d0 con 0x7fcac0101b00 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.306+0000 7fcac61ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0760 con 0x7fcac0100910 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0100910 0x7fcac0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33272/0 (socket says 172.21.15.3:33272) 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 -- 172.21.15.3:0/1772402991 learned_addr learned my addr 172.21.15.3:0/1772402991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabeffd700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0101b00 0x7fcac0198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 -- 172.21.15.3:0/1772402991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0100910 msgr2=0x7fcac0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0100910 0x7fcac0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 -- 172.21.15.3:0/1772402991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0101b00 msgr2=0x7fcac0198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0101b00 0x7fcac0198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 -- 172.21.15.3:0/1772402991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009520 con 0x7fcac01024a0 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.307+0000 7fcabffff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcab8008eb0 tx=0x7fcab8008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.308+0000 7fcabcff9700 1 -- 172.21.15.3:0/1772402991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800e040 con 0x7fcac01024a0 2023-08-11T14:03:42.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.308+0000 7fcabcff9700 1 -- 172.21.15.3:0/1772402991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab800bdb0 con 0x7fcac01024a0 2023-08-11T14:03:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.308+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab8009d00 con 0x7fcac01024a0 2023-08-11T14:03:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.308+0000 7fcabcff9700 1 -- 172.21.15.3:0/1772402991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab801d470 con 0x7fcac01024a0 2023-08-11T14:03:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.308+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a0d20 con 0x7fcac01024a0 2023-08-11T14:03:42.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.311+0000 7fcabcff9700 1 -- 172.21.15.3:0/1772402991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcab8005020 con 0x7fcac01024a0 2023-08-11T14:03:42.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.312+0000 7fcad9ad3b80 1 Processor -- start 2023-08-11T14:03:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.312+0000 7fcabcff9700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 0x7fcaa80a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabcff9700 1 -- 172.21.15.3:0/1772402991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcab80cc460 con 0x7fcac01024a0 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 -- start start 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabeffd700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 0x7fcaa80a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde3f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd8f5e0 con 0x55abbde3f290 2023-08-11T14:03:42.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd8cf00 con 0x55abbde405b0 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd887f0 con 0x55abbde3fbf0 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde3f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde3f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33276/0 (socket says 172.21.15.3:33276) 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.313+0000 7fcabf7fe700 1 -- 172.21.15.3:0/2863442954 learned_addr learned my addr 172.21.15.3:0/2863442954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabeffd700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 0x7fcaa80a2490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009cb0 tx=0x7fcab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 -- 172.21.15.3:0/2863442954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 msgr2=0x55abbde40070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde40070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 -- 172.21.15.3:0/2863442954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 msgr2=0x55abbde3f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde3f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 -- 172.21.15.3:0/2863442954 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbdc085e0 con 0x55abbde405b0 2023-08-11T14:03:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.314+0000 7fcabffff700 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde40a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcab80c9be0 tx=0x7fcab80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb760b3f77fda41 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.318+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2863442954 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8099700 con 0x55abbde405b0 2023-08-11T14:03:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.318+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2863442954 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab80cc020 con 0x55abbde405b0 2023-08-11T14:03:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.318+0000 7fcaa5ffb700 1 -- 172.21.15.3:0/2863442954 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80a1940 con 0x55abbde405b0 2023-08-11T14:03:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.318+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/2863442954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 msgr2=0x55abbde40a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.318+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde40a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcab80c9be0 tx=0x7fcab80099e0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/2863442954 shutdown_connections 2023-08-11T14:03:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde40a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde3f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/2863442954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/2863442954 >> 172.21.15.3:0/2863442954 conn(0x55abbdd63900 msgr2=0x55abbde313d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/2863442954 shutdown_connections 2023-08-11T14:03:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/2863442954 wait complete. 2023-08-11T14:03:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.319+0000 7fcad9ad3b80 1 -- start start 2023-08-11T14:03:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 Processor -- start 2023-08-11T14:03:42.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3f290 0x55abbde486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3fbf0 0x55abbde48c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd26a70 con 0x55abbde405b0 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdd26770 con 0x55abbde3fbf0 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.320+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbdc0a8d0 con 0x55abbde3f290 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcad9ad3b80 10 client.0 _ll_register_callbacks cb 0x55abbdd29a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33288/0 (socket says 172.21.15.3:33288) 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 -- 172.21.15.3:0/3784400440 learned_addr learned my addr 172.21.15.3:0/3784400440 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3f290 0x55abbde486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3fbf0 0x55abbde48c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 -- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3f290 msgr2=0x55abbde486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3f290 0x55abbde486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 -- 172.21.15.3:0/3784400440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3fbf0 msgr2=0x55abbde48c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3fbf0 0x55abbde48c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 -- 172.21.15.3:0/3784400440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a0d20 con 0x55abbde405b0 2023-08-11T14:03:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fcabffff700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab8096630 tx=0x7fcab800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda4dde32f376493 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.321+0000 7fca9effd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8099520 con 0x55abbde405b0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcab8011e50 con 0x55abbde405b0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abbde29a40 con 0x55abbde405b0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab80c9070 con 0x55abbde405b0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 10 client.15393 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abbdbb4d90 con 0x55abbde405b0 2023-08-11T14:03:42.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abbde29a40 con 0x55abbde405b0 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcab80c3d20 con 0x55abbde405b0 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 10 client.15393 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 20 client.15393 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.322+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbde53da0 0x55abbde52ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.323+0000 7fcad9ad3b80 4 client.15393 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.323+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abbde05490 con 0x55abbde53da0 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.323+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbde53da0 0x55abbde52ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.323+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbde53da0 0x55abbde52ba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77ae25b1b5a2be76 server_cookie=d9166738374f2f00 in_seq=0 out_seq=0 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.323+0000 7fca9effd700 10 client.15393 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.324+0000 7fca9effd700 1 -- 172.21.15.3:0/3784400440 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55abbde05490 con 0x55abbde53da0 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.324+0000 7fca9effd700 10 client.15393 handle_command_reply: tid=0 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 client.15393 shutdown 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 msgr2=0x55abbde4fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcab8096630 tx=0x7fcab800bf20 comp rx=0 tx=0).stop 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 shutdown_connections 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbde53da0 0x55abbde52ba0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde405b0 0x55abbde4fc70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3f290 0x55abbde486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fca9effd700 0 client.15393 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3784400440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3fbf0 0x55abbde48c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 >> 172.21.15.3:0/3784400440 conn(0x55abbdd63900 msgr2=0x55abbde2bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 shutdown_connections 2023-08-11T14:03:42.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3784400440 wait complete. 2023-08-11T14:03:42.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.325+0000 7fcad9ad3b80 20 client.15393 trim_cache size 0 max 16384 2023-08-11T14:03:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.343+0000 7fcad9ad3b80 1 Processor -- start 2023-08-11T14:03:42.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.343+0000 7fcad9ad3b80 1 -- start start 2023-08-11T14:03:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde31340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde47200 con 0x55abbde3f290 2023-08-11T14:03:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde05490 con 0x55abbde405b0 2023-08-11T14:03:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde32a90 con 0x55abbde3fbf0 2023-08-11T14:03:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:53780/0 (socket says 172.21.15.3:53780) 2023-08-11T14:03:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcabeffd700 1 -- 172.21.15.3:0/3107721693 learned_addr learned my addr 172.21.15.3:0/3107721693 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.344+0000 7fcabffff700 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde31340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 -- 172.21.15.3:0/3107721693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 msgr2=0x55abbde31340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde31340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 -- 172.21.15.3:0/3107721693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 msgr2=0x55abbde2acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde2acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 -- 172.21.15.3:0/3107721693 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbdb9dc10 con 0x55abbde3fbf0 2023-08-11T14:03:42.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcab40043d0 tx=0x7fcab4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bd440ef08d2269 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcaa67fc700 1 -- 172.21.15.3:0/3107721693 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab402a070 con 0x55abbde3fbf0 2023-08-11T14:03:42.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcaa67fc700 1 -- 172.21.15.3:0/3107721693 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab4022040 con 0x55abbde3fbf0 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.345+0000 7fcaa67fc700 1 -- 172.21.15.3:0/3107721693 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4021070 con 0x55abbde3fbf0 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3107721693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 msgr2=0x55abbde5c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcab40043d0 tx=0x7fcab4014000 comp rx=0 tx=0).stop 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3107721693 shutdown_connections 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde405b0 0x55abbde31340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3f290 0x55abbde2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3107721693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde3fbf0 0x55abbde5c700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3107721693 >> 172.21.15.3:0/3107721693 conn(0x55abbdd63900 msgr2=0x55abbde474a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3107721693 shutdown_connections 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3107721693 wait complete. 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.346+0000 7fcad9ad3b80 1 -- start start 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 Processor -- start 2023-08-11T14:03:42.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 0x55abbde56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde405b0 0x55abbde5c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde5c700 con 0x55abbde3fbf0 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde5c870 con 0x55abbde3f290 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbde5ca30 con 0x55abbde405b0 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcad9ad3b80 10 client.0 _ll_register_callbacks cb 0x55abbdd29a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 0x55abbde56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33298/0 (socket says 172.21.15.3:33298) 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.347+0000 7fcabeffd700 1 -- 172.21.15.3:0/3860486923 learned_addr learned my addr 172.21.15.3:0/3860486923 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabffff700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde405b0 0x55abbde5c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 -- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde405b0 msgr2=0x55abbde5c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde405b0 0x55abbde5c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 -- 172.21.15.3:0/3860486923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 msgr2=0x55abbde572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 -- 172.21.15.3:0/3860486923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0009db0 con 0x55abbde3f290 2023-08-11T14:03:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcabf7fe700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 0x55abbde56d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55abbde55430 tx=0x7fcab00125d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e90cccaceba5e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fca9effd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:42.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0017040 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcab80cc020 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.348+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcab0021370 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab0012a20 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 10 client.25436 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fcab4017050 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab80cc020 con 0x55abbde3f290 2023-08-11T14:03:42.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcab0010ab0 con 0x55abbde3f290 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 10 client.25436 resolve_mds: resolved name 'i' to daemon mds.i 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 20 client.25436 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbdeac5f0 0x55abbdea8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 4 client.25436 mds_command: new command op to 24532 tid=0[{"prefix": "status"}] 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.349+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55abbdd817d0 con 0x55abbdeac5f0 2023-08-11T14:03:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbdeac5f0 0x55abbdea8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fcabeffd700 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbdeac5f0 0x55abbdea8a50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b07f042403d860b server_cookie=179532dd2a28665f in_seq=0 out_seq=0 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fca9effd700 10 client.25436 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fca9effd700 1 -- 172.21.15.3:0/3860486923 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55abbdd817d0 con 0x55abbdeac5f0 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fca9effd700 10 client.25436 handle_command_reply: tid=0 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fcad9ad3b80 1 client.25436 shutdown 2023-08-11T14:03:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 msgr2=0x55abbde56d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.350+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 0x55abbde56d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55abbde55430 tx=0x7fcab00125d0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 shutdown_connections 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x55abbdeac5f0 0x55abbdea8a50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abbde405b0 0x55abbde5c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abbde3f290 0x55abbde56d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 --2- 172.21.15.3:0/3860486923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abbde3fbf0 0x55abbde572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fca9effd700 0 client.25436 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:03:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 >> 172.21.15.3:0/3860486923 conn(0x55abbdd63900 msgr2=0x55abbde52ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 shutdown_connections 2023-08-11T14:03:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 1 -- 172.21.15.3:0/3860486923 wait complete. 2023-08-11T14:03:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcad9ad3b80 20 client.25436 trim_cache size 0 max 16384 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24532, 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:42.365 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 105.06805566, 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 143.88576312399999 2023-08-11T14:03:42.366 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 msgr2=0x7fcaa80a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 0x7fcaa80a2490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009cb0 tx=0x7fcab4009260 comp rx=0 tx=0).stop 2023-08-11T14:03:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 msgr2=0x7fcac019fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.351+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcab8008eb0 tx=0x7fcab8008ee0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 shutdown_connections 2023-08-11T14:03:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac01024a0 0x7fcac019fec0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcac0100910 0x7fcac0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcaa809ffd0 0x7fcaa80a2490 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 --2- 172.21.15.3:0/1772402991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcac0101b00 0x7fcac0198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 >> 172.21.15.3:0/1772402991 conn(0x7fcac00fbe60 msgr2=0x7fcac01073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 shutdown_connections 2023-08-11T14:03:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.352+0000 7fcac61ff700 1 -- 172.21.15.3:0/1772402991 wait complete. 2023-08-11T14:03:42.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-11T14:03:42.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 -- 172.21.15.3:0/1063374386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44100910 msgr2=0x7efc44100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/1063374386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44100910 0x7efc44100d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efc400099c0 tx=0x7efc40009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 -- 172.21.15.3:0/1063374386 shutdown_connections 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/1063374386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc441024a0 0x7efc44108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/1063374386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44100910 0x7efc44100d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/1063374386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc44101b00 0x7efc44101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 -- 172.21.15.3:0/1063374386 >> 172.21.15.3:0/1063374386 conn(0x7efc440fbe60 msgr2=0x7efc440fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 -- 172.21.15.3:0/1063374386 shutdown_connections 2023-08-11T14:03:42.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.576+0000 7efc4b9ff700 1 -- 172.21.15.3:0/1063374386 wait complete. 2023-08-11T14:03:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.577+0000 7efc4b9ff700 1 Processor -- start 2023-08-11T14:03:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.577+0000 7efc4b9ff700 1 -- start start 2023-08-11T14:03:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 0x7efc441965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 0x7efc4419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4419e170 con 0x7efc44101b00 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4419e2b0 con 0x7efc44100910 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4b9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4419e440 con 0x7efc441024a0 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4979b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 0x7efc441965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33318/0 (socket says 172.21.15.3:33318) 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc4979b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 0x7efc441965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43570/0 (socket says 172.21.15.3:43570) 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc48f9a700 1 -- 172.21.15.3:0/77878466 learned_addr learned my addr 172.21.15.3:0/77878466 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.578+0000 7efc49f9c700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 0x7efc4419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 -- 172.21.15.3:0/77878466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 msgr2=0x7efc4419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 0x7efc4419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 -- 172.21.15.3:0/77878466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 msgr2=0x7efc441965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 0x7efc441965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 -- 172.21.15.3:0/77878466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400095f0 con 0x7efc44101b00 2023-08-11T14:03:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc49f9c700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 0x7efc4419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc48f9a700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc3400e9a0 tx=0x7efc3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc3a7fc700 1 -- 172.21.15.3:0/77878466 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34009790 con 0x7efc44101b00 2023-08-11T14:03:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc3a7fc700 1 -- 172.21.15.3:0/77878466 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc3400cbd0 con 0x7efc44101b00 2023-08-11T14:03:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4419e720 con 0x7efc44101b00 2023-08-11T14:03:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc3a7fc700 1 -- 172.21.15.3:0/77878466 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34018680 con 0x7efc44101b00 2023-08-11T14:03:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.579+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4419ec30 con 0x7efc44101b00 2023-08-11T14:03:42.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.583+0000 7efc3a7fc700 1 -- 172.21.15.3:0/77878466 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efc3400cd40 con 0x7efc44101b00 2023-08-11T14:03:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 Processor -- start 2023-08-11T14:03:42.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 -- start start 2023-08-11T14:03:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd6135e0 con 0x5646dd6c01b0 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd610f00 con 0x5646dd6bf7f0 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd60c7f0 con 0x5646dd6bee90 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33328/0 (socket says 172.21.15.3:33328) 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc4979b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc48f9a700 1 -- 172.21.15.3:0/1007906566 learned_addr learned my addr 172.21.15.3:0/1007906566 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.584+0000 7efc49f9c700 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc3a7fc700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 0x7efc300a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 -- 172.21.15.3:0/1007906566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 msgr2=0x5646dd6bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 -- 172.21.15.3:0/1007906566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 msgr2=0x5646dd6bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 -- 172.21.15.3:0/1007906566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dd48c5e0 con 0x5646dd6c01b0 2023-08-11T14:03:42.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc4979b700 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc3a7fc700 1 -- 172.21.15.3:0/77878466 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efc34014070 con 0x7efc44101b00 2023-08-11T14:03:42.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc49f9c700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 0x7efc300a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.585+0000 7efc48f9a700 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc3405b200 tx=0x7efc340953a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04ba3e38abd7be0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc2f7fe700 1 -- 172.21.15.3:0/1007906566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34098a70 con 0x5646dd6c01b0 2023-08-11T14:03:42.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc2f7fe700 1 -- 172.21.15.3:0/1007906566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc34098890 con 0x5646dd6c01b0 2023-08-11T14:03:42.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc2f7fe700 1 -- 172.21.15.3:0/1007906566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3409cdf0 con 0x5646dd6c01b0 2023-08-11T14:03:42.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/1007906566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 msgr2=0x5646dd6c0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc49f9c700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 0x7efc300a2410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009ef0 tx=0x7efc3c004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.586+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc3405b200 tx=0x7efc340953a0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/1007906566 shutdown_connections 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6c01b0 0x5646dd6c0610 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/1007906566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/1007906566 >> 172.21.15.3:0/1007906566 conn(0x5646dd5e7900 msgr2=0x5646dd6b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/1007906566 shutdown_connections 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/1007906566 wait complete. 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 -- start start 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.587+0000 7efc5f2d0b80 1 Processor -- start 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 0x5646dd6c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd5aaa70 con 0x5646dd6bf7f0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd5aa770 con 0x5646dd6bee90 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd48e8d0 con 0x5646dd6c01b0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc5f2d0b80 10 client.0 _ll_register_callbacks cb 0x5646dd5ada60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc49f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc4979b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 0x5646dd6c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc49f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:53838/0 (socket says 172.21.15.3:53838) 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.588+0000 7efc49f9c700 1 -- 172.21.15.3:0/3842523000 learned_addr learned my addr 172.21.15.3:0/3842523000 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 -- 172.21.15.3:0/3842523000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 msgr2=0x5646dd6cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 -- 172.21.15.3:0/3842523000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 msgr2=0x5646dd6c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 -- 172.21.15.3:0/3842523000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc441096a0 con 0x5646dd6bee90 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc49f9c700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc48f9a700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:42.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc4979b700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 0x5646dd6c83c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc400098e0 tx=0x7efc40005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fabcf7ad2d6af2a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:42.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40003da0 con 0x5646dd6bee90 2023-08-11T14:03:42.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646dd6ada40 con 0x5646dd6bee90 2023-08-11T14:03:42.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efc40005040 con 0x5646dd6bee90 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc40005200 con 0x5646dd6bee90 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc5f2d0b80 10 client.25448 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.589+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efc4419ec30 con 0x5646dd6bee90 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646dd6ada40 con 0x5646dd6bee90 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc40024bf0 con 0x5646dd6bee90 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 10 client.25448 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:03:42.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 20 client.25448 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd6d3f60 0x5646dd6d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 4 client.25448 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646dd689490 con 0x5646dd6d3f60 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.590+0000 7efc48f9a700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd6d3f60 0x5646dd6d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.591+0000 7efc48f9a700 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd6d3f60 0x5646dd6d2a90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f131513674d426cb server_cookie=73c02c9d4c633d01 in_seq=0 out_seq=0 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.591+0000 7efc2cff9700 10 client.25448 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:42.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc2cff9700 1 -- 172.21.15.3:0/3842523000 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5646dd689490 con 0x5646dd6d3f60 2023-08-11T14:03:42.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc2cff9700 10 client.25448 handle_command_reply: tid=0 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 client.25448 shutdown 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 msgr2=0x5646dd6c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 0x5646dd6c83c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc400098e0 tx=0x7efc40005c40 comp rx=0 tx=0).stop 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 shutdown_connections 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd6d3f60 0x5646dd6d2a90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bee90 0x5646dd6c83c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/3842523000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc2cff9700 0 client.25448 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 >> 172.21.15.3:0/3842523000 conn(0x5646dd5e7900 msgr2=0x5646dd6afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 shutdown_connections 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.592+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/3842523000 wait complete. 2023-08-11T14:03:42.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.593+0000 7efc5f2d0b80 20 client.25448 trim_cache size 0 max 16384 2023-08-11T14:03:42.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 Processor -- start 2023-08-11T14:03:42.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 -- start start 2023-08-11T14:03:42.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd5a6d50 con 0x5646dd6bee90 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd5d5ef0 con 0x5646dd6bf7f0 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd5c9480 con 0x5646dd6c01b0 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc4979b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.611+0000 7efc4979b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33340/0 (socket says 172.21.15.3:33340) 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 -- 172.21.15.3:0/788428235 learned_addr learned my addr 172.21.15.3:0/788428235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc49f9c700 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 -- 172.21.15.3:0/788428235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 msgr2=0x5646dd6c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 -- 172.21.15.3:0/788428235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 msgr2=0x5646dd6cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 -- 172.21.15.3:0/788428235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dd438d90 con 0x5646dd6bee90 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc4979b700 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc40024f80 tx=0x7efc40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c2af1edde697b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc2ffff700 1 -- 172.21.15.3:0/788428235 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40005940 con 0x5646dd6bee90 2023-08-11T14:03:42.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc2ffff700 1 -- 172.21.15.3:0/788428235 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc4001e040 con 0x5646dd6bee90 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.612+0000 7efc2ffff700 1 -- 172.21.15.3:0/788428235 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40005330 con 0x5646dd6bee90 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/788428235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 msgr2=0x5646dd6c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc40024f80 tx=0x7efc40000bc0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/788428235 shutdown_connections 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6c01b0 0x5646dd6c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bee90 0x5646dd6c7230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/788428235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6bf7f0 0x5646dd6cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/788428235 >> 172.21.15.3:0/788428235 conn(0x5646dd5e7900 msgr2=0x5646dd6b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/788428235 shutdown_connections 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/788428235 wait complete. 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.613+0000 7efc5f2d0b80 1 -- start start 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 Processor -- start 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd6cfa10 con 0x5646dd6bf7f0 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd6cfb80 con 0x5646dd6c01b0 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dd6cfd40 con 0x5646dd6bee90 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc5f2d0b80 10 client.0 _ll_register_callbacks cb 0x5646dd5ada60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc48f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33348/0 (socket says 172.21.15.3:33348) 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.614+0000 7efc48f9a700 1 -- 172.21.15.3:0/2198219527 learned_addr learned my addr 172.21.15.3:0/2198219527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:42.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc49f9c700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc4979b700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 -- 172.21.15.3:0/2198219527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 msgr2=0x5646dd6ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 -- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6c01b0 msgr2=0x5646dd6cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 -- 172.21.15.3:0/2198219527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4419ec30 con 0x5646dd6bf7f0 2023-08-11T14:03:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc48f9a700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc3404d0a0 tx=0x7efc340951a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912b50b820d5dda3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:42.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc2cff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340a0d40 con 0x5646dd6bf7f0 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc34098890 con 0x5646dd6bf7f0 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.615+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efc3405f0f0 con 0x5646dd6bf7f0 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc34099190 con 0x5646dd6bf7f0 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 10 client.15417 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:42.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5646dd421c10 con 0x5646dd6bf7f0 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc34098890 con 0x5646dd6bf7f0 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc340a08b0 con 0x5646dd6bf7f0 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 10 client.15417 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 20 client.15417 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd72c590 0x5646dd7289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 4 client.15417 mds_command: new command op to 24569 tid=0[{"prefix": "status"}] 2023-08-11T14:03:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5646dd6057d0 con 0x5646dd72c590 2023-08-11T14:03:42.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.616+0000 7efc4979b700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd72c590 0x5646dd7289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc4979b700 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd72c590 0x5646dd7289f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb03788455af70d2 server_cookie=b9b78e211ccd05e0 in_seq=0 out_seq=0 2023-08-11T14:03:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc2cff9700 10 client.15417 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc2cff9700 1 -- 172.21.15.3:0/2198219527 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5646dd6057d0 con 0x5646dd72c590 2023-08-11T14:03:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc2cff9700 10 client.15417 handle_command_reply: tid=0 2023-08-11T14:03:42.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 client.15417 shutdown 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 msgr2=0x5646dd6ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efc3404d0a0 tx=0x7efc340951a0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 shutdown_connections 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5646dd72c590 0x5646dd7289f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646dd6c01b0 0x5646dd6cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc2cff9700 0 client.15417 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:42.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5646dd6bee90 0x5646dd6ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 --2- 172.21.15.3:0/2198219527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5646dd6bf7f0 0x5646dd6ca600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 >> 172.21.15.3:0/2198219527 conn(0x5646dd5e7900 msgr2=0x5646dd6c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.617+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 shutdown_connections 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc5f2d0b80 1 -- 172.21.15.3:0/2198219527 wait complete. 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc5f2d0b80 20 client.15417 trim_cache size 0 max 16384 2023-08-11T14:03:42.629 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:42.630 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "69ee59aa-384e-11ee-9b37-001a4aab830c", 2023-08-11T14:03:42.630 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-08-11T14:03:42.630 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24569, 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 129.28852699399999, 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27, 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2023-08-11T14:03:42.631 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2023-08-11T14:03:42.632 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 135.52609575299999 2023-08-11T14:03:42.632 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:42.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 msgr2=0x7efc300a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 0x7efc300a2410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009ef0 tx=0x7efc3c004ad0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 msgr2=0x7efc44196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc3400e9a0 tx=0x7efc3400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 shutdown_connections 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc441024a0 0x7efc4419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efc3009ff50 0x7efc300a2410 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44100910 0x7efc441965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 --2- 172.21.15.3:0/77878466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc44101b00 0x7efc44196b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 >> 172.21.15.3:0/77878466 conn(0x7efc440fbe60 msgr2=0x7efc440fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.618+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 shutdown_connections 2023-08-11T14:03:42.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:42.619+0000 7efc4b9ff700 1 -- 172.21.15.3:0/77878466 wait complete. 2023-08-11T14:03:42.638 INFO:teuthology.run_tasks:Running task kclient... 2023-08-11T14:03:42.654 INFO:tasks.kclient:Mounting kernel clients... 2023-08-11T14:03:42.654 INFO:tasks.kclient:config is {'client.0': {}} 2023-08-11T14:03:42.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:42.655 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-08-11T14:03:42.699 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-08-11T14:03:42.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:42.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-08-11T14:03:42.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:42.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-08-11T14:03:42.920 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-08-11T14:03:42.921 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-11T14:03:42.921 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-08-11T14:03:42.921 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-08-11T14:03:42.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph fs ls 2023-08-11T14:03:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:03:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:43 smithi103 ceph-mon[130434]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:43.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:43 smithi003 ceph-mon[97195]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:43 smithi187 ceph-mon[130973]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:44.668 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:44 smithi003 ceph-mon[97195]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:44.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:44 smithi187 ceph-mon[130973]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:44.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:44 smithi103 ceph-mon[130434]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-08-11T14:03:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 -- 172.21.15.3:0/3778312879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 msgr2=0x7f092c10d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:45.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 --2- 172.21.15.3:0/3778312879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c10d810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0924009f90 tx=0x7f092400b330 comp rx=0 tx=0).stop 2023-08-11T14:03:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 -- 172.21.15.3:0/3778312879 shutdown_connections 2023-08-11T14:03:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 --2- 172.21.15.3:0/3778312879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c10d810 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 --2- 172.21.15.3:0/3778312879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c106940 0x7f092c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 --2- 172.21.15.3:0/3778312879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c100940 0x7f092c100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 -- 172.21.15.3:0/3778312879 >> 172.21.15.3:0/3778312879 conn(0x7f092c0fc410 msgr2=0x7f092c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 -- 172.21.15.3:0/3778312879 shutdown_connections 2023-08-11T14:03:45.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.318+0000 7f09324e8700 1 -- 172.21.15.3:0/3778312879 wait complete. 2023-08-11T14:03:45.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.319+0000 7f09324e8700 1 Processor -- start 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.319+0000 7f09324e8700 1 -- start start 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c100940 0x7f092c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c106940 0x7f092c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19dcf0 con 0x7f092c101300 2023-08-11T14:03:45.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19de60 con 0x7f092c106940 2023-08-11T14:03:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f09324e8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c19e020 con 0x7f092c100940 2023-08-11T14:03:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f092b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33366/0 (socket says 172.21.15.3:33366) 2023-08-11T14:03:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f092b7fe700 1 -- 172.21.15.3:0/2150063513 learned_addr learned my addr 172.21.15.3:0/2150063513 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:45.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f092bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c100940 0x7f092c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.320+0000 7f0930a85700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c106940 0x7f092c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 -- 172.21.15.3:0/2150063513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c100940 msgr2=0x7f092c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c100940 0x7f092c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 -- 172.21.15.3:0/2150063513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c106940 msgr2=0x7f092c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c106940 0x7f092c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 -- 172.21.15.3:0/2150063513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200095f0 con 0x7f092c101300 2023-08-11T14:03:45.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f092b7fe700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f092000ec10 tx=0x7f092000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000cc90 con 0x7f092c101300 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09200044e0 con 0x7f092c101300 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924009c40 con 0x7f092c101300 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.321+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920010430 con 0x7f092c101300 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.322+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c1a5990 con 0x7f092c101300 2023-08-11T14:03:45.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.322+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c04e620 con 0x7f092c101300 2023-08-11T14:03:45.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.324+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0920003fb0 con 0x7f092c101300 2023-08-11T14:03:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.326+0000 7f09297fa700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 0x7f09140a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.326+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0920014070 con 0x7f092c101300 2023-08-11T14:03:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.326+0000 7f092bfff700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 0x7f09140a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:45.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.327+0000 7f092bfff700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 0x7f09140a2410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f091c009be0 tx=0x7f091c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:45.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.330+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0920095510 con 0x7f092c101300 2023-08-11T14:03:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.586+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f092c110ac0 con 0x7f092c101300 2023-08-11T14:03:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.587+0000 7f09297fa700 1 -- 172.21.15.3:0/2150063513 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0920098b30 con 0x7f092c101300 2023-08-11T14:03:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-08-11T14:03:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 msgr2=0x7f09140a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 0x7f09140a2410 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f091c009be0 tx=0x7f091c009260 comp rx=0 tx=0).stop 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 msgr2=0x7f092c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f092000ec10 tx=0x7f092000ef20 comp rx=0 tx=0).stop 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 shutdown_connections 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f091409ff50 0x7f09140a2410 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f092c106940 0x7f092c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f092c100940 0x7f092c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 --2- 172.21.15.3:0/2150063513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f092c101300 0x7f092c199270 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 >> 172.21.15.3:0/2150063513 conn(0x7f092c0fc410 msgr2=0x7f092c10bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 shutdown_connections 2023-08-11T14:03:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:45.590+0000 7f09324e8700 1 -- 172.21.15.3:0/2150063513 wait complete. 2023-08-11T14:03:45.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:45 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2150063513' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:03:46.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:45 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2150063513' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:03:46.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:45 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2150063513' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-08-11T14:03:46.055 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-11T14:03:46.056 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-11T14:03:46.056 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-08-11T14:03:46.056 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-11T14:03:46.056 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-08-11T14:03:46.057 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-11T14:03:46.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-11T14:03:46.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-11T14:03:46.057 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-08-11T14:03:46.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.058 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-08-11T14:03:46.106 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-11T14:03:46.106 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-11T14:03:46.107 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-08-11T14:03:46.107 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:03:46.107 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-08-11T14:03:46.107 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:03:46.108 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:03:46.108 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-08-11T14:03:46.110 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41113sec preferred_lft 41113sec 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-08-11T14:03:46.111 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-08-11T14:03:46.113 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-11T14:03:46.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.114 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:03:46.114 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:03:46.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-08-11T14:03:46.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-08-11T14:03:46.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-08-11T14:03:46.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-11T14:03:46.115 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:03:46.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.369 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-11T14:03:46.440 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-08-11T14:03:46.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.443 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-08-11T14:03:46.490 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-08-11T14:03:46.490 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-11T14:03:46.491 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-11T14:03:46.491 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-11T14:03:46.491 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-11T14:03:46.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.494 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:03:46.494 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:03:46.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-11T14:03:46.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-11T14:03:46.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-11T14:03:46.496 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:03:46.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.674 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-08-11T14:03:46.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.719 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-08-11T14:03:46.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.804 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:03:46.804 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:03:46.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--tmp-cephfs 2023-08-11T14:03:46.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-08-11T14:03:46.805 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:03:46.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:46 smithi103 ceph-mon[130434]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:46.945 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:46 smithi003 ceph-mon[97195]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:46.990 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-08-11T14:03:46.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:46.991 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:03:46.991 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:03:46.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-08-11T14:03:46.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-11T14:03:46.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-08-11T14:03:46.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-08-11T14:03:46.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-08-11T14:03:46.993 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:03:47.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:46 smithi187 ceph-mon[130973]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:47.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:47.248 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-08-11T14:03:47.248 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-08-11T14:03:47.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-08-11T14:03:47.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-08-11T14:03:47.249 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-08-11T14:03:47.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T14:03:47.427 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /tmp/cephfs 2023-08-11T14:03:47.471 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/tmp/cephfs' 2023-08-11T14:03:47.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-11T14:03:47.472 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /tmp/cephfs 2023-08-11T14:03:47.555 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-08-11T14:03:47.556 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-08-11T14:03:47.556 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1 2023-08-11T14:03:47.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:47.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-08-11T14:03:47.703 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-08-11T14:03:47.703 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-08-11T14:03:48.154 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-08-11T14:03:48.154 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-08-11T14:03:48.154 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.103:3300,172.21.15.187:3300" 2023-08-11T14:03:48.154 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@69ee59aa-384e-11ee-9b37-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1 2023-08-11T14:03:48.154 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.3:3300/172.21.15.103:3300/172.21.15.187:3300" will pass to kernel 2023-08-11T14:03:48.161 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-11T14:03:48.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:48.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /tmp/cephfs 2023-08-11T14:03:48.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:48.235 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T14:03:48.324 INFO:teuthology.orchestra.run.smithi003.stdout:client15426 2023-08-11T14:03:48.324 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:03:48.325 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:03:48.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:48.370 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T14:03:48.495 INFO:teuthology.orchestra.run.smithi003.stdout:client15426 2023-08-11T14:03:48.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:48.497 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-11T14:03:48.585 INFO:teuthology.orchestra.run.smithi003.stdout:client15426 2023-08-11T14:03:48.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:03:48.585 DEBUG:teuthology.orchestra.run.smithi003:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c.client15426/status') 2023-08-11T14:03:48.695 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.15426 (3)192.168.0.1:0/3392505586 2023-08-11T14:03:48.696 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2023-08-11T14:03:48.696 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2023-08-11T14:03:48.696 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2023-08-11T14:03:48.696 INFO:teuthology.orchestra.run.smithi003.stderr:70 bytes copied, 5.7098e-05 s, 1.2 MB/s 2023-08-11T14:03:48.698 INFO:teuthology.run_tasks:Running task check-counter... 2023-08-11T14:03:48.710 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-08-11T14:03:48.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:48.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 -- 172.21.15.3:0/2134164055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0100920 msgr2=0x7f7ac0100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:48.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/2134164055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0100920 0x7f7ac0100d40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00099c0 tx=0x7f7ab0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 -- 172.21.15.3:0/2134164055 shutdown_connections 2023-08-11T14:03:48.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/2134164055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac01024b0 0x7f7ac0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/2134164055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0100920 0x7f7ac0100d40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/2134164055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac0101b10 0x7f7ac0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 -- 172.21.15.3:0/2134164055 >> 172.21.15.3:0/2134164055 conn(0x7f7ac00fbe50 msgr2=0x7f7ac00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 -- 172.21.15.3:0/2134164055 shutdown_connections 2023-08-11T14:03:48.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.930+0000 7f7ac49ff700 1 -- 172.21.15.3:0/2134164055 wait complete. 2023-08-11T14:03:48.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.931+0000 7f7ac49ff700 1 Processor -- start 2023-08-11T14:03:48.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.931+0000 7f7ac49ff700 1 -- start start 2023-08-11T14:03:48.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.931+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:48.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.931+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0101b10 0x7f7ac0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:48.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7ac49ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac01024b0 0x7f7ac019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a03a0 con 0x7f7ac0101b10 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a04e0 con 0x7f7ac0100920 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7ac49ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a0670 con 0x7f7ac01024b0 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abdd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0101b10 0x7f7ac0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:48.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43634/0 (socket says 172.21.15.3:43634) 2023-08-11T14:03:48.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 -- 172.21.15.3:0/131541505 learned_addr learned my addr 172.21.15.3:0/131541505 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:48.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abed9d700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac01024b0 0x7f7ac019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:48.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 -- 172.21.15.3:0/131541505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac01024b0 msgr2=0x7f7ac019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:48.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac01024b0 0x7f7ac019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 -- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0101b10 msgr2=0x7f7ac0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0101b10 0x7f7ac0198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.932+0000 7f7abe59c700 1 -- 172.21.15.3:0/131541505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8009c40 con 0x7f7ac0100920 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7abe59c700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab001d070 con 0x7f7ac0100920 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00095f0 con 0x7f7ac0100920 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab0005510 con 0x7f7ac0100920 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0004a00 con 0x7f7ac0100920 2023-08-11T14:03:48.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.933+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac01a0be0 con 0x7f7ac0100920 2023-08-11T14:03:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.934+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac0192f70 con 0x7f7ac0100920 2023-08-11T14:03:48.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.936+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7ab0004bc0 con 0x7f7ac0100920 2023-08-11T14:03:48.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.938+0000 7f7aaf7fe700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:48.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.938+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7ab00cc6f0 con 0x7f7ac0100920 2023-08-11T14:03:48.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.938+0000 7f7abdd9b700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:48.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.939+0000 7f7abdd9b700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 0x7f7aa80a2460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009c60 tx=0x7f7ab4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:48.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:48.940+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7ab0096190 con 0x7f7ac0100920 2023-08-11T14:03:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.153+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ac010c3d0 con 0x7f7ac0100920 2023-08-11T14:03:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.155+0000 7f7aaf7fe700 1 -- 172.21.15.3:0/131541505 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f7ab00997b0 con 0x7f7ac0100920 2023-08-11T14:03:49.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:49.157 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.157+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 msgr2=0x7f7aa80a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.157+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 0x7f7aa80a2460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009c60 tx=0x7f7ab4009260 comp rx=0 tx=0).stop 2023-08-11T14:03:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.157+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 msgr2=0x7f7ac0198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.157+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009fd0 tx=0x7f7ab000f690 comp rx=0 tx=0).stop 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 shutdown_connections 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac01024b0 0x7f7ac019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7aa809ffa0 0x7f7aa80a2460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ac0100920 0x7f7ac0198820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 --2- 172.21.15.3:0/131541505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ac0101b10 0x7f7ac0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 >> 172.21.15.3:0/131541505 conn(0x7f7ac00fbe50 msgr2=0x7f7ac01073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 shutdown_connections 2023-08-11T14:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.158+0000 7f7ac49ff700 1 -- 172.21.15.3:0/131541505 wait complete. 2023-08-11T14:03:49.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:49.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:49.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 -- 172.21.15.3:0/138669085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 msgr2=0x7f1c1810aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 --2- 172.21.15.3:0/138669085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1810aae0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b0a0 tx=0x7f1c1000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 -- 172.21.15.3:0/138669085 shutdown_connections 2023-08-11T14:03:49.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 --2- 172.21.15.3:0/138669085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1810aae0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 --2- 172.21.15.3:0/138669085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 --2- 172.21.15.3:0/138669085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c18073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 -- 172.21.15.3:0/138669085 >> 172.21.15.3:0/138669085 conn(0x7f1c180fbd00 msgr2=0x7f1c180fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:49.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 -- 172.21.15.3:0/138669085 shutdown_connections 2023-08-11T14:03:49.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.391+0000 7f1c17fff700 1 -- 172.21.15.3:0/138669085 wait complete. 2023-08-11T14:03:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.392+0000 7f1c17fff700 1 Processor -- start 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.392+0000 7f1c17fff700 1 -- start start 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c181986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181a0220 con 0x7f1c18108650 2023-08-11T14:03:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181a0360 con 0x7f1c18073380 2023-08-11T14:03:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c17fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181a04f0 con 0x7f1c18074960 2023-08-11T14:03:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c181986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c16ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c181986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43656/0 (socket says 172.21.15.3:43656) 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c16ffd700 1 -- 172.21.15.3:0/3296426362 learned_addr learned my addr 172.21.15.3:0/3296426362 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c167fc700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 -- 172.21.15.3:0/3296426362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 msgr2=0x7f1c18198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 -- 172.21.15.3:0/3296426362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 msgr2=0x7f1c181986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c181986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c177fe700 1 -- 172.21.15.3:0/3296426362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10009d00 con 0x7f1c18108650 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.393+0000 7f1c167fc700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1c177fe700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1819fc50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009fd0 tx=0x7f1c10007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000e040 con 0x7f1c18108650 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c181a0770 con 0x7f1c18108650 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c10004c50 con 0x7f1c18108650 2023-08-11T14:03:49.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1001b6c0 con 0x7f1c18108650 2023-08-11T14:03:49.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.394+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c181a0bf0 con 0x7f1c18108650 2023-08-11T14:03:49.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.395+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c18107410 con 0x7f1c18108650 2023-08-11T14:03:49.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.397+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1c1001d470 con 0x7f1c18108650 2023-08-11T14:03:49.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.399+0000 7f1bfffff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 0x7f1c000a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.400+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1c100cc530 con 0x7f1c18108650 2023-08-11T14:03:49.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.400+0000 7f1c16ffd700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 0x7f1c000a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.400+0000 7f1c16ffd700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 0x7f1c000a2530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d20 tx=0x7f1c08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.404+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1c10095fd0 con 0x7f1c18108650 2023-08-11T14:03:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:49 smithi003 ceph-mon[97195]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:49.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:49 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/131541505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:49.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.615+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c1802ccb0 con 0x7f1c18108650 2023-08-11T14:03:49.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.615+0000 7f1bfffff700 1 -- 172.21.15.3:0/3296426362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c100995f0 con 0x7f1c18108650 2023-08-11T14:03:49.617 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 msgr2=0x7f1c000a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 0x7f1c000a2530 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009d20 tx=0x7f1c08009260 comp rx=0 tx=0).stop 2023-08-11T14:03:49.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 msgr2=0x7f1c1819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1819fc50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009fd0 tx=0x7f1c10007930 comp rx=0 tx=0).stop 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 shutdown_connections 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18108650 0x7f1c1819fc50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1c000a0070 0x7f1c000a2530 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c18073380 0x7f1c181986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 --2- 172.21.15.3:0/3296426362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c18074960 0x7f1c18198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 >> 172.21.15.3:0/3296426362 conn(0x7f1c180fbd00 msgr2=0x7f1c18104550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 shutdown_connections 2023-08-11T14:03:49.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.618+0000 7f1c17fff700 1 -- 172.21.15.3:0/3296426362 wait complete. 2023-08-11T14:03:49.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:49 smithi103 ceph-mon[130434]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:49 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/131541505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:49 smithi187 ceph-mon[130973]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-08-11T14:03:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:49 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/131541505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 -- 172.21.15.3:0/1146876001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4100910 msgr2=0x7f45c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 --2- 172.21.15.3:0/1146876001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4100910 0x7f45c4100d30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f45c00099c0 tx=0x7f45c0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 -- 172.21.15.3:0/1146876001 shutdown_connections 2023-08-11T14:03:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 --2- 172.21.15.3:0/1146876001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 --2- 172.21.15.3:0/1146876001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4100910 0x7f45c4100d30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 --2- 172.21.15.3:0/1146876001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4101b00 0x7f45c4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 -- 172.21.15.3:0/1146876001 >> 172.21.15.3:0/1146876001 conn(0x7f45c40fbe60 msgr2=0x7f45c40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:49.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 -- 172.21.15.3:0/1146876001 shutdown_connections 2023-08-11T14:03:49.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.844+0000 7f45cbbff700 1 -- 172.21.15.3:0/1146876001 wait complete. 2023-08-11T14:03:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.845+0000 7f45cbbff700 1 Processor -- start 2023-08-11T14:03:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.845+0000 7f45cbbff700 1 -- start start 2023-08-11T14:03:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4100910 0x7f45c41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41a0460 con 0x7f45c4101b00 2023-08-11T14:03:49.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41a05a0 con 0x7f45c4100910 2023-08-11T14:03:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45cbbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41a0730 con 0x7f45c41024a0 2023-08-11T14:03:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33418/0 (socket says 172.21.15.3:33418) 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 -- 172.21.15.3:0/3216395632 learned_addr learned my addr 172.21.15.3:0/3216395632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45ca19c700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c999b700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4100910 0x7f45c41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 -- 172.21.15.3:0/3216395632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 msgr2=0x7f45c419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 -- 172.21.15.3:0/3216395632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4100910 msgr2=0x7f45c41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4100910 0x7f45c41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45c919a700 1 -- 172.21.15.3:0/3216395632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009520 con 0x7f45c4101b00 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.846+0000 7f45ca19c700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45c919a700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45b400eb30 tx=0x7f45b400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4009880 con 0x7f45c4101b00 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c00095f0 con 0x7f45c4101b00 2023-08-11T14:03:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b40044e0 con 0x7f45c4101b00 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4003d80 con 0x7f45c4101b00 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.847+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c41a0de0 con 0x7f45c4101b00 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.848+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4192f60 con 0x7f45c4101b00 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.850+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f45b4003f20 con 0x7f45c4101b00 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.851+0000 7f45baffd700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 0x7f45b00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.851+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f45b4014070 con 0x7f45c4101b00 2023-08-11T14:03:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.852+0000 7f45c999b700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 0x7f45b00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.852+0000 7f45c999b700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 0x7f45b00a2410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009990 tx=0x7f45c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:49.855+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f45b4095500 con 0x7f45c4101b00 2023-08-11T14:03:50.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.068+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45c402ccb0 con 0x7f45c4101b00 2023-08-11T14:03:50.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.069+0000 7f45baffd700 1 -- 172.21.15.3:0/3216395632 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45b4098b20 con 0x7f45c4101b00 2023-08-11T14:03:50.071 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 msgr2=0x7f45b00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 0x7f45b00a2410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009990 tx=0x7f45c0005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 msgr2=0x7f45c4198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45b400eb30 tx=0x7f45b400ee40 comp rx=0 tx=0).stop 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 shutdown_connections 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45c41024a0 0x7f45c419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45b009ff50 0x7f45b00a2410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c4100910 0x7f45c41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 --2- 172.21.15.3:0/3216395632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45c4101b00 0x7f45c4198e20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 >> 172.21.15.3:0/3216395632 conn(0x7f45c40fbe60 msgr2=0x7f45c41074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 shutdown_connections 2023-08-11T14:03:50.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.071+0000 7f45cbbff700 1 -- 172.21.15.3:0/3216395632 wait complete. 2023-08-11T14:03:50.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:50.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 -- 172.21.15.3:0/2142755813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a201024a0 msgr2=0x7f0a20108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2142755813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a201024a0 0x7f0a20108d50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800b0d0 tx=0x7f0a1800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 -- 172.21.15.3:0/2142755813 shutdown_connections 2023-08-11T14:03:50.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2142755813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a201024a0 0x7f0a20108d50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2142755813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a20100910 0x7f0a20100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2142755813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20101b00 0x7f0a20101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 -- 172.21.15.3:0/2142755813 >> 172.21.15.3:0/2142755813 conn(0x7f0a200fbe60 msgr2=0x7f0a200fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 -- 172.21.15.3:0/2142755813 shutdown_connections 2023-08-11T14:03:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.298+0000 7f0a279ff700 1 -- 172.21.15.3:0/2142755813 wait complete. 2023-08-11T14:03:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 Processor -- start 2023-08-11T14:03:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 -- start start 2023-08-11T14:03:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20100910 0x7f0a20196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2019e2b0 con 0x7f0a20101b00 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2019e3f0 con 0x7f0a20100910 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.300+0000 7f0a279ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2019e580 con 0x7f0a201024a0 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a25f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a2579b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20100910 0x7f0a20196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33436/0 (socket says 172.21.15.3:33436) 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a25f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:53948/0 (socket says 172.21.15.3:53948) 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 -- 172.21.15.3:0/2577687771 learned_addr learned my addr 172.21.15.3:0/2577687771 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 -- 172.21.15.3:0/2577687771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 msgr2=0x7f0a2019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 -- 172.21.15.3:0/2577687771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20100910 msgr2=0x7f0a20196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20100910 0x7f0a20196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a24f9a700 1 -- 172.21.15.3:0/2577687771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18009d00 con 0x7f0a20101b00 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.301+0000 7f0a25f9c700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a24f9a700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d980 tx=0x7f0a1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10004220 con 0x7f0a20101b00 2023-08-11T14:03:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a10007640 con 0x7f0a20101b00 2023-08-11T14:03:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2019e860 con 0x7f0a20101b00 2023-08-11T14:03:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10005690 con 0x7f0a20101b00 2023-08-11T14:03:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.302+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2019edc0 con 0x7f0a20101b00 2023-08-11T14:03:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.303+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2010c180 con 0x7f0a20101b00 2023-08-11T14:03:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.305+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0a10004750 con 0x7f0a20101b00 2023-08-11T14:03:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.307+0000 7f0a167fc700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 0x7f0a0c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.307+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0a100cb330 con 0x7f0a20101b00 2023-08-11T14:03:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.307+0000 7f0a2579b700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 0x7f0a0c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.308+0000 7f0a2579b700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 0x7f0a0c0a2490 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a20101960 tx=0x7f0a1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.311+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0a10094d40 con 0x7f0a20101b00 2023-08-11T14:03:50.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.527+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a201077f0 con 0x7f0a20101b00 2023-08-11T14:03:50.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.527+0000 7f0a167fc700 1 -- 172.21.15.3:0/2577687771 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a100160a0 con 0x7f0a20101b00 2023-08-11T14:03:50.529 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 msgr2=0x7f0a0c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 0x7f0a0c0a2490 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0a20101960 tx=0x7f0a1c00b410 comp rx=0 tx=0).stop 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 msgr2=0x7f0a20196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d980 tx=0x7f0a1000dc90 comp rx=0 tx=0).stop 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 shutdown_connections 2023-08-11T14:03:50.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a201024a0 0x7f0a2019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0a0c09ffd0 0x7f0a0c0a2490 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a20100910 0x7f0a20196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 --2- 172.21.15.3:0/2577687771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a20101b00 0x7f0a20196c70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 >> 172.21.15.3:0/2577687771 conn(0x7f0a200fbe60 msgr2=0x7f0a200fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 shutdown_connections 2023-08-11T14:03:50.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.530+0000 7f0a279ff700 1 -- 172.21.15.3:0/2577687771 wait complete. 2023-08-11T14:03:50.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3296426362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:50 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3216395632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3296426362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:50 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3216395632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 -- 172.21.15.3:0/3597588223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a80101b00 msgr2=0x7f7a80101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 --2- 172.21.15.3:0/3597588223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a80101b00 0x7f7a80101f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a740098f0 tx=0x7f7a74009c00 comp rx=0 tx=0).stop 2023-08-11T14:03:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 -- 172.21.15.3:0/3597588223 shutdown_connections 2023-08-11T14:03:50.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 --2- 172.21.15.3:0/3597588223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a801024a0 0x7f7a80108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 --2- 172.21.15.3:0/3597588223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 --2- 172.21.15.3:0/3597588223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a80101b00 0x7f7a80101f60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 -- 172.21.15.3:0/3597588223 >> 172.21.15.3:0/3597588223 conn(0x7f7a800fbe60 msgr2=0x7f7a800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 -- 172.21.15.3:0/3597588223 shutdown_connections 2023-08-11T14:03:50.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.757+0000 7f7a849ff700 1 -- 172.21.15.3:0/3597588223 wait complete. 2023-08-11T14:03:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.758+0000 7f7a849ff700 1 Processor -- start 2023-08-11T14:03:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 -- start start 2023-08-11T14:03:50.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 0x7f7a80196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a801024a0 0x7f7a8019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e1c0 con 0x7f7a80100910 2023-08-11T14:03:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e300 con 0x7f7a80101b00 2023-08-11T14:03:50.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a849ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8019e490 con 0x7f7a801024a0 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a7e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33462/0 (socket says 172.21.15.3:33462) 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a7e59c700 1 -- 172.21.15.3:0/114489882 learned_addr learned my addr 172.21.15.3:0/114489882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a7dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 0x7f7a80196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.759+0000 7f7a7ed9d700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a801024a0 0x7f7a8019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 -- 172.21.15.3:0/114489882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a801024a0 msgr2=0x7f7a8019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a801024a0 0x7f7a8019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 -- 172.21.15.3:0/114489882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 msgr2=0x7f7a80196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 0x7f7a80196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 -- 172.21.15.3:0/114489882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74009520 con 0x7f7a80100910 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7dd9b700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 0x7f7a80196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a7e59c700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000e9f0 tx=0x7f7a7000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:50.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70004220 con 0x7f7a80100910 2023-08-11T14:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8019e770 con 0x7f7a80100910 2023-08-11T14:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a70004380 con 0x7f7a80100910 2023-08-11T14:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.760+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70003be0 con 0x7f7a80100910 2023-08-11T14:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.761+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8019ecd0 con 0x7f7a80100910 2023-08-11T14:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.761+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8010c120 con 0x7f7a80100910 2023-08-11T14:03:50.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.764+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7a70003d40 con 0x7f7a80100910 2023-08-11T14:03:50.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.765+0000 7f7a6f7fe700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 0x7f7a680a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:50.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.765+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7a70014070 con 0x7f7a80100910 2023-08-11T14:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.765+0000 7f7a7dd9b700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 0x7f7a680a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.766+0000 7f7a7dd9b700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 0x7f7a680a2410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400fa70 tx=0x7f7a74019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.769+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7a70095090 con 0x7f7a80100910 2023-08-11T14:03:50.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3296426362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:50 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3216395632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:50.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.978+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7a80107880 con 0x7f7a80100910 2023-08-11T14:03:50.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.978+0000 7f7a6f7fe700 1 -- 172.21.15.3:0/114489882 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7a700986b0 con 0x7f7a80100910 2023-08-11T14:03:50.980 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:50.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 msgr2=0x7f7a680a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 0x7f7a680a2410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400fa70 tx=0x7f7a74019040 comp rx=0 tx=0).stop 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 msgr2=0x7f7a80196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000e9f0 tx=0x7f7a7000ed00 comp rx=0 tx=0).stop 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 shutdown_connections 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a801024a0 0x7f7a8019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a80100910 0x7f7a80196640 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7a6809ff50 0x7f7a680a2410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 --2- 172.21.15.3:0/114489882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a80101b00 0x7f7a80196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 >> 172.21.15.3:0/114489882 conn(0x7f7a800fbe60 msgr2=0x7f7a800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 shutdown_connections 2023-08-11T14:03:50.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:50.981+0000 7f7a849ff700 1 -- 172.21.15.3:0/114489882 wait complete. 2023-08-11T14:03:51.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:51.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 -- 172.21.15.3:0/999829828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c101b10 msgr2=0x7f062c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 --2- 172.21.15.3:0/999829828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c101b10 0x7f062c101f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f06140098f0 tx=0x7f0614009c00 comp rx=0 tx=0).stop 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 -- 172.21.15.3:0/999829828 shutdown_connections 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 --2- 172.21.15.3:0/999829828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c1024b0 0x7f062c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 --2- 172.21.15.3:0/999829828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 --2- 172.21.15.3:0/999829828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c101b10 0x7f062c101f70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 -- 172.21.15.3:0/999829828 >> 172.21.15.3:0/999829828 conn(0x7f062c0fbe50 msgr2=0x7f062c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 -- 172.21.15.3:0/999829828 shutdown_connections 2023-08-11T14:03:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.207+0000 7f0630dff700 1 -- 172.21.15.3:0/999829828 wait complete. 2023-08-11T14:03:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.208+0000 7f0630dff700 1 Processor -- start 2023-08-11T14:03:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 -- start start 2023-08-11T14:03:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 0x7f062c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c1024b0 0x7f062c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0739f0 con 0x7f062c100920 2023-08-11T14:03:51.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c073b60 con 0x7f062c101b10 2023-08-11T14:03:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0630dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1a0620 con 0x7f062c1024b0 2023-08-11T14:03:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f062a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f062a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33476/0 (socket says 172.21.15.3:33476) 2023-08-11T14:03:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f062a59c700 1 -- 172.21.15.3:0/1080775842 learned_addr learned my addr 172.21.15.3:0/1080775842 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:51.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f0623fff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 0x7f062c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.209+0000 7f062ad9d700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c1024b0 0x7f062c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 -- 172.21.15.3:0/1080775842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c1024b0 msgr2=0x7f062c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c1024b0 0x7f062c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 -- 172.21.15.3:0/1080775842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 msgr2=0x7f062c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 0x7f062c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 -- 172.21.15.3:0/1080775842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009520 con 0x7f062c100920 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f0623fff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 0x7f062c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f062a59c700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e9f0 tx=0x7f061c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c004220 con 0x7f062c100920 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c1a0900 con 0x7f062c100920 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061c004380 con 0x7f062c100920 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c003bb0 con 0x7f062c100920 2023-08-11T14:03:51.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.210+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c1a0e60 con 0x7f062c100920 2023-08-11T14:03:51.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.211+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c107ec0 con 0x7f062c100920 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.213+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f061c00c7c0 con 0x7f062c100920 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.215+0000 7f06237fe700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 0x7f06180a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.215+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f061c014070 con 0x7f062c100920 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.215+0000 7f0623fff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 0x7f06180a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.216+0000 7f0623fff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 0x7f06180a2460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f061400fa70 tx=0x7f0614019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:51.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.218+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f061c095390 con 0x7f062c100920 2023-08-11T14:03:51.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.436+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f062c02cda0 con 0x7f062c100920 2023-08-11T14:03:51.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.438+0000 7f06237fe700 1 -- 172.21.15.3:0/1080775842 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f061c0989b0 con 0x7f062c100920 2023-08-11T14:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:51.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.440+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 msgr2=0x7f06180a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.440+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 0x7f06180a2460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f061400fa70 tx=0x7f0614019040 comp rx=0 tx=0).stop 2023-08-11T14:03:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.440+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 msgr2=0x7f062c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.440+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f061c00e9f0 tx=0x7f061c00ed00 comp rx=0 tx=0).stop 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 shutdown_connections 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062c1024b0 0x7f062c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f062c100920 0x7f062c0746d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f061809ffa0 0x7f06180a2460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 --2- 172.21.15.3:0/1080775842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f062c101b10 0x7f062c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 >> 172.21.15.3:0/1080775842 conn(0x7f062c0fbe50 msgr2=0x7f062c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:51.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 shutdown_connections 2023-08-11T14:03:51.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.441+0000 7f0630dff700 1 -- 172.21.15.3:0/1080775842 wait complete. 2023-08-11T14:03:51.461 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:51.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 -- 172.21.15.3:0/1984836945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3181024a0 msgr2=0x7fd318108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/1984836945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3181024a0 0x7fd318108d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd31000b0d0 tx=0x7fd31000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 -- 172.21.15.3:0/1984836945 shutdown_connections 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/1984836945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3181024a0 0x7fd318108d50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/1984836945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 0x7fd318100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/1984836945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd318101b00 0x7fd318101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 -- 172.21.15.3:0/1984836945 >> 172.21.15.3:0/1984836945 conn(0x7fd3180fbe60 msgr2=0x7fd3180fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 -- 172.21.15.3:0/1984836945 shutdown_connections 2023-08-11T14:03:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.668+0000 7fd31e7ff700 1 -- 172.21.15.3:0/1984836945 wait complete. 2023-08-11T14:03:51.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.669+0000 7fd31e7ff700 1 Processor -- start 2023-08-11T14:03:51.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.669+0000 7fd31e7ff700 1 -- start start 2023-08-11T14:03:51.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.669+0000 7fd31e7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 0x7fd3181966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31e7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31e7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3181024a0 0x7fd31819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31e7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31819e260 con 0x7fd318101b00 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31e7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31819e3a0 con 0x7fd318100910 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31e7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31819e530 con 0x7fd3181024a0 2023-08-11T14:03:51.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59536/0 (socket says 172.21.15.3:59536) 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 -- 172.21.15.3:0/369128830 learned_addr learned my addr 172.21.15.3:0/369128830 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd317fff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 0x7fd3181966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd31cd9c700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3181024a0 0x7fd31819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 -- 172.21.15.3:0/369128830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3181024a0 msgr2=0x7fd31819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3181024a0 0x7fd31819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 -- 172.21.15.3:0/369128830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 msgr2=0x7fd3181966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 0x7fd3181966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.670+0000 7fd3177fe700 1 -- 172.21.15.3:0/369128830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd310009d00 con 0x7fd318101b00 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd3177fe700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e9a0 tx=0x7fd30c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c009790 con 0x7fd318101b00 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31819e810 con 0x7fd318101b00 2023-08-11T14:03:51.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30c010430 con 0x7fd318101b00 2023-08-11T14:03:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c018680 con 0x7fd318101b00 2023-08-11T14:03:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.671+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31819ed70 con 0x7fd318101b00 2023-08-11T14:03:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.672+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31810c130 con 0x7fd318101b00 2023-08-11T14:03:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.674+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd30c0105a0 con 0x7fd318101b00 2023-08-11T14:03:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.675+0000 7fd3157fa700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 0x7fd3000a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.676+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd30c014070 con 0x7fd318101b00 2023-08-11T14:03:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.676+0000 7fd317fff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 0x7fd3000a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:51.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.676+0000 7fd317fff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 0x7fd3000a2490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd318101960 tx=0x7fd308009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:51.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:51 smithi103 ceph-mon[130434]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2023-08-11T14:03:51.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2577687771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:51 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/114489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.679+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd30c051c30 con 0x7fd318101b00 2023-08-11T14:03:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:51 smithi003 ceph-mon[97195]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2023-08-11T14:03:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2577687771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:51 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/114489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:51 smithi187 ceph-mon[130973]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2023-08-11T14:03:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2577687771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:51 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/114489882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:51.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.892+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3181077f0 con 0x7fd318101b00 2023-08-11T14:03:51.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.892+0000 7fd3157fa700 1 -- 172.21.15.3:0/369128830 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd30c095100 con 0x7fd318101b00 2023-08-11T14:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.894+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 msgr2=0x7fd3000a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.894+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 0x7fd3000a2490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd318101960 tx=0x7fd308009260 comp rx=0 tx=0).stop 2023-08-11T14:03:51.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.894+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 msgr2=0x7fd318196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.894+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00e9a0 tx=0x7fd30c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 shutdown_connections 2023-08-11T14:03:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3181024a0 0x7fd31819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd30009ffd0 0x7fd3000a2490 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd318100910 0x7fd3181966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 --2- 172.21.15.3:0/369128830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd318101b00 0x7fd318196c20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 >> 172.21.15.3:0/369128830 conn(0x7fd3180fbe60 msgr2=0x7fd3180fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 shutdown_connections 2023-08-11T14:03:51.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:51.895+0000 7fd31e7ff700 1 -- 172.21.15.3:0/369128830 wait complete. 2023-08-11T14:03:51.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:52.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 -- 172.21.15.3:0/2825068499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144101b00 msgr2=0x7f5144101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 --2- 172.21.15.3:0/2825068499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144101b00 0x7f5144101f60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f51300099c0 tx=0x7f5130009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 -- 172.21.15.3:0/2825068499 shutdown_connections 2023-08-11T14:03:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 --2- 172.21.15.3:0/2825068499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51441024a0 0x7f5144108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 --2- 172.21.15.3:0/2825068499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5144100910 0x7f5144100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 --2- 172.21.15.3:0/2825068499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144101b00 0x7f5144101f60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 -- 172.21.15.3:0/2825068499 >> 172.21.15.3:0/2825068499 conn(0x7f51440fbe60 msgr2=0x7f51440fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 -- 172.21.15.3:0/2825068499 shutdown_connections 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.122+0000 7f514adff700 1 -- 172.21.15.3:0/2825068499 wait complete. 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 Processor -- start 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 -- start start 2023-08-11T14:03:52.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 0x7f5144198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51441024a0 0x7f514419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a0400 con 0x7f5144100910 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a0540 con 0x7f51441024a0 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.123+0000 7f514adff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51441a06d0 con 0x7f5144101b00 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5148b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 0x7f5144198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60544/0 (socket says 172.21.15.3:60544) 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 -- 172.21.15.3:0/1785545465 learned_addr learned my addr 172.21.15.3:0/1785545465 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f514939c700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51441024a0 0x7f514419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 -- 172.21.15.3:0/1785545465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51441024a0 msgr2=0x7f514419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51441024a0 0x7f514419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 -- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 msgr2=0x7f5144198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 0x7f5144198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 -- 172.21.15.3:0/1785545465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300095f0 con 0x7f5144101b00 2023-08-11T14:03:52.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5148b9b700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 0x7f5144198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5143fff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f513000bf40 tx=0x7f513000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.124+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513001d070 con 0x7f5144101b00 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.125+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441a0950 con 0x7f5144101b00 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.125+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5130005050 con 0x7f5144101b00 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.125+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130004b40 con 0x7f5144101b00 2023-08-11T14:03:52.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.125+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441a0d80 con 0x7f5144101b00 2023-08-11T14:03:52.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.126+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514410c170 con 0x7f5144101b00 2023-08-11T14:03:52.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.128+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5130004cc0 con 0x7f5144101b00 2023-08-11T14:03:52.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.129+0000 7f5141ffb700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 0x7f51340a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.129+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f51300cc840 con 0x7f5144101b00 2023-08-11T14:03:52.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.129+0000 7f5148b9b700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 0x7f51340a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.130+0000 7f5148b9b700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 0x7f51340a2410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5144101960 tx=0x7f5138009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:52.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.131+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f51300962e0 con 0x7f5144101b00 2023-08-11T14:03:52.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.346+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51441a0fa0 con 0x7f5144101b00 2023-08-11T14:03:52.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.347+0000 7f5141ffb700 1 -- 172.21.15.3:0/1785545465 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5130099900 con 0x7f5144101b00 2023-08-11T14:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:52.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 msgr2=0x7f51340a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 0x7f51340a2410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5144101960 tx=0x7f5138009260 comp rx=0 tx=0).stop 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 msgr2=0x7f5144198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f513000bf40 tx=0x7f513000f690 comp rx=0 tx=0).stop 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 shutdown_connections 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f513409ff50 0x7f51340a2410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51441024a0 0x7f514419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5144100910 0x7f5144198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 --2- 172.21.15.3:0/1785545465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5144101b00 0x7f5144198dc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 >> 172.21.15.3:0/1785545465 conn(0x7f51440fbe60 msgr2=0x7f51440fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 shutdown_connections 2023-08-11T14:03:52.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.350+0000 7f514adff700 1 -- 172.21.15.3:0/1785545465 wait complete. 2023-08-11T14:03:52.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 -- 172.21.15.3:0/289077197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4100910 msgr2=0x7f08c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 --2- 172.21.15.3:0/289077197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4100910 0x7f08c4100d30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08b40099c0 tx=0x7f08b4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 -- 172.21.15.3:0/289077197 shutdown_connections 2023-08-11T14:03:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 --2- 172.21.15.3:0/289077197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c41024a0 0x7f08c4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 --2- 172.21.15.3:0/289077197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4100910 0x7f08c4100d30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 --2- 172.21.15.3:0/289077197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c4101b00 0x7f08c4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 -- 172.21.15.3:0/289077197 >> 172.21.15.3:0/289077197 conn(0x7f08c40fbe60 msgr2=0x7f08c40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 -- 172.21.15.3:0/289077197 shutdown_connections 2023-08-11T14:03:52.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.577+0000 7f08c99ff700 1 -- 172.21.15.3:0/289077197 wait complete. 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.578+0000 7f08c99ff700 1 Processor -- start 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.578+0000 7f08c99ff700 1 -- start start 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4101b00 0x7f08c4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c41024a0 0x7f08c419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c41a03c0 con 0x7f08c4101b00 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c41a0500 con 0x7f08c41024a0 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c99ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c41a0690 con 0x7f08c4100910 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60572/0 (socket says 172.21.15.3:60572) 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2672204668 learned_addr learned my addr 172.21.15.3:0/2672204668 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c27fc700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4101b00 0x7f08c4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c37fe700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c41024a0 0x7f08c419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2672204668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c41024a0 msgr2=0x7f08c419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c41024a0 0x7f08c419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4101b00 msgr2=0x7f08c4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4101b00 0x7f08c4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.579+0000 7f08c2ffd700 1 -- 172.21.15.3:0/2672204668 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009520 con 0x7f08c4100910 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c2ffd700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009fd0 tx=0x7f08b400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:52.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b401d070 con 0x7f08c4100910 2023-08-11T14:03:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b4005510 con 0x7f08c4100910 2023-08-11T14:03:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b40095f0 con 0x7f08c4100910 2023-08-11T14:03:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4004b00 con 0x7f08c4100910 2023-08-11T14:03:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.580+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c41a0ba0 con 0x7f08c4100910 2023-08-11T14:03:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.581+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c4192f60 con 0x7f08c4100910 2023-08-11T14:03:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.583+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f08b400fb80 con 0x7f08c4100910 2023-08-11T14:03:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.585+0000 7f08c89fd700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 0x7f08ac0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.585+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f08b40cc820 con 0x7f08c4100910 2023-08-11T14:03:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.585+0000 7f08c27fc700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 0x7f08ac0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.586+0000 7f08c27fc700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 0x7f08ac0a2410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009df0 tx=0x7f08b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.587+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f08b40962c0 con 0x7f08c4100910 2023-08-11T14:03:52.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1080775842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:52.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:52 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/369128830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:52 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1080775842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:52 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/369128830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:52 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1080775842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:52.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:52 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/369128830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:52.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.801+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08c410c3c0 con 0x7f08c4100910 2023-08-11T14:03:52.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.802+0000 7f08c89fd700 1 -- 172.21.15.3:0/2672204668 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08b40998e0 con 0x7f08c4100910 2023-08-11T14:03:52.804 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.804+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 msgr2=0x7f08ac0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.804+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 0x7f08ac0a2410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009df0 tx=0x7f08b8009260 comp rx=0 tx=0).stop 2023-08-11T14:03:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.804+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 msgr2=0x7f08c4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:52.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.804+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009fd0 tx=0x7f08b400f690 comp rx=0 tx=0).stop 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 shutdown_connections 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f08ac09ff50 0x7f08ac0a2410 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08c41024a0 0x7f08c419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08c4100910 0x7f08c4198840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 --2- 172.21.15.3:0/2672204668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08c4101b00 0x7f08c4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 >> 172.21.15.3:0/2672204668 conn(0x7f08c40fbe60 msgr2=0x7f08c4107430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:52.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 shutdown_connections 2023-08-11T14:03:52.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:52.805+0000 7f08c99ff700 1 -- 172.21.15.3:0/2672204668 wait complete. 2023-08-11T14:03:52.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:53.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1674628397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b180ff180 msgr2=0x7f4b180ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1674628397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b180ff180 0x7f4b180ff5a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009990 tx=0x7f4b08009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1674628397 shutdown_connections 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1674628397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b181004a0 0x7f4b18100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1674628397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b180ff180 0x7f4b180ff5a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1674628397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ffae0 0x7f4b180fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1674628397 >> 172.21.15.3:0/1674628397 conn(0x7f4b180fad80 msgr2=0x7f4b180fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1674628397 shutdown_connections 2023-08-11T14:03:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.032+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1674628397 wait complete. 2023-08-11T14:03:53.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.033+0000 7f4b1e3ff700 1 Processor -- start 2023-08-11T14:03:53.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.033+0000 7f4b1e3ff700 1 -- start start 2023-08-11T14:03:53.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b180ffae0 0x7f4b18198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b181004a0 0x7f4b1819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181a03a0 con 0x7f4b181004a0 2023-08-11T14:03:53.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181a04e0 con 0x7f4b180ffae0 2023-08-11T14:03:53.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1e3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b181a0670 con 0x7f4b180ff180 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b1c99c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b181004a0 0x7f4b1819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60588/0 (socket says 172.21.15.3:60588) 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 -- 172.21.15.3:0/1442441392 learned_addr learned my addr 172.21.15.3:0/1442441392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b177fe700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b180ffae0 0x7f4b18198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 -- 172.21.15.3:0/1442441392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b180ffae0 msgr2=0x7f4b18198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b180ffae0 0x7f4b18198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 -- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b181004a0 msgr2=0x7f4b1819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b181004a0 0x7f4b1819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.034+0000 7f4b17fff700 1 -- 172.21.15.3:0/1442441392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080095f0 con 0x7f4b180ff180 2023-08-11T14:03:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b17fff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b680 tx=0x7f4b080057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0801d070 con 0x7f4b180ff180 2023-08-11T14:03:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b181a08f0 con 0x7f4b180ff180 2023-08-11T14:03:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b08022470 con 0x7f4b180ff180 2023-08-11T14:03:53.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08017650 con 0x7f4b180ff180 2023-08-11T14:03:53.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.035+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b181a0d70 con 0x7f4b180ff180 2023-08-11T14:03:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.036+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18109f60 con 0x7f4b180ff180 2023-08-11T14:03:53.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.038+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4b080225e0 con 0x7f4b180ff180 2023-08-11T14:03:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.040+0000 7f4b157fa700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 0x7f4b000a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.040+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4b080cd1c0 con 0x7f4b180ff180 2023-08-11T14:03:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.040+0000 7f4b177fe700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 0x7f4b000a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.041+0000 7f4b177fe700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 0x7f4b000a2410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009c60 tx=0x7f4b0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:53.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.042+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4b08096c60 con 0x7f4b180ff180 2023-08-11T14:03:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.260+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4b1810e340 con 0x7f4b180ff180 2023-08-11T14:03:53.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.261+0000 7f4b157fa700 1 -- 172.21.15.3:0/1442441392 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4b08085070 con 0x7f4b180ff180 2023-08-11T14:03:53.262 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 msgr2=0x7f4b000a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 0x7f4b000a2410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009c60 tx=0x7f4b0c009260 comp rx=0 tx=0).stop 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 msgr2=0x7f4b18198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800b680 tx=0x7f4b080057b0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 shutdown_connections 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b181004a0 0x7f4b1819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b180ff180 0x7f4b18198820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4b0009ff50 0x7f4b000a2410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 --2- 172.21.15.3:0/1442441392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b180ffae0 0x7f4b18198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 >> 172.21.15.3:0/1442441392 conn(0x7f4b180fad80 msgr2=0x7f4b180fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 shutdown_connections 2023-08-11T14:03:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.263+0000 7f4b1e3ff700 1 -- 172.21.15.3:0/1442441392 wait complete. 2023-08-11T14:03:53.286 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-08-11T14:03:53.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 -- 172.21.15.3:0/2346293706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 msgr2=0x7f104c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 --2- 172.21.15.3:0/2346293706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c100d50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f103c0099c0 tx=0x7f103c009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 -- 172.21.15.3:0/2346293706 shutdown_connections 2023-08-11T14:03:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 --2- 172.21.15.3:0/2346293706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c1024c0 0x7f104c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 --2- 172.21.15.3:0/2346293706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c100d50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 --2- 172.21.15.3:0/2346293706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c101b20 0x7f104c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 -- 172.21.15.3:0/2346293706 >> 172.21.15.3:0/2346293706 conn(0x7f104c0fbe60 msgr2=0x7f104c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:53.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 -- 172.21.15.3:0/2346293706 shutdown_connections 2023-08-11T14:03:53.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.500+0000 7f10521ff700 1 -- 172.21.15.3:0/2346293706 wait complete. 2023-08-11T14:03:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.501+0000 7f10521ff700 1 Processor -- start 2023-08-11T14:03:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 -- start start 2023-08-11T14:03:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 0x7f104c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c1024c0 0x7f104c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1a0310 con 0x7f104c100930 2023-08-11T14:03:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1a0450 con 0x7f104c101b20 2023-08-11T14:03:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f10521ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1a05e0 con 0x7f104c1024c0 2023-08-11T14:03:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f104b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f104b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59608/0 (socket says 172.21.15.3:59608) 2023-08-11T14:03:53.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f104b7fe700 1 -- 172.21.15.3:0/3000187800 learned_addr learned my addr 172.21.15.3:0/3000187800 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f104affd700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 0x7f104c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.502+0000 7f104bfff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c1024c0 0x7f104c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 -- 172.21.15.3:0/3000187800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c1024c0 msgr2=0x7f104c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c1024c0 0x7f104c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 -- 172.21.15.3:0/3000187800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 msgr2=0x7f104c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 0x7f104c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 -- 172.21.15.3:0/3000187800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0095f0 con 0x7f104c100930 2023-08-11T14:03:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104affd700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 0x7f104c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f104b7fe700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f103c009fd0 tx=0x7f103c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c01d070 con 0x7f104c100930 2023-08-11T14:03:53.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c1a0860 con 0x7f104c100930 2023-08-11T14:03:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f103c0054f0 con 0x7f104c100930 2023-08-11T14:03:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c004c40 con 0x7f104c100930 2023-08-11T14:03:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.503+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c1a0ce0 con 0x7f104c100930 2023-08-11T14:03:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.504+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c10c140 con 0x7f104c100930 2023-08-11T14:03:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.506+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f103c00fc30 con 0x7f104c100930 2023-08-11T14:03:53.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.509+0000 7f1048ff9700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 0x7f10340a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.509+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f103c0cc4a0 con 0x7f104c100930 2023-08-11T14:03:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.509+0000 7f104affd700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 0x7f10340a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.509+0000 7f104affd700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 0x7f10340a2460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1040009cb0 tx=0x7f1040009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.513+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f103c095f40 con 0x7f104c100930 2023-08-11T14:03:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:53 smithi003 ceph-mon[97195]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-08-11T14:03:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1785545465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2672204668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:53 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:03:53.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1442441392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:53 smithi103 ceph-mon[130434]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-08-11T14:03:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1785545465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2672204668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:53 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:03:53.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1442441392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:53 smithi187 ceph-mon[130973]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-08-11T14:03:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1785545465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2672204668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:53 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:03:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1442441392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:53.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.821+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104c10c330 con 0x7f104c100930 2023-08-11T14:03:53.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.822+0000 7f1048ff9700 1 -- 172.21.15.3:0/3000187800 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f103c099560 con 0x7f104c100930 2023-08-11T14:03:53.823 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:53.824 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 msgr2=0x7f10340a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 0x7f10340a2460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1040009cb0 tx=0x7f1040009260 comp rx=0 tx=0).stop 2023-08-11T14:03:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 msgr2=0x7f104c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f103c009fd0 tx=0x7f103c00f740 comp rx=0 tx=0).stop 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 shutdown_connections 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f104c1024c0 0x7f104c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f104c100930 0x7f104c198790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f103409ffa0 0x7f10340a2460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 --2- 172.21.15.3:0/3000187800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104c101b20 0x7f104c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:53.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 >> 172.21.15.3:0/3000187800 conn(0x7f104c0fbe60 msgr2=0x7f104c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 shutdown_connections 2023-08-11T14:03:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:53.825+0000 7f10521ff700 1 -- 172.21.15.3:0/3000187800 wait complete. 2023-08-11T14:03:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:53.845 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-08-11T14:03:53.846 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-08-11T14:03:53.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:03:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 -- 172.21.15.3:0/2169321179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 msgr2=0x7f6618073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 --2- 172.21.15.3:0/2169321179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618073800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f660c009990 tx=0x7f660c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 -- 172.21.15.3:0/2169321179 shutdown_connections 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 --2- 172.21.15.3:0/2169321179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 0x7f661810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 --2- 172.21.15.3:0/2169321179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 --2- 172.21.15.3:0/2169321179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618073800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 -- 172.21.15.3:0/2169321179 >> 172.21.15.3:0/2169321179 conn(0x7f66180fbe60 msgr2=0x7f66180fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 -- 172.21.15.3:0/2169321179 shutdown_connections 2023-08-11T14:03:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.056+0000 7f661cbff700 1 -- 172.21.15.3:0/2169321179 wait complete. 2023-08-11T14:03:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.057+0000 7f661cbff700 1 Processor -- start 2023-08-11T14:03:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.057+0000 7f661cbff700 1 -- start start 2023-08-11T14:03:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 0x7f661819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66181a0310 con 0x7f6618074960 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66181a0450 con 0x7f66181087f0 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66181a05e0 con 0x7f6618073380 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f661659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59628/0 (socket says 172.21.15.3:59628) 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 -- 172.21.15.3:0/2439516480 learned_addr learned my addr 172.21.15.3:0/2439516480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6616d9d700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 0x7f661819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 -- 172.21.15.3:0/2439516480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 msgr2=0x7f6618198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 -- 172.21.15.3:0/2439516480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 msgr2=0x7f661819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 0x7f661819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.058+0000 7f6615d9b700 1 -- 172.21.15.3:0/2439516480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f6618074960 2023-08-11T14:03:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f6615d9b700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01d070 con 0x7f6618074960 2023-08-11T14:03:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66181a0860 con 0x7f6618074960 2023-08-11T14:03:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c022470 con 0x7f6618074960 2023-08-11T14:03:54.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c017650 con 0x7f6618074960 2023-08-11T14:03:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.059+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66181a0ce0 con 0x7f6618074960 2023-08-11T14:03:54.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.060+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618106ea0 con 0x7f6618074960 2023-08-11T14:03:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.062+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f660c0225e0 con 0x7f6618074960 2023-08-11T14:03:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.063+0000 7f66077fe700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 0x7f66000a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.064+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f660c0cc850 con 0x7f6618074960 2023-08-11T14:03:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.064+0000 7f661659c700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 0x7f66000a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.064+0000 7f661659c700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 0x7f66000a2460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6608009c60 tx=0x7f6608009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:54.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.067+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f660c05cc30 con 0x7f6618074960 2023-08-11T14:03:54.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.278+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f661802cd60 con 0x7f660009ffa0 2023-08-11T14:03:54.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.300+0000 7f66077fe700 1 -- 172.21.15.3:0/2439516480 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427086 (secure 0 0 0) 0x7f661802cd60 con 0x7f660009ffa0 2023-08-11T14:03:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:54.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 msgr2=0x7f66000a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 0x7f66000a2460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6608009c60 tx=0x7f6608009260 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 msgr2=0x7f6618198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f660c000c00 tx=0x7f660c005720 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 shutdown_connections 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f660009ffa0 0x7f66000a2460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66181087f0 0x7f661819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618073380 0x7f6618198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 --2- 172.21.15.3:0/2439516480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6618074960 0x7f6618198cd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 >> 172.21.15.3:0/2439516480 conn(0x7f66180fbe60 msgr2=0x7f66180fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:54.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 shutdown_connections 2023-08-11T14:03:54.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.305+0000 7f661cbff700 1 -- 172.21.15.3:0/2439516480 wait complete. 2023-08-11T14:03:54.312 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:03:54.327 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2023-08-11T14:03:54.201169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663952,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1281,"num_read_kb":773,"num_write":433,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338564,"kb_used_data":14332,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603260,"statfs":{"total":1151940427776,"available":1151593738240,"internally_reserved":0,"allocated":14675968,"data_stored":8064089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":82,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343170+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:03:04.343170+0000","last_peered":"2023-08-11T14:03:04.343170+0000","last_clean":"2023-08-11T14:03:04.343170+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:03:04.343170+0000","last_undegraded":"2023-08-11T14:03:04.343170+0000","last_fullsized":"2023-08-11T14:03:04.343170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:30.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523051+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:03:04.523051+0000","last_peered":"2023-08-11T14:03:04.523051+0000","last_clean":"2023-08-11T14:03:04.523051+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:03:04.523051+0000","last_undegraded":"2023-08-11T14:03:04.523051+0000","last_fullsized":"2023-08-11T14:03:04.523051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:59.858967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522981+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:03:04.522981+0000","last_peered":"2023-08-11T14:03:04.522981+0000","last_clean":"2023-08-11T14:03:04.522981+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:03:04.522981+0000","last_undegraded":"2023-08-11T14:03:04.522981+0000","last_fullsized":"2023-08-11T14:03:04.522981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:37.543189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346016+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:03:04.346016+0000","last_peered":"2023-08-11T14:03:04.346016+0000","last_clean":"2023-08-11T14:03:04.346016+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:03:04.346016+0000","last_undegraded":"2023-08-11T14:03:04.346016+0000","last_fullsized":"2023-08-11T14:03:04.346016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:42.564980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345098+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:03:04.345098+0000","last_peered":"2023-08-11T14:03:04.345098+0000","last_clean":"2023-08-11T14:03:04.345098+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:03:04.345098+0000","last_undegraded":"2023-08-11T14:03:04.345098+0000","last_fullsized":"2023-08-11T14:03:04.345098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:03.679224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419950+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:03:04.419950+0000","last_peered":"2023-08-11T14:03:04.419950+0000","last_clean":"2023-08-11T14:03:04.419950+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:03:04.419950+0000","last_undegraded":"2023-08-11T14:03:04.419950+0000","last_fullsized":"2023-08-11T14:03:04.419950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:51:16.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426874+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:03:04.426874+0000","last_peered":"2023-08-11T14:03:04.426874+0000","last_clean":"2023-08-11T14:03:04.426874+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:03:04.426874+0000","last_undegraded":"2023-08-11T14:03:04.426874+0000","last_fullsized":"2023-08-11T14:03:04.426874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:25.534139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340357+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:03:04.340357+0000","last_peered":"2023-08-11T14:03:04.340357+0000","last_clean":"2023-08-11T14:03:04.340357+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:03:04.340357+0000","last_undegraded":"2023-08-11T14:03:04.340357+0000","last_fullsized":"2023-08-11T14:03:04.340357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:32.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523704+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:03:04.523704+0000","last_peered":"2023-08-11T14:03:04.523704+0000","last_clean":"2023-08-11T14:03:04.523704+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:03:04.523704+0000","last_undegraded":"2023-08-11T14:03:04.523704+0000","last_fullsized":"2023-08-11T14:03:04.523704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:36:08.283056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420429+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:03:04.420429+0000","last_peered":"2023-08-11T14:03:04.420429+0000","last_clean":"2023-08-11T14:03:04.420429+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:03:04.420429+0000","last_undegraded":"2023-08-11T14:03:04.420429+0000","last_fullsized":"2023-08-11T14:03:04.420429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:36:40.405178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427584+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:03:04.427584+0000","last_peered":"2023-08-11T14:03:04.427584+0000","last_clean":"2023-08-11T14:03:04.427584+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:03:04.427584+0000","last_undegraded":"2023-08-11T14:03:04.427584+0000","last_fullsized":"2023-08-11T14:03:04.427584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:57:34.234658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345884+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:03:04.345884+0000","last_peered":"2023-08-11T14:03:04.345884+0000","last_clean":"2023-08-11T14:03:04.345884+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:03:04.345884+0000","last_undegraded":"2023-08-11T14:03:04.345884+0000","last_fullsized":"2023-08-11T14:03:04.345884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:02:05.560872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340734+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:03:04.340734+0000","last_peered":"2023-08-11T14:03:04.340734+0000","last_clean":"2023-08-11T14:03:04.340734+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:03:04.340734+0000","last_undegraded":"2023-08-11T14:03:04.340734+0000","last_fullsized":"2023-08-11T14:03:04.340734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:14.969554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522728+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:03:04.522728+0000","last_peered":"2023-08-11T14:03:04.522728+0000","last_clean":"2023-08-11T14:03:04.522728+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:03:04.522728+0000","last_undegraded":"2023-08-11T14:03:04.522728+0000","last_fullsized":"2023-08-11T14:03:04.522728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:14:20.923343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522719+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:03:04.522719+0000","last_peered":"2023-08-11T14:03:04.522719+0000","last_clean":"2023-08-11T14:03:04.522719+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:03:04.522719+0000","last_undegraded":"2023-08-11T14:03:04.522719+0000","last_fullsized":"2023-08-11T14:03:04.522719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:47.730825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523337+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:03:04.523337+0000","last_peered":"2023-08-11T14:03:04.523337+0000","last_clean":"2023-08-11T14:03:04.523337+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:03:04.523337+0000","last_undegraded":"2023-08-11T14:03:04.523337+0000","last_fullsized":"2023-08-11T14:03:04.523337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:16.271940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343273+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:03:04.343273+0000","last_peered":"2023-08-11T14:03:04.343273+0000","last_clean":"2023-08-11T14:03:04.343273+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:03:04.343273+0000","last_undegraded":"2023-08-11T14:03:04.343273+0000","last_fullsized":"2023-08-11T14:03:04.343273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:44:22.475322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341624+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:03:04.341624+0000","last_peered":"2023-08-11T14:03:04.341624+0000","last_clean":"2023-08-11T14:03:04.341624+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:03:04.341624+0000","last_undegraded":"2023-08-11T14:03:04.341624+0000","last_fullsized":"2023-08-11T14:03:04.341624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:44:26.149153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341477+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:03:04.341477+0000","last_peered":"2023-08-11T14:03:04.341477+0000","last_clean":"2023-08-11T14:03:04.341477+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:03:04.341477+0000","last_undegraded":"2023-08-11T14:03:04.341477+0000","last_fullsized":"2023-08-11T14:03:04.341477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:32.692687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340660+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:03:04.340660+0000","last_peered":"2023-08-11T14:03:04.340660+0000","last_clean":"2023-08-11T14:03:04.340660+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:03:04.340660+0000","last_undegraded":"2023-08-11T14:03:04.340660+0000","last_fullsized":"2023-08-11T14:03:04.340660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:35.403525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.490717+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:03:51.490717+0000","last_peered":"2023-08-11T14:03:51.490717+0000","last_clean":"2023-08-11T14:03:51.490717+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:03:51.490717+0000","last_undegraded":"2023-08-11T14:03:51.490717+0000","last_fullsized":"2023-08-11T14:03:51.490717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:30.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429485+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:03:04.429485+0000","last_peered":"2023-08-11T14:03:04.429485+0000","last_clean":"2023-08-11T14:03:04.429485+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:03:04.429485+0000","last_undegraded":"2023-08-11T14:03:04.429485+0000","last_fullsized":"2023-08-11T14:03:04.429485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:08.247797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345171+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:03:04.345171+0000","last_peered":"2023-08-11T14:03:04.345171+0000","last_clean":"2023-08-11T14:03:04.345171+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:03:04.345171+0000","last_undegraded":"2023-08-11T14:03:04.345171+0000","last_fullsized":"2023-08-11T14:03:04.345171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:28:23.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429535+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:03:04.429535+0000","last_peered":"2023-08-11T14:03:04.429535+0000","last_clean":"2023-08-11T14:03:04.429535+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:03:04.429535+0000","last_undegraded":"2023-08-11T14:03:04.429535+0000","last_fullsized":"2023-08-11T14:03:04.429535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:02.749730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523055+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:03:04.523055+0000","last_peered":"2023-08-11T14:03:04.523055+0000","last_clean":"2023-08-11T14:03:04.523055+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:03:04.523055+0000","last_undegraded":"2023-08-11T14:03:04.523055+0000","last_fullsized":"2023-08-11T14:03:04.523055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:21.968021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345306+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:03:04.345306+0000","last_peered":"2023-08-11T14:03:04.345306+0000","last_clean":"2023-08-11T14:03:04.345306+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:03:04.345306+0000","last_undegraded":"2023-08-11T14:03:04.345306+0000","last_fullsized":"2023-08-11T14:03:04.345306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:27:36.143460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345867+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:03:04.345867+0000","last_peered":"2023-08-11T14:03:04.345867+0000","last_clean":"2023-08-11T14:03:04.345867+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:03:04.345867+0000","last_undegraded":"2023-08-11T14:03:04.345867+0000","last_fullsized":"2023-08-11T14:03:04.345867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:28.879262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523408+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:03:04.523408+0000","last_peered":"2023-08-11T14:03:04.523408+0000","last_clean":"2023-08-11T14:03:04.523408+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:03:04.523408+0000","last_undegraded":"2023-08-11T14:03:04.523408+0000","last_fullsized":"2023-08-11T14:03:04.523408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:56.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345326+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:03:04.345326+0000","last_peered":"2023-08-11T14:03:04.345326+0000","last_clean":"2023-08-11T14:03:04.345326+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:03:04.345326+0000","last_undegraded":"2023-08-11T14:03:04.345326+0000","last_fullsized":"2023-08-11T14:03:04.345326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:11.483871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345233+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:03:04.345233+0000","last_peered":"2023-08-11T14:03:04.345233+0000","last_clean":"2023-08-11T14:03:04.345233+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:03:04.345233+0000","last_undegraded":"2023-08-11T14:03:04.345233+0000","last_fullsized":"2023-08-11T14:03:04.345233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:08.629247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346154+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:03:04.346154+0000","last_peered":"2023-08-11T14:03:04.346154+0000","last_clean":"2023-08-11T14:03:04.346154+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:03:04.346154+0000","last_undegraded":"2023-08-11T14:03:04.346154+0000","last_fullsized":"2023-08-11T14:03:04.346154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:05:50.190228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341191+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:03:04.341191+0000","last_peered":"2023-08-11T14:03:04.341191+0000","last_clean":"2023-08-11T14:03:04.341191+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:03:04.341191+0000","last_undegraded":"2023-08-11T14:03:04.341191+0000","last_fullsized":"2023-08-11T14:03:04.341191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:20.949554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:50.490894+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:03:50.490894+0000","last_peered":"2023-08-11T14:03:50.490894+0000","last_clean":"2023-08-11T14:03:50.490894+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:03:50.490894+0000","last_undegraded":"2023-08-11T14:03:50.490894+0000","last_fullsized":"2023-08-11T14:03:50.490894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:13.008649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345556+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:03:04.345556+0000","last_peered":"2023-08-11T14:03:04.345556+0000","last_clean":"2023-08-11T14:03:04.345556+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:03:04.345556+0000","last_undegraded":"2023-08-11T14:03:04.345556+0000","last_fullsized":"2023-08-11T14:03:04.345556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:40:55.597917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341002+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:03:04.341002+0000","last_peered":"2023-08-11T14:03:04.341002+0000","last_clean":"2023-08-11T14:03:04.341002+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:03:04.341002+0000","last_undegraded":"2023-08-11T14:03:04.341002+0000","last_fullsized":"2023-08-11T14:03:04.341002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:43:57.181073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419897+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:03:04.419897+0000","last_peered":"2023-08-11T14:03:04.419897+0000","last_clean":"2023-08-11T14:03:04.419897+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:03:04.419897+0000","last_undegraded":"2023-08-11T14:03:04.419897+0000","last_fullsized":"2023-08-11T14:03:04.419897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:40:06.849906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346383+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:03:04.346383+0000","last_peered":"2023-08-11T14:03:04.346383+0000","last_clean":"2023-08-11T14:03:04.346383+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:03:04.346383+0000","last_undegraded":"2023-08-11T14:03:04.346383+0000","last_fullsized":"2023-08-11T14:03:04.346383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:53.656558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:52.322697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339638+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:03:04.339638+0000","last_peered":"2023-08-11T14:03:04.339638+0000","last_clean":"2023-08-11T14:03:04.339638+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:03:04.339638+0000","last_undegraded":"2023-08-11T14:03:04.339638+0000","last_fullsized":"2023-08-11T14:03:04.339638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:12:09.761922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429002+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:03:04.429002+0000","last_peered":"2023-08-11T14:03:04.429002+0000","last_clean":"2023-08-11T14:03:04.429002+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:03:04.429002+0000","last_undegraded":"2023-08-11T14:03:04.429002+0000","last_fullsized":"2023-08-11T14:03:04.429002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:22.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523029+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:03:04.523029+0000","last_peered":"2023-08-11T14:03:04.523029+0000","last_clean":"2023-08-11T14:03:04.523029+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:03:04.523029+0000","last_undegraded":"2023-08-11T14:03:04.523029+0000","last_fullsized":"2023-08-11T14:03:04.523029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:41.402588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345435+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:03:04.345435+0000","last_peered":"2023-08-11T14:03:04.345435+0000","last_clean":"2023-08-11T14:03:04.345435+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:03:04.345435+0000","last_undegraded":"2023-08-11T14:03:04.345435+0000","last_fullsized":"2023-08-11T14:03:04.345435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:35:33.118766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341241+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:03:04.341241+0000","last_peered":"2023-08-11T14:03:04.341241+0000","last_clean":"2023-08-11T14:03:04.341241+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:03:04.341241+0000","last_undegraded":"2023-08-11T14:03:04.341241+0000","last_fullsized":"2023-08-11T14:03:04.341241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:48.999413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429059+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:03:04.429059+0000","last_peered":"2023-08-11T14:03:04.429059+0000","last_clean":"2023-08-11T14:03:04.429059+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:03:04.429059+0000","last_undegraded":"2023-08-11T14:03:04.429059+0000","last_fullsized":"2023-08-11T14:03:04.429059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:25.445721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.494522+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:03:53.494522+0000","last_peered":"2023-08-11T14:03:53.494522+0000","last_clean":"2023-08-11T14:03:53.494522+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:03:53.494522+0000","last_undegraded":"2023-08-11T14:03:53.494522+0000","last_fullsized":"2023-08-11T14:03:53.494522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:08.555355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523699+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:03:04.523699+0000","last_peered":"2023-08-11T14:03:04.523699+0000","last_clean":"2023-08-11T14:03:04.523699+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:03:04.523699+0000","last_undegraded":"2023-08-11T14:03:04.523699+0000","last_fullsized":"2023-08-11T14:03:04.523699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:27.682173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340632+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:03:04.340632+0000","last_peered":"2023-08-11T14:03:04.340632+0000","last_clean":"2023-08-11T14:03:04.340632+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:03:04.340632+0000","last_undegraded":"2023-08-11T14:03:04.340632+0000","last_fullsized":"2023-08-11T14:03:04.340632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:07.774291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427459+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:03:04.427459+0000","last_peered":"2023-08-11T14:03:04.427459+0000","last_clean":"2023-08-11T14:03:04.427459+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:03:04.427459+0000","last_undegraded":"2023-08-11T14:03:04.427459+0000","last_fullsized":"2023-08-11T14:03:04.427459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:37.501349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523164+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:03:04.523164+0000","last_peered":"2023-08-11T14:03:04.523164+0000","last_clean":"2023-08-11T14:03:04.523164+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:03:04.523164+0000","last_undegraded":"2023-08-11T14:03:04.523164+0000","last_fullsized":"2023-08-11T14:03:04.523164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:14:29.876746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345553+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:03:04.345553+0000","last_peered":"2023-08-11T14:03:04.345553+0000","last_clean":"2023-08-11T14:03:04.345553+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:03:04.345553+0000","last_undegraded":"2023-08-11T14:03:04.345553+0000","last_fullsized":"2023-08-11T14:03:04.345553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:34.946284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346031+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:03:04.346031+0000","last_peered":"2023-08-11T14:03:04.346031+0000","last_clean":"2023-08-11T14:03:04.346031+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:03:04.346031+0000","last_undegraded":"2023-08-11T14:03:04.346031+0000","last_fullsized":"2023-08-11T14:03:04.346031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:45.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523852+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:03:04.523852+0000","last_peered":"2023-08-11T14:03:04.523852+0000","last_clean":"2023-08-11T14:03:04.523852+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:03:04.523852+0000","last_undegraded":"2023-08-11T14:03:04.523852+0000","last_fullsized":"2023-08-11T14:03:04.523852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:36:24.226941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339856+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:03:04.339856+0000","last_peered":"2023-08-11T14:03:04.339856+0000","last_clean":"2023-08-11T14:03:04.339856+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:03:04.339856+0000","last_undegraded":"2023-08-11T14:03:04.339856+0000","last_fullsized":"2023-08-11T14:03:04.339856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:30.100680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":306,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.495578+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:03:53.495578+0000","last_peered":"2023-08-11T14:03:53.495578+0000","last_clean":"2023-08-11T14:03:53.495578+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:03:53.495578+0000","last_undegraded":"2023-08-11T14:03:53.495578+0000","last_fullsized":"2023-08-11T14:03:53.495578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:31.930235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345541+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:03:04.345541+0000","last_peered":"2023-08-11T14:03:04.345541+0000","last_clean":"2023-08-11T14:03:04.345541+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:03:04.345541+0000","last_undegraded":"2023-08-11T14:03:04.345541+0000","last_fullsized":"2023-08-11T14:03:04.345541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:02:34.156179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.347052+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:03:04.347052+0000","last_peered":"2023-08-11T14:03:04.347052+0000","last_clean":"2023-08-11T14:03:04.347052+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:03:04.347052+0000","last_undegraded":"2023-08-11T14:03:04.347052+0000","last_fullsized":"2023-08-11T14:03:04.347052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:11:15.433011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342697+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:03:04.342697+0000","last_peered":"2023-08-11T14:03:04.342697+0000","last_clean":"2023-08-11T14:03:04.342697+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:03:04.342697+0000","last_undegraded":"2023-08-11T14:03:04.342697+0000","last_fullsized":"2023-08-11T14:03:04.342697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:41.784623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340323+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:03:04.340323+0000","last_peered":"2023-08-11T14:03:04.340323+0000","last_clean":"2023-08-11T14:03:04.340323+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:03:04.340323+0000","last_undegraded":"2023-08-11T14:03:04.340323+0000","last_fullsized":"2023-08-11T14:03:04.340323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:34.035219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342711+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:03:04.342711+0000","last_peered":"2023-08-11T14:03:04.342711+0000","last_clean":"2023-08-11T14:03:04.342711+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:03:04.342711+0000","last_undegraded":"2023-08-11T14:03:04.342711+0000","last_fullsized":"2023-08-11T14:03:04.342711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:29:44.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346763+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:03:04.346763+0000","last_peered":"2023-08-11T14:03:04.346763+0000","last_clean":"2023-08-11T14:03:04.346763+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:03:04.346763+0000","last_undegraded":"2023-08-11T14:03:04.346763+0000","last_fullsized":"2023-08-11T14:03:04.346763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:42.194578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427345+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:03:04.427345+0000","last_peered":"2023-08-11T14:03:04.427345+0000","last_clean":"2023-08-11T14:03:04.427345+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:03:04.427345+0000","last_undegraded":"2023-08-11T14:03:04.427345+0000","last_fullsized":"2023-08-11T14:03:04.427345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:29.378966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346097+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:03:04.346097+0000","last_peered":"2023-08-11T14:03:04.346097+0000","last_clean":"2023-08-11T14:03:04.346097+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:03:04.346097+0000","last_undegraded":"2023-08-11T14:03:04.346097+0000","last_fullsized":"2023-08-11T14:03:04.346097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:18.989452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.491776+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:03:51.491776+0000","last_peered":"2023-08-11T14:03:51.491776+0000","last_clean":"2023-08-11T14:03:51.491776+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:03:51.491776+0000","last_undegraded":"2023-08-11T14:03:51.491776+0000","last_fullsized":"2023-08-11T14:03:51.491776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:47.290694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426969+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:03:04.426969+0000","last_peered":"2023-08-11T14:03:04.426969+0000","last_clean":"2023-08-11T14:03:04.426969+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:03:04.426969+0000","last_undegraded":"2023-08-11T14:03:04.426969+0000","last_fullsized":"2023-08-11T14:03:04.426969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:46.867384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340459+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:03:04.340459+0000","last_peered":"2023-08-11T14:03:04.340459+0000","last_clean":"2023-08-11T14:03:04.340459+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:03:04.340459+0000","last_undegraded":"2023-08-11T14:03:04.340459+0000","last_fullsized":"2023-08-11T14:03:04.340459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:51.746271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.344937+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:03:04.344937+0000","last_peered":"2023-08-11T14:03:04.344937+0000","last_clean":"2023-08-11T14:03:04.344937+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:03:04.344937+0000","last_undegraded":"2023-08-11T14:03:04.344937+0000","last_fullsized":"2023-08-11T14:03:04.344937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:13.455539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341159+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:03:04.341159+0000","last_peered":"2023-08-11T14:03:04.341159+0000","last_clean":"2023-08-11T14:03:04.341159+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:03:04.341159+0000","last_undegraded":"2023-08-11T14:03:04.341159+0000","last_fullsized":"2023-08-11T14:03:04.341159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:34.163158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346090+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:03:04.346090+0000","last_peered":"2023-08-11T14:03:04.346090+0000","last_clean":"2023-08-11T14:03:04.346090+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:03:04.346090+0000","last_undegraded":"2023-08-11T14:03:04.346090+0000","last_fullsized":"2023-08-11T14:03:04.346090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:22:02.526755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.352593+0000","last_change":"2023-08-11T14:03:04.352113+0000","last_active":"2023-08-11T14:03:04.352593+0000","last_peered":"2023-08-11T14:03:04.352593+0000","last_clean":"2023-08-11T14:03:04.352593+0000","last_became_active":"2023-08-11T14:03:04.347837+0000","last_became_peered":"2023-08-11T14:03:04.347837+0000","last_unstale":"2023-08-11T14:03:04.352593+0000","last_undegraded":"2023-08-11T14:03:04.352593+0000","last_fullsized":"2023-08-11T14:03:04.352593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:47.346364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342848+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:03:04.342848+0000","last_peered":"2023-08-11T14:03:04.342848+0000","last_clean":"2023-08-11T14:03:04.342848+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:03:04.342848+0000","last_undegraded":"2023-08-11T14:03:04.342848+0000","last_fullsized":"2023-08-11T14:03:04.342848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:43.902027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345467+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:03:04.345467+0000","last_peered":"2023-08-11T14:03:04.345467+0000","last_clean":"2023-08-11T14:03:04.345467+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:03:04.345467+0000","last_undegraded":"2023-08-11T14:03:04.345467+0000","last_fullsized":"2023-08-11T14:03:04.345467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:57.057812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341336+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:03:04.341336+0000","last_peered":"2023-08-11T14:03:04.341336+0000","last_clean":"2023-08-11T14:03:04.341336+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:03:04.341336+0000","last_undegraded":"2023-08-11T14:03:04.341336+0000","last_fullsized":"2023-08-11T14:03:04.341336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:09.795680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420336+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:03:04.420336+0000","last_peered":"2023-08-11T14:03:04.420336+0000","last_clean":"2023-08-11T14:03:04.420336+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:03:04.420336+0000","last_undegraded":"2023-08-11T14:03:04.420336+0000","last_fullsized":"2023-08-11T14:03:04.420336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:26:04.032679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339836+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:03:04.339836+0000","last_peered":"2023-08-11T14:03:04.339836+0000","last_clean":"2023-08-11T14:03:04.339836+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:03:04.339836+0000","last_undegraded":"2023-08-11T14:03:04.339836+0000","last_fullsized":"2023-08-11T14:03:04.339836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:42:17.950224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":146,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:48.487314+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:03:48.487314+0000","last_peered":"2023-08-11T14:03:48.487314+0000","last_clean":"2023-08-11T14:03:48.487314+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:03:48.487314+0000","last_undegraded":"2023-08-11T14:03:48.487314+0000","last_fullsized":"2023-08-11T14:03:48.487314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:25.158801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420203+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:03:04.420203+0000","last_peered":"2023-08-11T14:03:04.420203+0000","last_clean":"2023-08-11T14:03:04.420203+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:03:04.420203+0000","last_undegraded":"2023-08-11T14:03:04.420203+0000","last_fullsized":"2023-08-11T14:03:04.420203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:57.775826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341077+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:03:04.341077+0000","last_peered":"2023-08-11T14:03:04.341077+0000","last_clean":"2023-08-11T14:03:04.341077+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:03:04.341077+0000","last_undegraded":"2023-08-11T14:03:04.341077+0000","last_fullsized":"2023-08-11T14:03:04.341077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:59.875215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345872+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:03:04.345872+0000","last_peered":"2023-08-11T14:03:04.345872+0000","last_clean":"2023-08-11T14:03:04.345872+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:03:04.345872+0000","last_undegraded":"2023-08-11T14:03:04.345872+0000","last_fullsized":"2023-08-11T14:03:04.345872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:35:57.958298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:52.492527+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:03:52.492527+0000","last_peered":"2023-08-11T14:03:52.492527+0000","last_clean":"2023-08-11T14:03:52.492527+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:03:52.492527+0000","last_undegraded":"2023-08-11T14:03:52.492527+0000","last_fullsized":"2023-08-11T14:03:52.492527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:28.567968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427398+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:03:04.427398+0000","last_peered":"2023-08-11T14:03:04.427398+0000","last_clean":"2023-08-11T14:03:04.427398+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:03:04.427398+0000","last_undegraded":"2023-08-11T14:03:04.427398+0000","last_fullsized":"2023-08-11T14:03:04.427398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:14:04.821908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429434+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:03:04.429434+0000","last_peered":"2023-08-11T14:03:04.429434+0000","last_clean":"2023-08-11T14:03:04.429434+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:03:04.429434+0000","last_undegraded":"2023-08-11T14:03:04.429434+0000","last_fullsized":"2023-08-11T14:03:04.429434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:51:44.310088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345441+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:03:04.345441+0000","last_peered":"2023-08-11T14:03:04.345441+0000","last_clean":"2023-08-11T14:03:04.345441+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:03:04.345441+0000","last_undegraded":"2023-08-11T14:03:04.345441+0000","last_fullsized":"2023-08-11T14:03:04.345441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:06.862831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426824+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:03:04.426824+0000","last_peered":"2023-08-11T14:03:04.426824+0000","last_clean":"2023-08-11T14:03:04.426824+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:03:04.426824+0000","last_undegraded":"2023-08-11T14:03:04.426824+0000","last_fullsized":"2023-08-11T14:03:04.426824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:07.631487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346982+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:03:04.346982+0000","last_peered":"2023-08-11T14:03:04.346982+0000","last_clean":"2023-08-11T14:03:04.346982+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:03:04.346982+0000","last_undegraded":"2023-08-11T14:03:04.346982+0000","last_fullsized":"2023-08-11T14:03:04.346982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:24.515141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340804+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:03:04.340804+0000","last_peered":"2023-08-11T14:03:04.340804+0000","last_clean":"2023-08-11T14:03:04.340804+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:03:04.340804+0000","last_undegraded":"2023-08-11T14:03:04.340804+0000","last_fullsized":"2023-08-11T14:03:04.340804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:23.071563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426987+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:03:04.426987+0000","last_peered":"2023-08-11T14:03:04.426987+0000","last_clean":"2023-08-11T14:03:04.426987+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:03:04.426987+0000","last_undegraded":"2023-08-11T14:03:04.426987+0000","last_fullsized":"2023-08-11T14:03:04.426987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:54:16.816623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342995+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:03:04.342995+0000","last_peered":"2023-08-11T14:03:04.342995+0000","last_clean":"2023-08-11T14:03:04.342995+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:03:04.342995+0000","last_undegraded":"2023-08-11T14:03:04.342995+0000","last_fullsized":"2023-08-11T14:03:04.342995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:05.194595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340208+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:03:04.340208+0000","last_peered":"2023-08-11T14:03:04.340208+0000","last_clean":"2023-08-11T14:03:04.340208+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:03:04.340208+0000","last_undegraded":"2023-08-11T14:03:04.340208+0000","last_fullsized":"2023-08-11T14:03:04.340208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:36.744960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.428991+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:03:04.428991+0000","last_peered":"2023-08-11T14:03:04.428991+0000","last_clean":"2023-08-11T14:03:04.428991+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:03:04.428991+0000","last_undegraded":"2023-08-11T14:03:04.428991+0000","last_fullsized":"2023-08-11T14:03:04.428991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:06:57.242042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:43.252925+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:03:43.252925+0000","last_peered":"2023-08-11T14:03:43.252925+0000","last_clean":"2023-08-11T14:03:43.252925+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:03:43.252925+0000","last_undegraded":"2023-08-11T14:03:43.252925+0000","last_fullsized":"2023-08-11T14:03:43.252925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:19:42.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429596+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:03:04.429596+0000","last_peered":"2023-08-11T14:03:04.429596+0000","last_clean":"2023-08-11T14:03:04.429596+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:03:04.429596+0000","last_undegraded":"2023-08-11T14:03:04.429596+0000","last_fullsized":"2023-08-11T14:03:04.429596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:21:59.987965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340537+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:03:04.340537+0000","last_peered":"2023-08-11T14:03:04.340537+0000","last_clean":"2023-08-11T14:03:04.340537+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:03:04.340537+0000","last_undegraded":"2023-08-11T14:03:04.340537+0000","last_fullsized":"2023-08-11T14:03:04.340537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:10:35.951459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346004+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:03:04.346004+0000","last_peered":"2023-08-11T14:03:04.346004+0000","last_clean":"2023-08-11T14:03:04.346004+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:03:04.346004+0000","last_undegraded":"2023-08-11T14:03:04.346004+0000","last_fullsized":"2023-08-11T14:03:04.346004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:55.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523190+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:03:04.523190+0000","last_peered":"2023-08-11T14:03:04.523190+0000","last_clean":"2023-08-11T14:03:04.523190+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:03:04.523190+0000","last_undegraded":"2023-08-11T14:03:04.523190+0000","last_fullsized":"2023-08-11T14:03:04.523190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:13:04.607004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345962+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:03:04.345962+0000","last_peered":"2023-08-11T14:03:04.345962+0000","last_clean":"2023-08-11T14:03:04.345962+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:03:04.345962+0000","last_undegraded":"2023-08-11T14:03:04.345962+0000","last_fullsized":"2023-08-11T14:03:04.345962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:46.950942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523374+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:03:04.523374+0000","last_peered":"2023-08-11T14:03:04.523374+0000","last_clean":"2023-08-11T14:03:04.523374+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:03:04.523374+0000","last_undegraded":"2023-08-11T14:03:04.523374+0000","last_fullsized":"2023-08-11T14:03:04.523374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:40:06.480953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345005+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:03:04.345005+0000","last_peered":"2023-08-11T14:03:04.345005+0000","last_clean":"2023-08-11T14:03:04.345005+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:03:04.345005+0000","last_undegraded":"2023-08-11T14:03:04.345005+0000","last_fullsized":"2023-08-11T14:03:04.345005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:58.380684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341596+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:03:04.341596+0000","last_peered":"2023-08-11T14:03:04.341596+0000","last_clean":"2023-08-11T14:03:04.341596+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:03:04.341596+0000","last_undegraded":"2023-08-11T14:03:04.341596+0000","last_fullsized":"2023-08-11T14:03:04.341596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:48.675324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429201+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:03:04.429201+0000","last_peered":"2023-08-11T14:03:04.429201+0000","last_clean":"2023-08-11T14:03:04.429201+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:03:04.429201+0000","last_undegraded":"2023-08-11T14:03:04.429201+0000","last_fullsized":"2023-08-11T14:03:04.429201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:03.363760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342610+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:03:04.342610+0000","last_peered":"2023-08-11T14:03:04.342610+0000","last_clean":"2023-08-11T14:03:04.342610+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:03:04.342610+0000","last_undegraded":"2023-08-11T14:03:04.342610+0000","last_fullsized":"2023-08-11T14:03:04.342610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:33:37.384946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:08.705807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340884+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:03:04.340884+0000","last_peered":"2023-08-11T14:03:04.340884+0000","last_clean":"2023-08-11T14:03:04.340884+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:03:04.340884+0000","last_undegraded":"2023-08-11T14:03:04.340884+0000","last_fullsized":"2023-08-11T14:03:04.340884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:52.040366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429419+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:03:04.429419+0000","last_peered":"2023-08-11T14:03:04.429419+0000","last_clean":"2023-08-11T14:03:04.429419+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:03:04.429419+0000","last_undegraded":"2023-08-11T14:03:04.429419+0000","last_fullsized":"2023-08-11T14:03:04.429419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:31.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345789+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:03:04.345789+0000","last_peered":"2023-08-11T14:03:04.345789+0000","last_clean":"2023-08-11T14:03:04.345789+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:03:04.345789+0000","last_undegraded":"2023-08-11T14:03:04.345789+0000","last_fullsized":"2023-08-11T14:03:04.345789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:14.187908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345206+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:03:04.345206+0000","last_peered":"2023-08-11T14:03:04.345206+0000","last_clean":"2023-08-11T14:03:04.345206+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:03:04.345206+0000","last_undegraded":"2023-08-11T14:03:04.345206+0000","last_fullsized":"2023-08-11T14:03:04.345206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:21:37.922008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427322+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:03:04.427322+0000","last_peered":"2023-08-11T14:03:04.427322+0000","last_clean":"2023-08-11T14:03:04.427322+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:03:04.427322+0000","last_undegraded":"2023-08-11T14:03:04.427322+0000","last_fullsized":"2023-08-11T14:03:04.427322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:55.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523362+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:03:04.523362+0000","last_peered":"2023-08-11T14:03:04.523362+0000","last_clean":"2023-08-11T14:03:04.523362+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:03:04.523362+0000","last_undegraded":"2023-08-11T14:03:04.523362+0000","last_fullsized":"2023-08-11T14:03:04.523362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:36.812617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:49.502291+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:03:49.502291+0000","last_peered":"2023-08-11T14:03:49.502291+0000","last_clean":"2023-08-11T14:03:49.502291+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:03:49.502291+0000","last_undegraded":"2023-08-11T14:03:49.502291+0000","last_fullsized":"2023-08-11T14:03:49.502291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:16:25.101731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427708+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:03:04.427708+0000","last_peered":"2023-08-11T14:03:04.427708+0000","last_clean":"2023-08-11T14:03:04.427708+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:03:04.427708+0000","last_undegraded":"2023-08-11T14:03:04.427708+0000","last_fullsized":"2023-08-11T14:03:04.427708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:45:18.108583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522691+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:03:04.522691+0000","last_peered":"2023-08-11T14:03:04.522691+0000","last_clean":"2023-08-11T14:03:04.522691+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:03:04.522691+0000","last_undegraded":"2023-08-11T14:03:04.522691+0000","last_fullsized":"2023-08-11T14:03:04.522691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:56.757871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346920+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:03:04.346920+0000","last_peered":"2023-08-11T14:03:04.346920+0000","last_clean":"2023-08-11T14:03:04.346920+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:03:04.346920+0000","last_undegraded":"2023-08-11T14:03:04.346920+0000","last_fullsized":"2023-08-11T14:03:04.346920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:44.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427304+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:03:04.427304+0000","last_peered":"2023-08-11T14:03:04.427304+0000","last_clean":"2023-08-11T14:03:04.427304+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:03:04.427304+0000","last_undegraded":"2023-08-11T14:03:04.427304+0000","last_fullsized":"2023-08-11T14:03:04.427304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:53:39.819984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345073+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:03:04.345073+0000","last_peered":"2023-08-11T14:03:04.345073+0000","last_clean":"2023-08-11T14:03:04.345073+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:03:04.345073+0000","last_undegraded":"2023-08-11T14:03:04.345073+0000","last_fullsized":"2023-08-11T14:03:04.345073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:05.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340636+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:03:04.340636+0000","last_peered":"2023-08-11T14:03:04.340636+0000","last_clean":"2023-08-11T14:03:04.340636+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:03:04.340636+0000","last_undegraded":"2023-08-11T14:03:04.340636+0000","last_fullsized":"2023-08-11T14:03:04.340636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:43.294958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427646+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:03:04.427646+0000","last_peered":"2023-08-11T14:03:04.427646+0000","last_clean":"2023-08-11T14:03:04.427646+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:03:04.427646+0000","last_undegraded":"2023-08-11T14:03:04.427646+0000","last_fullsized":"2023-08-11T14:03:04.427646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:53.158734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342712+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:03:04.342712+0000","last_peered":"2023-08-11T14:03:04.342712+0000","last_clean":"2023-08-11T14:03:04.342712+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:03:04.342712+0000","last_undegraded":"2023-08-11T14:03:04.342712+0000","last_fullsized":"2023-08-11T14:03:04.342712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:26.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340809+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:03:04.340809+0000","last_peered":"2023-08-11T14:03:04.340809+0000","last_clean":"2023-08-11T14:03:04.340809+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:03:04.340809+0000","last_undegraded":"2023-08-11T14:03:04.340809+0000","last_fullsized":"2023-08-11T14:03:04.340809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:39.383017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340630+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:03:04.340630+0000","last_peered":"2023-08-11T14:03:04.340630+0000","last_clean":"2023-08-11T14:03:04.340630+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:03:04.340630+0000","last_undegraded":"2023-08-11T14:03:04.340630+0000","last_fullsized":"2023-08-11T14:03:04.340630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:09:51.511408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522979+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:03:04.522979+0000","last_peered":"2023-08-11T14:03:04.522979+0000","last_clean":"2023-08-11T14:03:04.522979+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:03:04.522979+0000","last_undegraded":"2023-08-11T14:03:04.522979+0000","last_fullsized":"2023-08-11T14:03:04.522979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:38:58.624580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340049+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:03:04.340049+0000","last_peered":"2023-08-11T14:03:04.340049+0000","last_clean":"2023-08-11T14:03:04.340049+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:03:04.340049+0000","last_undegraded":"2023-08-11T14:03:04.340049+0000","last_fullsized":"2023-08-11T14:03:04.340049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:10:16.822501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:34.479972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341526+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:03:04.341526+0000","last_peered":"2023-08-11T14:03:04.341526+0000","last_clean":"2023-08-11T14:03:04.341526+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:03:04.341526+0000","last_undegraded":"2023-08-11T14:03:04.341526+0000","last_fullsized":"2023-08-11T14:03:04.341526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.188787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429642+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:03:04.429642+0000","last_peered":"2023-08-11T14:03:04.429642+0000","last_clean":"2023-08-11T14:03:04.429642+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:03:04.429642+0000","last_undegraded":"2023-08-11T14:03:04.429642+0000","last_fullsized":"2023-08-11T14:03:04.429642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:01:08.200835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345624+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:03:04.345624+0000","last_peered":"2023-08-11T14:03:04.345624+0000","last_clean":"2023-08-11T14:03:04.345624+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:03:04.345624+0000","last_undegraded":"2023-08-11T14:03:04.345624+0000","last_fullsized":"2023-08-11T14:03:04.345624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:08.040487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429411+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:03:04.429411+0000","last_peered":"2023-08-11T14:03:04.429411+0000","last_clean":"2023-08-11T14:03:04.429411+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:03:04.429411+0000","last_undegraded":"2023-08-11T14:03:04.429411+0000","last_fullsized":"2023-08-11T14:03:04.429411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:43:43.729999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523915+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:03:04.523915+0000","last_peered":"2023-08-11T14:03:04.523915+0000","last_clean":"2023-08-11T14:03:04.523915+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:03:04.523915+0000","last_undegraded":"2023-08-11T14:03:04.523915+0000","last_fullsized":"2023-08-11T14:03:04.523915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:25:07.770441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340919+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:03:04.340919+0000","last_peered":"2023-08-11T14:03:04.340919+0000","last_clean":"2023-08-11T14:03:04.340919+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:03:04.340919+0000","last_undegraded":"2023-08-11T14:03:04.340919+0000","last_fullsized":"2023-08-11T14:03:04.340919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:14.465240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342974+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:03:04.342974+0000","last_peered":"2023-08-11T14:03:04.342974+0000","last_clean":"2023-08-11T14:03:04.342974+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:03:04.342974+0000","last_undegraded":"2023-08-11T14:03:04.342974+0000","last_fullsized":"2023-08-11T14:03:04.342974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:01.425835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73338,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1155,"num_read_kb":664,"num_write":250,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":214635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":509222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.56999999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.88,"15min":0.88},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.296,"15min":1.296},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":1.208,"15min":1.208},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.048,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.876,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84599999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80600000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.006}]},{"osd":9,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0109999999999999,"5min":1.27,"15min":1.27},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0720000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97399999999999998,"5min":1.139,"15min":1.139},"last":0.93500000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678060,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1162864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.278,"5min":1.526,"15min":1.526},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.117,"5min":1.516,"15min":1.516},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92400000000000004,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.70399999999999996},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":0.33700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.378,"15min":0.378},"max":{"1min":0.97799999999999998,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.438},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.085,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77800000000000002,"15min":0.81100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.012,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.57199999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":1.605,"15min":1.605},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.55100000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98499999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84499999999999997,"15min":0.88400000000000001},"min":{"1min":0.52600000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.92100000000000004,"15min":0.90300000000000002},"min":{"1min":0.67000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95199999999999996,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Aug 11 14:03:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.65700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":1.234,"15min":1.234},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.748,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.129,"15min":1.129},"last":0.54300000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":513158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.58999999999999997,"15min":0.63200000000000001},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71999999999999997,"15min":0.748},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":1,"15min":1},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.53700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69799999999999995,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89800000000000002,"5min":1.123,"15min":1.123},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.74199999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92700000000000005,"5min":1.165,"15min":1.165},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.85599999999999998,"15min":0.90000000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.2149999999999999,"5min":3.2149999999999999,"15min":3.2149999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.86499999999999999,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":0.95299999999999996},"min":{"1min":0.60299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1040000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.93999999999999995,"15min":1.018},"min":{"1min":0.505,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.94799999999999995,"15min":1.0229999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0309999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.878}]},{"osd":8,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.079,"5min":1.484,"15min":1.484},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0580000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.629}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.96799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":0.84099999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.88600000000000001,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.86799999999999999,"15min":0.84099999999999997},"min":{"1min":0.72299999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.68799999999999994}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":509418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.625,"15min":0.623},"min":{"1min":0.47099999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":5.3159999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.53200000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.56399999999999995,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.626}]},{"osd":2,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72799999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.40899999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.39900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":1.022,"15min":1.022},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.56399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.501,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.59699999999999998},"min":{"1min":0.46400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.91000000000000003,"5min":1.101,"15min":1.101},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.79300000000000004,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.65300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.042,"15min":1.042},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.67300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98599999999999999,"5min":1.083,"15min":1.083},"last":0.877},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96999999999999997,"5min":1.29,"15min":1.29},"last":0.66600000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0509999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.82799999999999996,"15min":0.79800000000000004},"min":{"1min":0.79700000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.72799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.79800000000000004,"15min":0.754},"min":{"1min":0.78900000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84799999999999998,"15min":0.84199999999999997},"min":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.92400000000000004},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.02,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71699999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1105804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56799999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.34300000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66800000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.64400000000000002,"15min":0.59399999999999997},"min":{"1min":0.47699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.6130000000000004,"5min":4.6130000000000004,"15min":4.6130000000000004},"last":0.504},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.753,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.377},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83899999999999997,"5min":1.034,"15min":1.034},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.82799999999999996,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.798,"5min":6.798,"15min":6.798},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0129999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.66000000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.96899999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.6050000000000004,"5min":5.6050000000000004,"15min":5.6050000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.86099999999999999,"15min":0.83899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97399999999999998,"5min":1.095,"15min":1.095},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.60099999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88900000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66700000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85599999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.67600000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.95599999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77600000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":504566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.73399999999999999,"15min":0.70399999999999996},"min":{"1min":0.52300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.52800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.131,"15min":1.131},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.433,"15min":0.433},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.133,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.82299999999999995,"15min":0.80700000000000005},"min":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Aug 11 14:03:24 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81399999999999995,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.70299999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1339999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92700000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84899999999999998,"15min":0.88200000000000001},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.9220000000000002,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.55700000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9420000000000002,"5min":3.9420000000000002,"15min":3.9420000000000002},"last":0.66300000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:05 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.93999999999999995,"15min":0.95099999999999996},"min":{"1min":0.63900000000000001,"5min":0.5,"15min":0.5},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86899999999999999,"15min":0.85299999999999998},"min":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.7909999999999999,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92100000000000004,"15min":0.92600000000000005},"min":{"1min":0.77600000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1519999999999999,"5min":1.363,"15min":1.363},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Aug 11 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91400000000000003,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.98,"5min":3.98,"15min":3.98},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.93799999999999994,"15min":0.92000000000000004},"min":{"1min":0.72399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":571708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.054},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82299999999999995,"5min":1.234,"15min":1.234},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.63400000000000001,"15min":0.54800000000000004},"min":{"1min":0.48299999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.95899999999999996,"5min":0.995,"15min":0.995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.55900000000000005},"min":{"1min":0.50700000000000001,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:02:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.51300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.60599999999999998},"min":{"1min":0.55800000000000005,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.625}]},{"osd":4,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48999999999999999,"15min":0.41799999999999998},"min":{"1min":0.36299999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80300000000000005,"5min":1.119,"15min":1.119},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.47099999999999997,"15min":0.43099999999999999},"min":{"1min":0.39500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.65600000000000003,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96899999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69299999999999995,"15min":0.59899999999999998},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94399999999999995,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.54300000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99299999999999999,"5min":3.153,"15min":3.153},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.69299999999999995,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0349999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63400000000000001,"15min":0.55600000000000005},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.089,"5min":1.159,"15min":1.159},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.67500000000000004,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.115,"5min":1.345,"15min":1.345},"last":0.61299999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76700000000000002,"15min":0.70899999999999996},"min":{"1min":0.73599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1799999999999999,"5min":1.302,"15min":1.302},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78000000000000003,"15min":0.71599999999999997},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.131,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:03:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.641,"5min":3.194,"15min":3.194},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.85399999999999998,"15min":0.83999999999999997},"min":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1839999999999999,"5min":1.393,"15min":1.393},"last":0.751}]},{"osd":10,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.59099999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.091,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.073,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.151,"5min":1.21,"15min":1.21},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":506022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64500000000000002,"15min":0.59099999999999997},"min":{"1min":0.51900000000000002,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.002,"5min":0.70599999999999996,"15min":0.621},"min":{"1min":0.495,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.35399999999999998},"max":{"1min":0.93700000000000006,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":1.724,"5min":6.2910000000000004,"15min":6.2910000000000004},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67300000000000004,"15min":0.68899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0049999999999999,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:03:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.72999999999999998,"15min":0.627},"min":{"1min":0.496,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.1619999999999999,"5min":6.1619999999999999,"15min":6.1619999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.68999999999999995,"15min":0.60599999999999998},"min":{"1min":0.54900000000000004,"5min":0.317,"15min":0.317},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79100000000000004,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.8029999999999999,"5min":6.25,"15min":6.25},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0169999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.52300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79000000000000004,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":0.755}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.93300000000000005,"15min":0.94799999999999995},"min":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.091,"5min":3.254,"15min":3.254},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.069,"5min":1.234,"15min":1.234},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.63900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0669999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91100000000000003,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.093,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.98199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":0.92400000000000004,"15min":0.90500000000000003},"min":{"1min":0.625,"5min":0.376,"15min":0.376},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.93999999999999995,"15min":0.88600000000000001},"min":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7629999999999999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92500000000000004,"15min":0.90000000000000002},"min":{"1min":0.69699999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81799999999999995}]},{"osd":11,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.77200000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.153,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.90500000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.177,"5min":6.3739999999999997,"15min":6.3739999999999997},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":566754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.60799999999999998,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.0780000000000003,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.82399999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.30099999999999999},"max":{"1min":0.92300000000000004,"5min":1.036,"15min":1.036},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60599999999999998,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.92500000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81899999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.624,"15min":0.57399999999999995},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.86299999999999999,"5min":0.876,"15min":0.876},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.56799999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.41999999999999998},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":3.218,"15min":3.218},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.44},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.72199999999999998,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.436,"15min":0.436},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":0.90400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.80000000000000004,"15min":0.79400000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Aug 11 14:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.58699999999999997},"min":{"1min":0.48799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.59799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.006,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.63}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.67900000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.73199999999999998},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1120000000000001,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":0.94999999999999996,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.745},"min":{"1min":0.60399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.63900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0680000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":504274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.42199999999999999},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.68600000000000005,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.423,"5min":6.423,"15min":6.423},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.61299999999999999},"min":{"1min":0.57199999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":2.548,"15min":2.548},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.72499999999999998,"15min":0.65500000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":6.5010000000000003,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69299999999999995,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2849999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.2010000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.751,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.89400000000000002,"15min":0.83399999999999996},"min":{"1min":0.65900000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":8.6370000000000005,"5min":8.6370000000000005,"15min":8.6370000000000005},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96199999999999997,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.92800000000000005},"min":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.92200000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.179,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1879999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.92200000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.254,"5min":1.379,"15min":1.379},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.169,"5min":0.89200000000000002,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":6.4610000000000003,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.98799999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.33000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61199999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.38600000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65800000000000003,"5min":0.875,"15min":0.875},"last":0.376}]},{"osd":2,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64900000000000002,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.436},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.496}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.98899999999999999,"5min":1.02,"15min":1.02},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.65500000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0409999999999999,"5min":1.075,"15min":1.075},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.64100000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.67500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78100000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1101128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":1.46,"15min":1.46},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.38,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.52800000000000002},"min":{"1min":0.46899999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82199999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.56100000000000005,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.39900000000000002},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.59899999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.442,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.409,"15min":1.409},"last":0.626},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63200000000000001,"15min":0.57699999999999996},"min":{"1min":0.5,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59599999999999997},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":1.393,"15min":1.393},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89400000000000002,"5min":1.484,"15min":1.484},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.878},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.82599999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98999999999999999,"5min":1.181,"15min":1.181},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.78500000000000003,"15min":0.76100000000000001},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96899999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:03:54.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:03:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 -- 172.21.15.3:0/538160248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8073380 msgr2=0x7f93a8073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 --2- 172.21.15.3:0/538160248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8073380 0x7f93a8073800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9394009990 tx=0x7f9394009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 -- 172.21.15.3:0/538160248 shutdown_connections 2023-08-11T14:03:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 --2- 172.21.15.3:0/538160248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 0x7f93a810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 --2- 172.21.15.3:0/538160248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8074960 0x7f93a8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 --2- 172.21.15.3:0/538160248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8073380 0x7f93a8073800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 -- 172.21.15.3:0/538160248 >> 172.21.15.3:0/538160248 conn(0x7f93a80fbe60 msgr2=0x7f93a80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 -- 172.21.15.3:0/538160248 shutdown_connections 2023-08-11T14:03:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.592+0000 7f93af1ff700 1 -- 172.21.15.3:0/538160248 wait complete. 2023-08-11T14:03:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.593+0000 7f93af1ff700 1 Processor -- start 2023-08-11T14:03:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 -- start start 2023-08-11T14:03:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 0x7f93a819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 0x7f93a81a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81a4760 con 0x7f93a8074960 2023-08-11T14:03:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81a48d0 con 0x7f93a8073380 2023-08-11T14:03:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93af1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81a4a90 con 0x7f93a810cf00 2023-08-11T14:03:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 0x7f93a819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93acf9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93a7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 0x7f93a819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59652/0 (socket says 172.21.15.3:59652) 2023-08-11T14:03:54.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93acf9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48790/0 (socket says 172.21.15.3:48790) 2023-08-11T14:03:54.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93a7fff700 1 -- 172.21.15.3:0/1439733280 learned_addr learned my addr 172.21.15.3:0/1439733280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:54.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.594+0000 7f93ad79c700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 0x7f93a81a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 -- 172.21.15.3:0/1439733280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 msgr2=0x7f93a81a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 0x7f93a81a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 -- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 msgr2=0x7f93a819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 0x7f93a819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 -- 172.21.15.3:0/1439733280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940095f0 con 0x7f93a8073380 2023-08-11T14:03:54.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93acf9b700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f939c00e9f0 tx=0x7f939c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:54.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00c9c0 con 0x7f93a8073380 2023-08-11T14:03:54.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f939c010430 con 0x7f93a8073380 2023-08-11T14:03:54.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c018680 con 0x7f93a8073380 2023-08-11T14:03:54.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81a4d70 con 0x7f93a8073380 2023-08-11T14:03:54.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.595+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a81a51d0 con 0x7f93a8073380 2023-08-11T14:03:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.597+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8102a50 con 0x7f93a8073380 2023-08-11T14:03:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.598+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f939c010860 con 0x7f93a8073380 2023-08-11T14:03:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.600+0000 7f93a5ffb700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 0x7f93980a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.601+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f939c014070 con 0x7f93a8073380 2023-08-11T14:03:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.601+0000 7f93a7fff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 0x7f93980a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.601+0000 7f93a7fff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 0x7f93980a2460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f939400b2e0 tx=0x7f9394005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.603+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f939c051c30 con 0x7f93a8073380 2023-08-11T14:03:54.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3000187800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:54.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3000187800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:54.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3000187800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:54.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.840+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93a802ce10 con 0x7f939809ffa0 2023-08-11T14:03:54.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.856+0000 7f93a5ffb700 1 -- 172.21.15.3:0/1439733280 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427086 (secure 0 0 0) 0x7f93a802ce10 con 0x7f939809ffa0 2023-08-11T14:03:54.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:54.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 msgr2=0x7f93980a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 0x7f93980a2460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f939400b2e0 tx=0x7f9394005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 msgr2=0x7f93a819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:54.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f939c00e9f0 tx=0x7f939c00ed00 comp rx=0 tx=0).stop 2023-08-11T14:03:54.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 shutdown_connections 2023-08-11T14:03:54.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93a810cf00 0x7f93a81a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f939809ffa0 0x7f93980a2460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93a8073380 0x7f93a819cc20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 --2- 172.21.15.3:0/1439733280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93a8074960 0x7f93a819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:54.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 >> 172.21.15.3:0/1439733280 conn(0x7f93a80fbe60 msgr2=0x7f93a80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:54.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 shutdown_connections 2023-08-11T14:03:54.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:54.861+0000 7f93af1ff700 1 -- 172.21.15.3:0/1439733280 wait complete. 2023-08-11T14:03:54.872 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:03:54.883 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2023-08-11T14:03:54.201169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663952,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1281,"num_read_kb":773,"num_write":433,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338564,"kb_used_data":14332,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603260,"statfs":{"total":1151940427776,"available":1151593738240,"internally_reserved":0,"allocated":14675968,"data_stored":8064089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":82,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343170+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:03:04.343170+0000","last_peered":"2023-08-11T14:03:04.343170+0000","last_clean":"2023-08-11T14:03:04.343170+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:03:04.343170+0000","last_undegraded":"2023-08-11T14:03:04.343170+0000","last_fullsized":"2023-08-11T14:03:04.343170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:30.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523051+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:03:04.523051+0000","last_peered":"2023-08-11T14:03:04.523051+0000","last_clean":"2023-08-11T14:03:04.523051+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:03:04.523051+0000","last_undegraded":"2023-08-11T14:03:04.523051+0000","last_fullsized":"2023-08-11T14:03:04.523051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:59.858967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522981+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:03:04.522981+0000","last_peered":"2023-08-11T14:03:04.522981+0000","last_clean":"2023-08-11T14:03:04.522981+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:03:04.522981+0000","last_undegraded":"2023-08-11T14:03:04.522981+0000","last_fullsized":"2023-08-11T14:03:04.522981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:37.543189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346016+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:03:04.346016+0000","last_peered":"2023-08-11T14:03:04.346016+0000","last_clean":"2023-08-11T14:03:04.346016+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:03:04.346016+0000","last_undegraded":"2023-08-11T14:03:04.346016+0000","last_fullsized":"2023-08-11T14:03:04.346016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:42.564980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345098+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:03:04.345098+0000","last_peered":"2023-08-11T14:03:04.345098+0000","last_clean":"2023-08-11T14:03:04.345098+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:03:04.345098+0000","last_undegraded":"2023-08-11T14:03:04.345098+0000","last_fullsized":"2023-08-11T14:03:04.345098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:03.679224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419950+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:03:04.419950+0000","last_peered":"2023-08-11T14:03:04.419950+0000","last_clean":"2023-08-11T14:03:04.419950+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:03:04.419950+0000","last_undegraded":"2023-08-11T14:03:04.419950+0000","last_fullsized":"2023-08-11T14:03:04.419950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:51:16.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426874+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:03:04.426874+0000","last_peered":"2023-08-11T14:03:04.426874+0000","last_clean":"2023-08-11T14:03:04.426874+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:03:04.426874+0000","last_undegraded":"2023-08-11T14:03:04.426874+0000","last_fullsized":"2023-08-11T14:03:04.426874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:25.534139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340357+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:03:04.340357+0000","last_peered":"2023-08-11T14:03:04.340357+0000","last_clean":"2023-08-11T14:03:04.340357+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:03:04.340357+0000","last_undegraded":"2023-08-11T14:03:04.340357+0000","last_fullsized":"2023-08-11T14:03:04.340357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:32.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523704+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:03:04.523704+0000","last_peered":"2023-08-11T14:03:04.523704+0000","last_clean":"2023-08-11T14:03:04.523704+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:03:04.523704+0000","last_undegraded":"2023-08-11T14:03:04.523704+0000","last_fullsized":"2023-08-11T14:03:04.523704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:36:08.283056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420429+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:03:04.420429+0000","last_peered":"2023-08-11T14:03:04.420429+0000","last_clean":"2023-08-11T14:03:04.420429+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:03:04.420429+0000","last_undegraded":"2023-08-11T14:03:04.420429+0000","last_fullsized":"2023-08-11T14:03:04.420429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:36:40.405178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427584+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:03:04.427584+0000","last_peered":"2023-08-11T14:03:04.427584+0000","last_clean":"2023-08-11T14:03:04.427584+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:03:04.427584+0000","last_undegraded":"2023-08-11T14:03:04.427584+0000","last_fullsized":"2023-08-11T14:03:04.427584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:57:34.234658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345884+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:03:04.345884+0000","last_peered":"2023-08-11T14:03:04.345884+0000","last_clean":"2023-08-11T14:03:04.345884+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:03:04.345884+0000","last_undegraded":"2023-08-11T14:03:04.345884+0000","last_fullsized":"2023-08-11T14:03:04.345884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:02:05.560872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340734+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:03:04.340734+0000","last_peered":"2023-08-11T14:03:04.340734+0000","last_clean":"2023-08-11T14:03:04.340734+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:03:04.340734+0000","last_undegraded":"2023-08-11T14:03:04.340734+0000","last_fullsized":"2023-08-11T14:03:04.340734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:14.969554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522728+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:03:04.522728+0000","last_peered":"2023-08-11T14:03:04.522728+0000","last_clean":"2023-08-11T14:03:04.522728+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:03:04.522728+0000","last_undegraded":"2023-08-11T14:03:04.522728+0000","last_fullsized":"2023-08-11T14:03:04.522728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:14:20.923343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522719+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:03:04.522719+0000","last_peered":"2023-08-11T14:03:04.522719+0000","last_clean":"2023-08-11T14:03:04.522719+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:03:04.522719+0000","last_undegraded":"2023-08-11T14:03:04.522719+0000","last_fullsized":"2023-08-11T14:03:04.522719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:47.730825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523337+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:03:04.523337+0000","last_peered":"2023-08-11T14:03:04.523337+0000","last_clean":"2023-08-11T14:03:04.523337+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:03:04.523337+0000","last_undegraded":"2023-08-11T14:03:04.523337+0000","last_fullsized":"2023-08-11T14:03:04.523337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:16.271940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343273+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:03:04.343273+0000","last_peered":"2023-08-11T14:03:04.343273+0000","last_clean":"2023-08-11T14:03:04.343273+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:03:04.343273+0000","last_undegraded":"2023-08-11T14:03:04.343273+0000","last_fullsized":"2023-08-11T14:03:04.343273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:44:22.475322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341624+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:03:04.341624+0000","last_peered":"2023-08-11T14:03:04.341624+0000","last_clean":"2023-08-11T14:03:04.341624+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:03:04.341624+0000","last_undegraded":"2023-08-11T14:03:04.341624+0000","last_fullsized":"2023-08-11T14:03:04.341624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:44:26.149153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341477+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:03:04.341477+0000","last_peered":"2023-08-11T14:03:04.341477+0000","last_clean":"2023-08-11T14:03:04.341477+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:03:04.341477+0000","last_undegraded":"2023-08-11T14:03:04.341477+0000","last_fullsized":"2023-08-11T14:03:04.341477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:32.692687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340660+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:03:04.340660+0000","last_peered":"2023-08-11T14:03:04.340660+0000","last_clean":"2023-08-11T14:03:04.340660+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:03:04.340660+0000","last_undegraded":"2023-08-11T14:03:04.340660+0000","last_fullsized":"2023-08-11T14:03:04.340660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:35.403525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.490717+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:03:51.490717+0000","last_peered":"2023-08-11T14:03:51.490717+0000","last_clean":"2023-08-11T14:03:51.490717+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:03:51.490717+0000","last_undegraded":"2023-08-11T14:03:51.490717+0000","last_fullsized":"2023-08-11T14:03:51.490717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:30.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429485+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:03:04.429485+0000","last_peered":"2023-08-11T14:03:04.429485+0000","last_clean":"2023-08-11T14:03:04.429485+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:03:04.429485+0000","last_undegraded":"2023-08-11T14:03:04.429485+0000","last_fullsized":"2023-08-11T14:03:04.429485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:08.247797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345171+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:03:04.345171+0000","last_peered":"2023-08-11T14:03:04.345171+0000","last_clean":"2023-08-11T14:03:04.345171+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:03:04.345171+0000","last_undegraded":"2023-08-11T14:03:04.345171+0000","last_fullsized":"2023-08-11T14:03:04.345171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:28:23.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429535+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:03:04.429535+0000","last_peered":"2023-08-11T14:03:04.429535+0000","last_clean":"2023-08-11T14:03:04.429535+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:03:04.429535+0000","last_undegraded":"2023-08-11T14:03:04.429535+0000","last_fullsized":"2023-08-11T14:03:04.429535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:02.749730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523055+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:03:04.523055+0000","last_peered":"2023-08-11T14:03:04.523055+0000","last_clean":"2023-08-11T14:03:04.523055+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:03:04.523055+0000","last_undegraded":"2023-08-11T14:03:04.523055+0000","last_fullsized":"2023-08-11T14:03:04.523055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:21.968021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345306+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:03:04.345306+0000","last_peered":"2023-08-11T14:03:04.345306+0000","last_clean":"2023-08-11T14:03:04.345306+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:03:04.345306+0000","last_undegraded":"2023-08-11T14:03:04.345306+0000","last_fullsized":"2023-08-11T14:03:04.345306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:27:36.143460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345867+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:03:04.345867+0000","last_peered":"2023-08-11T14:03:04.345867+0000","last_clean":"2023-08-11T14:03:04.345867+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:03:04.345867+0000","last_undegraded":"2023-08-11T14:03:04.345867+0000","last_fullsized":"2023-08-11T14:03:04.345867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:28.879262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523408+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:03:04.523408+0000","last_peered":"2023-08-11T14:03:04.523408+0000","last_clean":"2023-08-11T14:03:04.523408+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:03:04.523408+0000","last_undegraded":"2023-08-11T14:03:04.523408+0000","last_fullsized":"2023-08-11T14:03:04.523408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:56.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345326+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:03:04.345326+0000","last_peered":"2023-08-11T14:03:04.345326+0000","last_clean":"2023-08-11T14:03:04.345326+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:03:04.345326+0000","last_undegraded":"2023-08-11T14:03:04.345326+0000","last_fullsized":"2023-08-11T14:03:04.345326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:11.483871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345233+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:03:04.345233+0000","last_peered":"2023-08-11T14:03:04.345233+0000","last_clean":"2023-08-11T14:03:04.345233+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:03:04.345233+0000","last_undegraded":"2023-08-11T14:03:04.345233+0000","last_fullsized":"2023-08-11T14:03:04.345233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:08.629247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346154+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:03:04.346154+0000","last_peered":"2023-08-11T14:03:04.346154+0000","last_clean":"2023-08-11T14:03:04.346154+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:03:04.346154+0000","last_undegraded":"2023-08-11T14:03:04.346154+0000","last_fullsized":"2023-08-11T14:03:04.346154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:05:50.190228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341191+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:03:04.341191+0000","last_peered":"2023-08-11T14:03:04.341191+0000","last_clean":"2023-08-11T14:03:04.341191+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:03:04.341191+0000","last_undegraded":"2023-08-11T14:03:04.341191+0000","last_fullsized":"2023-08-11T14:03:04.341191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:20.949554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:50.490894+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:03:50.490894+0000","last_peered":"2023-08-11T14:03:50.490894+0000","last_clean":"2023-08-11T14:03:50.490894+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:03:50.490894+0000","last_undegraded":"2023-08-11T14:03:50.490894+0000","last_fullsized":"2023-08-11T14:03:50.490894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:13.008649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345556+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:03:04.345556+0000","last_peered":"2023-08-11T14:03:04.345556+0000","last_clean":"2023-08-11T14:03:04.345556+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:03:04.345556+0000","last_undegraded":"2023-08-11T14:03:04.345556+0000","last_fullsized":"2023-08-11T14:03:04.345556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:40:55.597917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341002+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:03:04.341002+0000","last_peered":"2023-08-11T14:03:04.341002+0000","last_clean":"2023-08-11T14:03:04.341002+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:03:04.341002+0000","last_undegraded":"2023-08-11T14:03:04.341002+0000","last_fullsized":"2023-08-11T14:03:04.341002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:43:57.181073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419897+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:03:04.419897+0000","last_peered":"2023-08-11T14:03:04.419897+0000","last_clean":"2023-08-11T14:03:04.419897+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:03:04.419897+0000","last_undegraded":"2023-08-11T14:03:04.419897+0000","last_fullsized":"2023-08-11T14:03:04.419897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:40:06.849906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346383+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:03:04.346383+0000","last_peered":"2023-08-11T14:03:04.346383+0000","last_clean":"2023-08-11T14:03:04.346383+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:03:04.346383+0000","last_undegraded":"2023-08-11T14:03:04.346383+0000","last_fullsized":"2023-08-11T14:03:04.346383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:53.656558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:52.322697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339638+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:03:04.339638+0000","last_peered":"2023-08-11T14:03:04.339638+0000","last_clean":"2023-08-11T14:03:04.339638+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:03:04.339638+0000","last_undegraded":"2023-08-11T14:03:04.339638+0000","last_fullsized":"2023-08-11T14:03:04.339638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:12:09.761922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429002+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:03:04.429002+0000","last_peered":"2023-08-11T14:03:04.429002+0000","last_clean":"2023-08-11T14:03:04.429002+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:03:04.429002+0000","last_undegraded":"2023-08-11T14:03:04.429002+0000","last_fullsized":"2023-08-11T14:03:04.429002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:22.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523029+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:03:04.523029+0000","last_peered":"2023-08-11T14:03:04.523029+0000","last_clean":"2023-08-11T14:03:04.523029+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:03:04.523029+0000","last_undegraded":"2023-08-11T14:03:04.523029+0000","last_fullsized":"2023-08-11T14:03:04.523029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:41.402588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345435+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:03:04.345435+0000","last_peered":"2023-08-11T14:03:04.345435+0000","last_clean":"2023-08-11T14:03:04.345435+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:03:04.345435+0000","last_undegraded":"2023-08-11T14:03:04.345435+0000","last_fullsized":"2023-08-11T14:03:04.345435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:35:33.118766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341241+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:03:04.341241+0000","last_peered":"2023-08-11T14:03:04.341241+0000","last_clean":"2023-08-11T14:03:04.341241+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:03:04.341241+0000","last_undegraded":"2023-08-11T14:03:04.341241+0000","last_fullsized":"2023-08-11T14:03:04.341241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:48.999413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429059+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:03:04.429059+0000","last_peered":"2023-08-11T14:03:04.429059+0000","last_clean":"2023-08-11T14:03:04.429059+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:03:04.429059+0000","last_undegraded":"2023-08-11T14:03:04.429059+0000","last_fullsized":"2023-08-11T14:03:04.429059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:25.445721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.494522+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:03:53.494522+0000","last_peered":"2023-08-11T14:03:53.494522+0000","last_clean":"2023-08-11T14:03:53.494522+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:03:53.494522+0000","last_undegraded":"2023-08-11T14:03:53.494522+0000","last_fullsized":"2023-08-11T14:03:53.494522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:08.555355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523699+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:03:04.523699+0000","last_peered":"2023-08-11T14:03:04.523699+0000","last_clean":"2023-08-11T14:03:04.523699+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:03:04.523699+0000","last_undegraded":"2023-08-11T14:03:04.523699+0000","last_fullsized":"2023-08-11T14:03:04.523699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:27.682173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340632+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:03:04.340632+0000","last_peered":"2023-08-11T14:03:04.340632+0000","last_clean":"2023-08-11T14:03:04.340632+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:03:04.340632+0000","last_undegraded":"2023-08-11T14:03:04.340632+0000","last_fullsized":"2023-08-11T14:03:04.340632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:07.774291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427459+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:03:04.427459+0000","last_peered":"2023-08-11T14:03:04.427459+0000","last_clean":"2023-08-11T14:03:04.427459+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:03:04.427459+0000","last_undegraded":"2023-08-11T14:03:04.427459+0000","last_fullsized":"2023-08-11T14:03:04.427459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:37.501349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523164+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:03:04.523164+0000","last_peered":"2023-08-11T14:03:04.523164+0000","last_clean":"2023-08-11T14:03:04.523164+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:03:04.523164+0000","last_undegraded":"2023-08-11T14:03:04.523164+0000","last_fullsized":"2023-08-11T14:03:04.523164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:14:29.876746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345553+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:03:04.345553+0000","last_peered":"2023-08-11T14:03:04.345553+0000","last_clean":"2023-08-11T14:03:04.345553+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:03:04.345553+0000","last_undegraded":"2023-08-11T14:03:04.345553+0000","last_fullsized":"2023-08-11T14:03:04.345553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:34.946284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346031+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:03:04.346031+0000","last_peered":"2023-08-11T14:03:04.346031+0000","last_clean":"2023-08-11T14:03:04.346031+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:03:04.346031+0000","last_undegraded":"2023-08-11T14:03:04.346031+0000","last_fullsized":"2023-08-11T14:03:04.346031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:45.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523852+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:03:04.523852+0000","last_peered":"2023-08-11T14:03:04.523852+0000","last_clean":"2023-08-11T14:03:04.523852+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:03:04.523852+0000","last_undegraded":"2023-08-11T14:03:04.523852+0000","last_fullsized":"2023-08-11T14:03:04.523852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:36:24.226941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339856+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:03:04.339856+0000","last_peered":"2023-08-11T14:03:04.339856+0000","last_clean":"2023-08-11T14:03:04.339856+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:03:04.339856+0000","last_undegraded":"2023-08-11T14:03:04.339856+0000","last_fullsized":"2023-08-11T14:03:04.339856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:30.100680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":306,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.495578+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:03:53.495578+0000","last_peered":"2023-08-11T14:03:53.495578+0000","last_clean":"2023-08-11T14:03:53.495578+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:03:53.495578+0000","last_undegraded":"2023-08-11T14:03:53.495578+0000","last_fullsized":"2023-08-11T14:03:53.495578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:31.930235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345541+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:03:04.345541+0000","last_peered":"2023-08-11T14:03:04.345541+0000","last_clean":"2023-08-11T14:03:04.345541+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:03:04.345541+0000","last_undegraded":"2023-08-11T14:03:04.345541+0000","last_fullsized":"2023-08-11T14:03:04.345541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:02:34.156179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.347052+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:03:04.347052+0000","last_peered":"2023-08-11T14:03:04.347052+0000","last_clean":"2023-08-11T14:03:04.347052+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:03:04.347052+0000","last_undegraded":"2023-08-11T14:03:04.347052+0000","last_fullsized":"2023-08-11T14:03:04.347052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:11:15.433011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342697+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:03:04.342697+0000","last_peered":"2023-08-11T14:03:04.342697+0000","last_clean":"2023-08-11T14:03:04.342697+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:03:04.342697+0000","last_undegraded":"2023-08-11T14:03:04.342697+0000","last_fullsized":"2023-08-11T14:03:04.342697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:41.784623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340323+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:03:04.340323+0000","last_peered":"2023-08-11T14:03:04.340323+0000","last_clean":"2023-08-11T14:03:04.340323+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:03:04.340323+0000","last_undegraded":"2023-08-11T14:03:04.340323+0000","last_fullsized":"2023-08-11T14:03:04.340323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:34.035219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342711+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:03:04.342711+0000","last_peered":"2023-08-11T14:03:04.342711+0000","last_clean":"2023-08-11T14:03:04.342711+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:03:04.342711+0000","last_undegraded":"2023-08-11T14:03:04.342711+0000","last_fullsized":"2023-08-11T14:03:04.342711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:29:44.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346763+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:03:04.346763+0000","last_peered":"2023-08-11T14:03:04.346763+0000","last_clean":"2023-08-11T14:03:04.346763+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:03:04.346763+0000","last_undegraded":"2023-08-11T14:03:04.346763+0000","last_fullsized":"2023-08-11T14:03:04.346763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:42.194578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427345+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:03:04.427345+0000","last_peered":"2023-08-11T14:03:04.427345+0000","last_clean":"2023-08-11T14:03:04.427345+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:03:04.427345+0000","last_undegraded":"2023-08-11T14:03:04.427345+0000","last_fullsized":"2023-08-11T14:03:04.427345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:29.378966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346097+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:03:04.346097+0000","last_peered":"2023-08-11T14:03:04.346097+0000","last_clean":"2023-08-11T14:03:04.346097+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:03:04.346097+0000","last_undegraded":"2023-08-11T14:03:04.346097+0000","last_fullsized":"2023-08-11T14:03:04.346097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:18.989452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.491776+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:03:51.491776+0000","last_peered":"2023-08-11T14:03:51.491776+0000","last_clean":"2023-08-11T14:03:51.491776+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:03:51.491776+0000","last_undegraded":"2023-08-11T14:03:51.491776+0000","last_fullsized":"2023-08-11T14:03:51.491776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:47.290694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426969+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:03:04.426969+0000","last_peered":"2023-08-11T14:03:04.426969+0000","last_clean":"2023-08-11T14:03:04.426969+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:03:04.426969+0000","last_undegraded":"2023-08-11T14:03:04.426969+0000","last_fullsized":"2023-08-11T14:03:04.426969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:46.867384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340459+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:03:04.340459+0000","last_peered":"2023-08-11T14:03:04.340459+0000","last_clean":"2023-08-11T14:03:04.340459+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:03:04.340459+0000","last_undegraded":"2023-08-11T14:03:04.340459+0000","last_fullsized":"2023-08-11T14:03:04.340459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:51.746271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.344937+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:03:04.344937+0000","last_peered":"2023-08-11T14:03:04.344937+0000","last_clean":"2023-08-11T14:03:04.344937+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:03:04.344937+0000","last_undegraded":"2023-08-11T14:03:04.344937+0000","last_fullsized":"2023-08-11T14:03:04.344937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:13.455539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341159+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:03:04.341159+0000","last_peered":"2023-08-11T14:03:04.341159+0000","last_clean":"2023-08-11T14:03:04.341159+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:03:04.341159+0000","last_undegraded":"2023-08-11T14:03:04.341159+0000","last_fullsized":"2023-08-11T14:03:04.341159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:34.163158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346090+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:03:04.346090+0000","last_peered":"2023-08-11T14:03:04.346090+0000","last_clean":"2023-08-11T14:03:04.346090+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:03:04.346090+0000","last_undegraded":"2023-08-11T14:03:04.346090+0000","last_fullsized":"2023-08-11T14:03:04.346090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:22:02.526755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.352593+0000","last_change":"2023-08-11T14:03:04.352113+0000","last_active":"2023-08-11T14:03:04.352593+0000","last_peered":"2023-08-11T14:03:04.352593+0000","last_clean":"2023-08-11T14:03:04.352593+0000","last_became_active":"2023-08-11T14:03:04.347837+0000","last_became_peered":"2023-08-11T14:03:04.347837+0000","last_unstale":"2023-08-11T14:03:04.352593+0000","last_undegraded":"2023-08-11T14:03:04.352593+0000","last_fullsized":"2023-08-11T14:03:04.352593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:47.346364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342848+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:03:04.342848+0000","last_peered":"2023-08-11T14:03:04.342848+0000","last_clean":"2023-08-11T14:03:04.342848+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:03:04.342848+0000","last_undegraded":"2023-08-11T14:03:04.342848+0000","last_fullsized":"2023-08-11T14:03:04.342848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:43.902027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345467+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:03:04.345467+0000","last_peered":"2023-08-11T14:03:04.345467+0000","last_clean":"2023-08-11T14:03:04.345467+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:03:04.345467+0000","last_undegraded":"2023-08-11T14:03:04.345467+0000","last_fullsized":"2023-08-11T14:03:04.345467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:57.057812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341336+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:03:04.341336+0000","last_peered":"2023-08-11T14:03:04.341336+0000","last_clean":"2023-08-11T14:03:04.341336+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:03:04.341336+0000","last_undegraded":"2023-08-11T14:03:04.341336+0000","last_fullsized":"2023-08-11T14:03:04.341336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:09.795680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420336+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:03:04.420336+0000","last_peered":"2023-08-11T14:03:04.420336+0000","last_clean":"2023-08-11T14:03:04.420336+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:03:04.420336+0000","last_undegraded":"2023-08-11T14:03:04.420336+0000","last_fullsized":"2023-08-11T14:03:04.420336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:26:04.032679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339836+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:03:04.339836+0000","last_peered":"2023-08-11T14:03:04.339836+0000","last_clean":"2023-08-11T14:03:04.339836+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:03:04.339836+0000","last_undegraded":"2023-08-11T14:03:04.339836+0000","last_fullsized":"2023-08-11T14:03:04.339836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:42:17.950224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":146,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:48.487314+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:03:48.487314+0000","last_peered":"2023-08-11T14:03:48.487314+0000","last_clean":"2023-08-11T14:03:48.487314+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:03:48.487314+0000","last_undegraded":"2023-08-11T14:03:48.487314+0000","last_fullsized":"2023-08-11T14:03:48.487314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:25.158801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420203+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:03:04.420203+0000","last_peered":"2023-08-11T14:03:04.420203+0000","last_clean":"2023-08-11T14:03:04.420203+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:03:04.420203+0000","last_undegraded":"2023-08-11T14:03:04.420203+0000","last_fullsized":"2023-08-11T14:03:04.420203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:57.775826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341077+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:03:04.341077+0000","last_peered":"2023-08-11T14:03:04.341077+0000","last_clean":"2023-08-11T14:03:04.341077+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:03:04.341077+0000","last_undegraded":"2023-08-11T14:03:04.341077+0000","last_fullsized":"2023-08-11T14:03:04.341077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:59.875215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345872+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:03:04.345872+0000","last_peered":"2023-08-11T14:03:04.345872+0000","last_clean":"2023-08-11T14:03:04.345872+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:03:04.345872+0000","last_undegraded":"2023-08-11T14:03:04.345872+0000","last_fullsized":"2023-08-11T14:03:04.345872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:35:57.958298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:52.492527+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:03:52.492527+0000","last_peered":"2023-08-11T14:03:52.492527+0000","last_clean":"2023-08-11T14:03:52.492527+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:03:52.492527+0000","last_undegraded":"2023-08-11T14:03:52.492527+0000","last_fullsized":"2023-08-11T14:03:52.492527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:28.567968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427398+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:03:04.427398+0000","last_peered":"2023-08-11T14:03:04.427398+0000","last_clean":"2023-08-11T14:03:04.427398+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:03:04.427398+0000","last_undegraded":"2023-08-11T14:03:04.427398+0000","last_fullsized":"2023-08-11T14:03:04.427398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:14:04.821908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429434+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:03:04.429434+0000","last_peered":"2023-08-11T14:03:04.429434+0000","last_clean":"2023-08-11T14:03:04.429434+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:03:04.429434+0000","last_undegraded":"2023-08-11T14:03:04.429434+0000","last_fullsized":"2023-08-11T14:03:04.429434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:51:44.310088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345441+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:03:04.345441+0000","last_peered":"2023-08-11T14:03:04.345441+0000","last_clean":"2023-08-11T14:03:04.345441+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:03:04.345441+0000","last_undegraded":"2023-08-11T14:03:04.345441+0000","last_fullsized":"2023-08-11T14:03:04.345441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:06.862831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426824+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:03:04.426824+0000","last_peered":"2023-08-11T14:03:04.426824+0000","last_clean":"2023-08-11T14:03:04.426824+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:03:04.426824+0000","last_undegraded":"2023-08-11T14:03:04.426824+0000","last_fullsized":"2023-08-11T14:03:04.426824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:07.631487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346982+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:03:04.346982+0000","last_peered":"2023-08-11T14:03:04.346982+0000","last_clean":"2023-08-11T14:03:04.346982+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:03:04.346982+0000","last_undegraded":"2023-08-11T14:03:04.346982+0000","last_fullsized":"2023-08-11T14:03:04.346982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:24.515141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340804+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:03:04.340804+0000","last_peered":"2023-08-11T14:03:04.340804+0000","last_clean":"2023-08-11T14:03:04.340804+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:03:04.340804+0000","last_undegraded":"2023-08-11T14:03:04.340804+0000","last_fullsized":"2023-08-11T14:03:04.340804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:23.071563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426987+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:03:04.426987+0000","last_peered":"2023-08-11T14:03:04.426987+0000","last_clean":"2023-08-11T14:03:04.426987+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:03:04.426987+0000","last_undegraded":"2023-08-11T14:03:04.426987+0000","last_fullsized":"2023-08-11T14:03:04.426987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:54:16.816623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342995+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:03:04.342995+0000","last_peered":"2023-08-11T14:03:04.342995+0000","last_clean":"2023-08-11T14:03:04.342995+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:03:04.342995+0000","last_undegraded":"2023-08-11T14:03:04.342995+0000","last_fullsized":"2023-08-11T14:03:04.342995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:05.194595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340208+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:03:04.340208+0000","last_peered":"2023-08-11T14:03:04.340208+0000","last_clean":"2023-08-11T14:03:04.340208+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:03:04.340208+0000","last_undegraded":"2023-08-11T14:03:04.340208+0000","last_fullsized":"2023-08-11T14:03:04.340208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:36.744960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.428991+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:03:04.428991+0000","last_peered":"2023-08-11T14:03:04.428991+0000","last_clean":"2023-08-11T14:03:04.428991+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:03:04.428991+0000","last_undegraded":"2023-08-11T14:03:04.428991+0000","last_fullsized":"2023-08-11T14:03:04.428991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:06:57.242042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:43.252925+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:03:43.252925+0000","last_peered":"2023-08-11T14:03:43.252925+0000","last_clean":"2023-08-11T14:03:43.252925+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:03:43.252925+0000","last_undegraded":"2023-08-11T14:03:43.252925+0000","last_fullsized":"2023-08-11T14:03:43.252925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:19:42.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429596+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:03:04.429596+0000","last_peered":"2023-08-11T14:03:04.429596+0000","last_clean":"2023-08-11T14:03:04.429596+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:03:04.429596+0000","last_undegraded":"2023-08-11T14:03:04.429596+0000","last_fullsized":"2023-08-11T14:03:04.429596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:21:59.987965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340537+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:03:04.340537+0000","last_peered":"2023-08-11T14:03:04.340537+0000","last_clean":"2023-08-11T14:03:04.340537+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:03:04.340537+0000","last_undegraded":"2023-08-11T14:03:04.340537+0000","last_fullsized":"2023-08-11T14:03:04.340537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:10:35.951459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346004+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:03:04.346004+0000","last_peered":"2023-08-11T14:03:04.346004+0000","last_clean":"2023-08-11T14:03:04.346004+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:03:04.346004+0000","last_undegraded":"2023-08-11T14:03:04.346004+0000","last_fullsized":"2023-08-11T14:03:04.346004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:55.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523190+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:03:04.523190+0000","last_peered":"2023-08-11T14:03:04.523190+0000","last_clean":"2023-08-11T14:03:04.523190+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:03:04.523190+0000","last_undegraded":"2023-08-11T14:03:04.523190+0000","last_fullsized":"2023-08-11T14:03:04.523190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:13:04.607004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345962+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:03:04.345962+0000","last_peered":"2023-08-11T14:03:04.345962+0000","last_clean":"2023-08-11T14:03:04.345962+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:03:04.345962+0000","last_undegraded":"2023-08-11T14:03:04.345962+0000","last_fullsized":"2023-08-11T14:03:04.345962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:46.950942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523374+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:03:04.523374+0000","last_peered":"2023-08-11T14:03:04.523374+0000","last_clean":"2023-08-11T14:03:04.523374+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:03:04.523374+0000","last_undegraded":"2023-08-11T14:03:04.523374+0000","last_fullsized":"2023-08-11T14:03:04.523374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:40:06.480953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345005+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:03:04.345005+0000","last_peered":"2023-08-11T14:03:04.345005+0000","last_clean":"2023-08-11T14:03:04.345005+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:03:04.345005+0000","last_undegraded":"2023-08-11T14:03:04.345005+0000","last_fullsized":"2023-08-11T14:03:04.345005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:58.380684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341596+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:03:04.341596+0000","last_peered":"2023-08-11T14:03:04.341596+0000","last_clean":"2023-08-11T14:03:04.341596+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:03:04.341596+0000","last_undegraded":"2023-08-11T14:03:04.341596+0000","last_fullsized":"2023-08-11T14:03:04.341596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:48.675324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429201+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:03:04.429201+0000","last_peered":"2023-08-11T14:03:04.429201+0000","last_clean":"2023-08-11T14:03:04.429201+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:03:04.429201+0000","last_undegraded":"2023-08-11T14:03:04.429201+0000","last_fullsized":"2023-08-11T14:03:04.429201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:03.363760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342610+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:03:04.342610+0000","last_peered":"2023-08-11T14:03:04.342610+0000","last_clean":"2023-08-11T14:03:04.342610+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:03:04.342610+0000","last_undegraded":"2023-08-11T14:03:04.342610+0000","last_fullsized":"2023-08-11T14:03:04.342610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:33:37.384946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:08.705807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340884+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:03:04.340884+0000","last_peered":"2023-08-11T14:03:04.340884+0000","last_clean":"2023-08-11T14:03:04.340884+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:03:04.340884+0000","last_undegraded":"2023-08-11T14:03:04.340884+0000","last_fullsized":"2023-08-11T14:03:04.340884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:52.040366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429419+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:03:04.429419+0000","last_peered":"2023-08-11T14:03:04.429419+0000","last_clean":"2023-08-11T14:03:04.429419+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:03:04.429419+0000","last_undegraded":"2023-08-11T14:03:04.429419+0000","last_fullsized":"2023-08-11T14:03:04.429419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:31.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345789+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:03:04.345789+0000","last_peered":"2023-08-11T14:03:04.345789+0000","last_clean":"2023-08-11T14:03:04.345789+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:03:04.345789+0000","last_undegraded":"2023-08-11T14:03:04.345789+0000","last_fullsized":"2023-08-11T14:03:04.345789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:14.187908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345206+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:03:04.345206+0000","last_peered":"2023-08-11T14:03:04.345206+0000","last_clean":"2023-08-11T14:03:04.345206+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:03:04.345206+0000","last_undegraded":"2023-08-11T14:03:04.345206+0000","last_fullsized":"2023-08-11T14:03:04.345206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:21:37.922008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427322+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:03:04.427322+0000","last_peered":"2023-08-11T14:03:04.427322+0000","last_clean":"2023-08-11T14:03:04.427322+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:03:04.427322+0000","last_undegraded":"2023-08-11T14:03:04.427322+0000","last_fullsized":"2023-08-11T14:03:04.427322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:55.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523362+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:03:04.523362+0000","last_peered":"2023-08-11T14:03:04.523362+0000","last_clean":"2023-08-11T14:03:04.523362+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:03:04.523362+0000","last_undegraded":"2023-08-11T14:03:04.523362+0000","last_fullsized":"2023-08-11T14:03:04.523362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:36.812617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:49.502291+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:03:49.502291+0000","last_peered":"2023-08-11T14:03:49.502291+0000","last_clean":"2023-08-11T14:03:49.502291+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:03:49.502291+0000","last_undegraded":"2023-08-11T14:03:49.502291+0000","last_fullsized":"2023-08-11T14:03:49.502291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:16:25.101731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427708+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:03:04.427708+0000","last_peered":"2023-08-11T14:03:04.427708+0000","last_clean":"2023-08-11T14:03:04.427708+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:03:04.427708+0000","last_undegraded":"2023-08-11T14:03:04.427708+0000","last_fullsized":"2023-08-11T14:03:04.427708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:45:18.108583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522691+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:03:04.522691+0000","last_peered":"2023-08-11T14:03:04.522691+0000","last_clean":"2023-08-11T14:03:04.522691+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:03:04.522691+0000","last_undegraded":"2023-08-11T14:03:04.522691+0000","last_fullsized":"2023-08-11T14:03:04.522691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:56.757871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346920+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:03:04.346920+0000","last_peered":"2023-08-11T14:03:04.346920+0000","last_clean":"2023-08-11T14:03:04.346920+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:03:04.346920+0000","last_undegraded":"2023-08-11T14:03:04.346920+0000","last_fullsized":"2023-08-11T14:03:04.346920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:44.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427304+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:03:04.427304+0000","last_peered":"2023-08-11T14:03:04.427304+0000","last_clean":"2023-08-11T14:03:04.427304+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:03:04.427304+0000","last_undegraded":"2023-08-11T14:03:04.427304+0000","last_fullsized":"2023-08-11T14:03:04.427304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:53:39.819984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345073+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:03:04.345073+0000","last_peered":"2023-08-11T14:03:04.345073+0000","last_clean":"2023-08-11T14:03:04.345073+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:03:04.345073+0000","last_undegraded":"2023-08-11T14:03:04.345073+0000","last_fullsized":"2023-08-11T14:03:04.345073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:05.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340636+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:03:04.340636+0000","last_peered":"2023-08-11T14:03:04.340636+0000","last_clean":"2023-08-11T14:03:04.340636+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:03:04.340636+0000","last_undegraded":"2023-08-11T14:03:04.340636+0000","last_fullsized":"2023-08-11T14:03:04.340636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:43.294958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427646+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:03:04.427646+0000","last_peered":"2023-08-11T14:03:04.427646+0000","last_clean":"2023-08-11T14:03:04.427646+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:03:04.427646+0000","last_undegraded":"2023-08-11T14:03:04.427646+0000","last_fullsized":"2023-08-11T14:03:04.427646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:53.158734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342712+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:03:04.342712+0000","last_peered":"2023-08-11T14:03:04.342712+0000","last_clean":"2023-08-11T14:03:04.342712+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:03:04.342712+0000","last_undegraded":"2023-08-11T14:03:04.342712+0000","last_fullsized":"2023-08-11T14:03:04.342712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:26.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340809+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:03:04.340809+0000","last_peered":"2023-08-11T14:03:04.340809+0000","last_clean":"2023-08-11T14:03:04.340809+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:03:04.340809+0000","last_undegraded":"2023-08-11T14:03:04.340809+0000","last_fullsized":"2023-08-11T14:03:04.340809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:39.383017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340630+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:03:04.340630+0000","last_peered":"2023-08-11T14:03:04.340630+0000","last_clean":"2023-08-11T14:03:04.340630+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:03:04.340630+0000","last_undegraded":"2023-08-11T14:03:04.340630+0000","last_fullsized":"2023-08-11T14:03:04.340630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:09:51.511408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522979+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:03:04.522979+0000","last_peered":"2023-08-11T14:03:04.522979+0000","last_clean":"2023-08-11T14:03:04.522979+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:03:04.522979+0000","last_undegraded":"2023-08-11T14:03:04.522979+0000","last_fullsized":"2023-08-11T14:03:04.522979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:38:58.624580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340049+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:03:04.340049+0000","last_peered":"2023-08-11T14:03:04.340049+0000","last_clean":"2023-08-11T14:03:04.340049+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:03:04.340049+0000","last_undegraded":"2023-08-11T14:03:04.340049+0000","last_fullsized":"2023-08-11T14:03:04.340049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:10:16.822501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:34.479972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341526+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:03:04.341526+0000","last_peered":"2023-08-11T14:03:04.341526+0000","last_clean":"2023-08-11T14:03:04.341526+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:03:04.341526+0000","last_undegraded":"2023-08-11T14:03:04.341526+0000","last_fullsized":"2023-08-11T14:03:04.341526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.188787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429642+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:03:04.429642+0000","last_peered":"2023-08-11T14:03:04.429642+0000","last_clean":"2023-08-11T14:03:04.429642+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:03:04.429642+0000","last_undegraded":"2023-08-11T14:03:04.429642+0000","last_fullsized":"2023-08-11T14:03:04.429642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:01:08.200835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345624+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:03:04.345624+0000","last_peered":"2023-08-11T14:03:04.345624+0000","last_clean":"2023-08-11T14:03:04.345624+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:03:04.345624+0000","last_undegraded":"2023-08-11T14:03:04.345624+0000","last_fullsized":"2023-08-11T14:03:04.345624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:08.040487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429411+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:03:04.429411+0000","last_peered":"2023-08-11T14:03:04.429411+0000","last_clean":"2023-08-11T14:03:04.429411+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:03:04.429411+0000","last_undegraded":"2023-08-11T14:03:04.429411+0000","last_fullsized":"2023-08-11T14:03:04.429411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:43:43.729999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523915+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:03:04.523915+0000","last_peered":"2023-08-11T14:03:04.523915+0000","last_clean":"2023-08-11T14:03:04.523915+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:03:04.523915+0000","last_undegraded":"2023-08-11T14:03:04.523915+0000","last_fullsized":"2023-08-11T14:03:04.523915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:25:07.770441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340919+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:03:04.340919+0000","last_peered":"2023-08-11T14:03:04.340919+0000","last_clean":"2023-08-11T14:03:04.340919+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:03:04.340919+0000","last_undegraded":"2023-08-11T14:03:04.340919+0000","last_fullsized":"2023-08-11T14:03:04.340919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:14.465240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342974+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:03:04.342974+0000","last_peered":"2023-08-11T14:03:04.342974+0000","last_clean":"2023-08-11T14:03:04.342974+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:03:04.342974+0000","last_undegraded":"2023-08-11T14:03:04.342974+0000","last_fullsized":"2023-08-11T14:03:04.342974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:01.425835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73338,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1155,"num_read_kb":664,"num_write":250,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":214635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":509222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.56999999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.88,"15min":0.88},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.296,"15min":1.296},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":1.208,"15min":1.208},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.048,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.876,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84599999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80600000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.006}]},{"osd":9,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0109999999999999,"5min":1.27,"15min":1.27},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0720000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97399999999999998,"5min":1.139,"15min":1.139},"last":0.93500000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678060,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1162864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.278,"5min":1.526,"15min":1.526},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.117,"5min":1.516,"15min":1.516},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92400000000000004,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.70399999999999996},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":0.33700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.378,"15min":0.378},"max":{"1min":0.97799999999999998,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.438},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.085,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77800000000000002,"15min":0.81100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.012,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.57199999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":1.605,"15min":1.605},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.55100000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98499999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84499999999999997,"15min":0.88400000000000001},"min":{"1min":0.52600000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.92100000000000004,"15min":0.90300000000000002},"min":{"1min":0.67000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95199999999999996,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Aug 11 14:03:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.65700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":1.234,"15min":1.234},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.748,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.129,"15min":1.129},"last":0.54300000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":513158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.58999999999999997,"15min":0.63200000000000001},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71999999999999997,"15min":0.748},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":1,"15min":1},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.53700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69799999999999995,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89800000000000002,"5min":1.123,"15min":1.123},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.74199999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92700000000000005,"5min":1.165,"15min":1.165},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.85599999999999998,"15min":0.90000000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.2149999999999999,"5min":3.2149999999999999,"15min":3.2149999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.86499999999999999,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":0.95299999999999996},"min":{"1min":0.60299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1040000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.93999999999999995,"15min":1.018},"min":{"1min":0.505,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.94799999999999995,"15min":1.0229999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0309999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.878}]},{"osd":8,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.079,"5min":1.484,"15min":1.484},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0580000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.629}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.96799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":0.84099999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.88600000000000001,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.86799999999999999,"15min":0.84099999999999997},"min":{"1min":0.72299999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.68799999999999994}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":509418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.625,"15min":0.623},"min":{"1min":0.47099999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":5.3159999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.53200000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.56399999999999995,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.626}]},{"osd":2,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72799999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.40899999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.39900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":1.022,"15min":1.022},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.56399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.501,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.59699999999999998},"min":{"1min":0.46400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.91000000000000003,"5min":1.101,"15min":1.101},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.79300000000000004,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.65300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.042,"15min":1.042},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.67300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98599999999999999,"5min":1.083,"15min":1.083},"last":0.877},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96999999999999997,"5min":1.29,"15min":1.29},"last":0.66600000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0509999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.82799999999999996,"15min":0.79800000000000004},"min":{"1min":0.79700000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.72799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.79800000000000004,"15min":0.754},"min":{"1min":0.78900000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84799999999999998,"15min":0.84199999999999997},"min":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.92400000000000004},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.02,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71699999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1105804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56799999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.34300000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66800000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.64400000000000002,"15min":0.59399999999999997},"min":{"1min":0.47699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.6130000000000004,"5min":4.6130000000000004,"15min":4.6130000000000004},"last":0.504},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.753,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.377},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83899999999999997,"5min":1.034,"15min":1.034},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.82799999999999996,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.798,"5min":6.798,"15min":6.798},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0129999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.66000000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.96899999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.6050000000000004,"5min":5.6050000000000004,"15min":5.6050000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.86099999999999999,"15min":0.83899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97399999999999998,"5min":1.095,"15min":1.095},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.60099999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88900000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66700000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85599999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.67600000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.95599999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77600000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":504566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.73399999999999999,"15min":0.70399999999999996},"min":{"1min":0.52300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.52800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.131,"15min":1.131},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.433,"15min":0.433},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.133,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.82299999999999995,"15min":0.80700000000000005},"min":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Aug 11 14:03:24 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81399999999999995,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.70299999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1339999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92700000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84899999999999998,"15min":0.88200000000000001},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.9220000000000002,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.55700000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9420000000000002,"5min":3.9420000000000002,"15min":3.9420000000000002},"last":0.66300000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:05 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.93999999999999995,"15min":0.95099999999999996},"min":{"1min":0.63900000000000001,"5min":0.5,"15min":0.5},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86899999999999999,"15min":0.85299999999999998},"min":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.7909999999999999,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92100000000000004,"15min":0.92600000000000005},"min":{"1min":0.77600000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1519999999999999,"5min":1.363,"15min":1.363},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Aug 11 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91400000000000003,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.98,"5min":3.98,"15min":3.98},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.93799999999999994,"15min":0.92000000000000004},"min":{"1min":0.72399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":571708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.054},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82299999999999995,"5min":1.234,"15min":1.234},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.63400000000000001,"15min":0.54800000000000004},"min":{"1min":0.48299999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.95899999999999996,"5min":0.995,"15min":0.995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.55900000000000005},"min":{"1min":0.50700000000000001,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:02:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.51300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.60599999999999998},"min":{"1min":0.55800000000000005,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.625}]},{"osd":4,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48999999999999999,"15min":0.41799999999999998},"min":{"1min":0.36299999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80300000000000005,"5min":1.119,"15min":1.119},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.47099999999999997,"15min":0.43099999999999999},"min":{"1min":0.39500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.65600000000000003,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96899999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69299999999999995,"15min":0.59899999999999998},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94399999999999995,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.54300000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99299999999999999,"5min":3.153,"15min":3.153},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.69299999999999995,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0349999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63400000000000001,"15min":0.55600000000000005},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.089,"5min":1.159,"15min":1.159},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.67500000000000004,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.115,"5min":1.345,"15min":1.345},"last":0.61299999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76700000000000002,"15min":0.70899999999999996},"min":{"1min":0.73599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1799999999999999,"5min":1.302,"15min":1.302},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78000000000000003,"15min":0.71599999999999997},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.131,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:03:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.641,"5min":3.194,"15min":3.194},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.85399999999999998,"15min":0.83999999999999997},"min":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1839999999999999,"5min":1.393,"15min":1.393},"last":0.751}]},{"osd":10,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.59099999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.091,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.073,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.151,"5min":1.21,"15min":1.21},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":506022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64500000000000002,"15min":0.59099999999999997},"min":{"1min":0.51900000000000002,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.002,"5min":0.70599999999999996,"15min":0.621},"min":{"1min":0.495,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.35399999999999998},"max":{"1min":0.93700000000000006,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":1.724,"5min":6.2910000000000004,"15min":6.2910000000000004},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67300000000000004,"15min":0.68899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0049999999999999,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:03:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.72999999999999998,"15min":0.627},"min":{"1min":0.496,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.1619999999999999,"5min":6.1619999999999999,"15min":6.1619999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.68999999999999995,"15min":0.60599999999999998},"min":{"1min":0.54900000000000004,"5min":0.317,"15min":0.317},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79100000000000004,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.8029999999999999,"5min":6.25,"15min":6.25},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0169999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.52300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79000000000000004,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":0.755}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.93300000000000005,"15min":0.94799999999999995},"min":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.091,"5min":3.254,"15min":3.254},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.069,"5min":1.234,"15min":1.234},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.63900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0669999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91100000000000003,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.093,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.98199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":0.92400000000000004,"15min":0.90500000000000003},"min":{"1min":0.625,"5min":0.376,"15min":0.376},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.93999999999999995,"15min":0.88600000000000001},"min":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7629999999999999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92500000000000004,"15min":0.90000000000000002},"min":{"1min":0.69699999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81799999999999995}]},{"osd":11,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.77200000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.153,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.90500000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.177,"5min":6.3739999999999997,"15min":6.3739999999999997},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":566754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.60799999999999998,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.0780000000000003,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.82399999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.30099999999999999},"max":{"1min":0.92300000000000004,"5min":1.036,"15min":1.036},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60599999999999998,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.92500000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81899999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.624,"15min":0.57399999999999995},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.86299999999999999,"5min":0.876,"15min":0.876},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.56799999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.41999999999999998},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":3.218,"15min":3.218},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.44},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.72199999999999998,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.436,"15min":0.436},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":0.90400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.80000000000000004,"15min":0.79400000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Aug 11 14:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.58699999999999997},"min":{"1min":0.48799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.59799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.006,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.63}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.67900000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.73199999999999998},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1120000000000001,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":0.94999999999999996,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.745},"min":{"1min":0.60399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.63900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0680000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":504274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.42199999999999999},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.68600000000000005,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.423,"5min":6.423,"15min":6.423},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.61299999999999999},"min":{"1min":0.57199999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":2.548,"15min":2.548},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.72499999999999998,"15min":0.65500000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":6.5010000000000003,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69299999999999995,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2849999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.2010000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.751,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.89400000000000002,"15min":0.83399999999999996},"min":{"1min":0.65900000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":8.6370000000000005,"5min":8.6370000000000005,"15min":8.6370000000000005},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96199999999999997,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.92800000000000005},"min":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.92200000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.179,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1879999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.92200000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.254,"5min":1.379,"15min":1.379},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.169,"5min":0.89200000000000002,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":6.4610000000000003,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.98799999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.33000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61199999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.38600000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65800000000000003,"5min":0.875,"15min":0.875},"last":0.376}]},{"osd":2,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64900000000000002,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.436},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.496}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.98899999999999999,"5min":1.02,"15min":1.02},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.65500000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0409999999999999,"5min":1.075,"15min":1.075},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.64100000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.67500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78100000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1101128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":1.46,"15min":1.46},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.38,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.52800000000000002},"min":{"1min":0.46899999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82199999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.56100000000000005,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.39900000000000002},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.59899999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.442,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.409,"15min":1.409},"last":0.626},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63200000000000001,"15min":0.57699999999999996},"min":{"1min":0.5,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59599999999999997},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":1.393,"15min":1.393},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89400000000000002,"5min":1.484,"15min":1.484},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.878},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.82599999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98999999999999999,"5min":1.181,"15min":1.181},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.78500000000000003,"15min":0.76100000000000001},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96899999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:03:54.895 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-08-11T14:03:54.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-11T14:03:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 -- 172.21.15.3:0/704972716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec101b10 msgr2=0x7fe6ec101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/704972716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec101b10 0x7fe6ec101f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009920 tx=0x7fe6dc009c30 comp rx=0 tx=0).stop 2023-08-11T14:03:55.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 -- 172.21.15.3:0/704972716 shutdown_connections 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/704972716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec1024b0 0x7fe6ec108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/704972716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec100920 0x7fe6ec100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/704972716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec101b10 0x7fe6ec101f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.137+0000 7fe6f45ff700 1 -- 172.21.15.3:0/704972716 >> 172.21.15.3:0/704972716 conn(0x7fe6ec0fbe50 msgr2=0x7fe6ec0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.138+0000 7fe6f45ff700 1 -- 172.21.15.3:0/704972716 shutdown_connections 2023-08-11T14:03:55.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.138+0000 7fe6f45ff700 1 -- 172.21.15.3:0/704972716 wait complete. 2023-08-11T14:03:55.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.138+0000 7fe6f45ff700 1 Processor -- start 2023-08-11T14:03:55.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 -- start start 2023-08-11T14:03:55.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec101b10 0x7fe6ec196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec1024b0 0x7fe6ec19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19e210 con 0x7fe6ec1024b0 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19e350 con 0x7fe6ec100920 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f45ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec19e4e0 con 0x7fe6ec101b10 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f239b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f239b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48804/0 (socket says 172.21.15.3:48804) 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f239b700 1 -- 172.21.15.3:0/979858852 learned_addr learned my addr 172.21.15.3:0/979858852 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f2b9c700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec1024b0 0x7fe6ec19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.139+0000 7fe6f1b9a700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec101b10 0x7fe6ec196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 -- 172.21.15.3:0/979858852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec101b10 msgr2=0x7fe6ec196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec101b10 0x7fe6ec196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 -- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec1024b0 msgr2=0x7fe6ec19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec1024b0 0x7fe6ec19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 -- 172.21.15.3:0/979858852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80095f0 con 0x7fe6ec100920 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f239b700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800ebf0 tx=0x7fe6e800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8009a10 con 0x7fe6ec100920 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e800cc90 con 0x7fe6ec100920 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8018750 con 0x7fe6ec100920 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc009520 con 0x7fe6ec100920 2023-08-11T14:03:55.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.140+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec19eaf0 con 0x7fe6ec100920 2023-08-11T14:03:55.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.141+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec1078a0 con 0x7fe6ec100920 2023-08-11T14:03:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.143+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe6e80188b0 con 0x7fe6ec100920 2023-08-11T14:03:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.145+0000 7fe6e37fe700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 0x7fe6d80a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.145+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe6e8014070 con 0x7fe6ec100920 2023-08-11T14:03:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.145+0000 7fe6f1b9a700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 0x7fe6d80a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.145+0000 7fe6f1b9a700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 0x7fe6d80a24e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00fa40 tx=0x7fe6dc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:55.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.146+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe6e8095670 con 0x7fe6ec100920 2023-08-11T14:03:55.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.376+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 --> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6ec02cdf0 con 0x7fe6d80a0020 2023-08-11T14:03:55.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.389+0000 7fe6e37fe700 1 -- 172.21.15.3:0/979858852 <== mgr.14150 v2:172.21.15.3:6800/2363526909 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427086 (secure 0 0 0) 0x7fe6ec02cdf0 con 0x7fe6d80a0020 2023-08-11T14:03:55.395 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 msgr2=0x7fe6d80a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 0x7fe6d80a24e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00fa40 tx=0x7fe6dc019040 comp rx=0 tx=0).stop 2023-08-11T14:03:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 msgr2=0x7fe6ec196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800ebf0 tx=0x7fe6e800ef00 comp rx=0 tx=0).stop 2023-08-11T14:03:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 shutdown_connections 2023-08-11T14:03:55.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6ec1024b0 0x7fe6ec19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe6d80a0020 0x7fe6d80a24e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6ec100920 0x7fe6ec196690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 --2- 172.21.15.3:0/979858852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6ec101b10 0x7fe6ec196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 >> 172.21.15.3:0/979858852 conn(0x7fe6ec0fbe50 msgr2=0x7fe6ec0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 shutdown_connections 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.394+0000 7fe6f45ff700 1 -- 172.21.15.3:0/979858852 wait complete. 2023-08-11T14:03:55.399 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-08-11T14:03:55.416 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2023-08-11T14:03:54.201169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663952,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1281,"num_read_kb":773,"num_write":433,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338564,"kb_used_data":14332,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124603260,"statfs":{"total":1151940427776,"available":1151593738240,"internally_reserved":0,"allocated":14675968,"data_stored":8064089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5695,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":82,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343170+0000","last_change":"2023-08-11T14:01:33.301421+0000","last_active":"2023-08-11T14:03:04.343170+0000","last_peered":"2023-08-11T14:03:04.343170+0000","last_clean":"2023-08-11T14:03:04.343170+0000","last_became_active":"2023-08-11T14:01:33.300411+0000","last_became_peered":"2023-08-11T14:01:33.300411+0000","last_unstale":"2023-08-11T14:03:04.343170+0000","last_undegraded":"2023-08-11T14:03:04.343170+0000","last_fullsized":"2023-08-11T14:03:04.343170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:14:30.456837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523051+0000","last_change":"2023-08-11T14:01:33.312158+0000","last_active":"2023-08-11T14:03:04.523051+0000","last_peered":"2023-08-11T14:03:04.523051+0000","last_clean":"2023-08-11T14:03:04.523051+0000","last_became_active":"2023-08-11T14:01:33.311853+0000","last_became_peered":"2023-08-11T14:01:33.311853+0000","last_unstale":"2023-08-11T14:03:04.523051+0000","last_undegraded":"2023-08-11T14:03:04.523051+0000","last_fullsized":"2023-08-11T14:03:04.523051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:10:59.858967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522981+0000","last_change":"2023-08-11T14:01:33.307464+0000","last_active":"2023-08-11T14:03:04.522981+0000","last_peered":"2023-08-11T14:03:04.522981+0000","last_clean":"2023-08-11T14:03:04.522981+0000","last_became_active":"2023-08-11T14:01:33.307141+0000","last_became_peered":"2023-08-11T14:01:33.307141+0000","last_unstale":"2023-08-11T14:03:04.522981+0000","last_undegraded":"2023-08-11T14:03:04.522981+0000","last_fullsized":"2023-08-11T14:03:04.522981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:29:37.543189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346016+0000","last_change":"2023-08-11T14:01:33.303707+0000","last_active":"2023-08-11T14:03:04.346016+0000","last_peered":"2023-08-11T14:03:04.346016+0000","last_clean":"2023-08-11T14:03:04.346016+0000","last_became_active":"2023-08-11T14:01:33.303406+0000","last_became_peered":"2023-08-11T14:01:33.303406+0000","last_unstale":"2023-08-11T14:03:04.346016+0000","last_undegraded":"2023-08-11T14:03:04.346016+0000","last_fullsized":"2023-08-11T14:03:04.346016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:26:42.564980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345098+0000","last_change":"2023-08-11T14:01:33.311967+0000","last_active":"2023-08-11T14:03:04.345098+0000","last_peered":"2023-08-11T14:03:04.345098+0000","last_clean":"2023-08-11T14:03:04.345098+0000","last_became_active":"2023-08-11T14:01:33.311659+0000","last_became_peered":"2023-08-11T14:01:33.311659+0000","last_unstale":"2023-08-11T14:03:04.345098+0000","last_undegraded":"2023-08-11T14:03:04.345098+0000","last_fullsized":"2023-08-11T14:03:04.345098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:37:03.679224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419950+0000","last_change":"2023-08-11T14:01:33.303041+0000","last_active":"2023-08-11T14:03:04.419950+0000","last_peered":"2023-08-11T14:03:04.419950+0000","last_clean":"2023-08-11T14:03:04.419950+0000","last_became_active":"2023-08-11T14:01:33.302692+0000","last_became_peered":"2023-08-11T14:01:33.302692+0000","last_unstale":"2023-08-11T14:03:04.419950+0000","last_undegraded":"2023-08-11T14:03:04.419950+0000","last_fullsized":"2023-08-11T14:03:04.419950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:51:16.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426874+0000","last_change":"2023-08-11T14:01:33.304901+0000","last_active":"2023-08-11T14:03:04.426874+0000","last_peered":"2023-08-11T14:03:04.426874+0000","last_clean":"2023-08-11T14:03:04.426874+0000","last_became_active":"2023-08-11T14:01:33.304600+0000","last_became_peered":"2023-08-11T14:01:33.304600+0000","last_unstale":"2023-08-11T14:03:04.426874+0000","last_undegraded":"2023-08-11T14:03:04.426874+0000","last_fullsized":"2023-08-11T14:03:04.426874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:30:25.534139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340357+0000","last_change":"2023-08-11T14:02:03.339952+0000","last_active":"2023-08-11T14:03:04.340357+0000","last_peered":"2023-08-11T14:03:04.340357+0000","last_clean":"2023-08-11T14:03:04.340357+0000","last_became_active":"2023-08-11T14:02:03.339595+0000","last_became_peered":"2023-08-11T14:02:03.339595+0000","last_unstale":"2023-08-11T14:03:04.340357+0000","last_undegraded":"2023-08-11T14:03:04.340357+0000","last_fullsized":"2023-08-11T14:03:04.340357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:28:32.086038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523704+0000","last_change":"2023-08-11T14:01:33.296198+0000","last_active":"2023-08-11T14:03:04.523704+0000","last_peered":"2023-08-11T14:03:04.523704+0000","last_clean":"2023-08-11T14:03:04.523704+0000","last_became_active":"2023-08-11T14:01:33.295881+0000","last_became_peered":"2023-08-11T14:01:33.295881+0000","last_unstale":"2023-08-11T14:03:04.523704+0000","last_undegraded":"2023-08-11T14:03:04.523704+0000","last_fullsized":"2023-08-11T14:03:04.523704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:36:08.283056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420429+0000","last_change":"2023-08-11T14:01:33.304364+0000","last_active":"2023-08-11T14:03:04.420429+0000","last_peered":"2023-08-11T14:03:04.420429+0000","last_clean":"2023-08-11T14:03:04.420429+0000","last_became_active":"2023-08-11T14:01:33.304039+0000","last_became_peered":"2023-08-11T14:01:33.304039+0000","last_unstale":"2023-08-11T14:03:04.420429+0000","last_undegraded":"2023-08-11T14:03:04.420429+0000","last_fullsized":"2023-08-11T14:03:04.420429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:36:40.405178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427584+0000","last_change":"2023-08-11T14:01:33.299216+0000","last_active":"2023-08-11T14:03:04.427584+0000","last_peered":"2023-08-11T14:03:04.427584+0000","last_clean":"2023-08-11T14:03:04.427584+0000","last_became_active":"2023-08-11T14:01:33.298902+0000","last_became_peered":"2023-08-11T14:01:33.298902+0000","last_unstale":"2023-08-11T14:03:04.427584+0000","last_undegraded":"2023-08-11T14:03:04.427584+0000","last_fullsized":"2023-08-11T14:03:04.427584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:57:34.234658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345884+0000","last_change":"2023-08-11T14:01:33.302896+0000","last_active":"2023-08-11T14:03:04.345884+0000","last_peered":"2023-08-11T14:03:04.345884+0000","last_clean":"2023-08-11T14:03:04.345884+0000","last_became_active":"2023-08-11T14:01:33.302583+0000","last_became_peered":"2023-08-11T14:01:33.302583+0000","last_unstale":"2023-08-11T14:03:04.345884+0000","last_undegraded":"2023-08-11T14:03:04.345884+0000","last_fullsized":"2023-08-11T14:03:04.345884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:02:05.560872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340734+0000","last_change":"2023-08-11T14:01:33.298373+0000","last_active":"2023-08-11T14:03:04.340734+0000","last_peered":"2023-08-11T14:03:04.340734+0000","last_clean":"2023-08-11T14:03:04.340734+0000","last_became_active":"2023-08-11T14:01:33.296908+0000","last_became_peered":"2023-08-11T14:01:33.296908+0000","last_unstale":"2023-08-11T14:03:04.340734+0000","last_undegraded":"2023-08-11T14:03:04.340734+0000","last_fullsized":"2023-08-11T14:03:04.340734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:14.969554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522728+0000","last_change":"2023-08-11T14:01:33.312008+0000","last_active":"2023-08-11T14:03:04.522728+0000","last_peered":"2023-08-11T14:03:04.522728+0000","last_clean":"2023-08-11T14:03:04.522728+0000","last_became_active":"2023-08-11T14:01:33.311603+0000","last_became_peered":"2023-08-11T14:01:33.311603+0000","last_unstale":"2023-08-11T14:03:04.522728+0000","last_undegraded":"2023-08-11T14:03:04.522728+0000","last_fullsized":"2023-08-11T14:03:04.522728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:14:20.923343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522719+0000","last_change":"2023-08-11T14:01:33.306193+0000","last_active":"2023-08-11T14:03:04.522719+0000","last_peered":"2023-08-11T14:03:04.522719+0000","last_clean":"2023-08-11T14:03:04.522719+0000","last_became_active":"2023-08-11T14:01:33.305864+0000","last_became_peered":"2023-08-11T14:01:33.305864+0000","last_unstale":"2023-08-11T14:03:04.522719+0000","last_undegraded":"2023-08-11T14:03:04.522719+0000","last_fullsized":"2023-08-11T14:03:04.522719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:38:47.730825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523337+0000","last_change":"2023-08-11T14:01:33.300484+0000","last_active":"2023-08-11T14:03:04.523337+0000","last_peered":"2023-08-11T14:03:04.523337+0000","last_clean":"2023-08-11T14:03:04.523337+0000","last_became_active":"2023-08-11T14:01:33.300150+0000","last_became_peered":"2023-08-11T14:01:33.300150+0000","last_unstale":"2023-08-11T14:03:04.523337+0000","last_undegraded":"2023-08-11T14:03:04.523337+0000","last_fullsized":"2023-08-11T14:03:04.523337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:39:16.271940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343273+0000","last_change":"2023-08-11T14:01:33.300501+0000","last_active":"2023-08-11T14:03:04.343273+0000","last_peered":"2023-08-11T14:03:04.343273+0000","last_clean":"2023-08-11T14:03:04.343273+0000","last_became_active":"2023-08-11T14:01:33.300177+0000","last_became_peered":"2023-08-11T14:01:33.300177+0000","last_unstale":"2023-08-11T14:03:04.343273+0000","last_undegraded":"2023-08-11T14:03:04.343273+0000","last_fullsized":"2023-08-11T14:03:04.343273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:44:22.475322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341624+0000","last_change":"2023-08-11T14:01:33.303456+0000","last_active":"2023-08-11T14:03:04.341624+0000","last_peered":"2023-08-11T14:03:04.341624+0000","last_clean":"2023-08-11T14:03:04.341624+0000","last_became_active":"2023-08-11T14:01:33.303077+0000","last_became_peered":"2023-08-11T14:01:33.303077+0000","last_unstale":"2023-08-11T14:03:04.341624+0000","last_undegraded":"2023-08-11T14:03:04.341624+0000","last_fullsized":"2023-08-11T14:03:04.341624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:44:26.149153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341477+0000","last_change":"2023-08-11T14:01:33.311975+0000","last_active":"2023-08-11T14:03:04.341477+0000","last_peered":"2023-08-11T14:03:04.341477+0000","last_clean":"2023-08-11T14:03:04.341477+0000","last_became_active":"2023-08-11T14:01:33.311594+0000","last_became_peered":"2023-08-11T14:01:33.311594+0000","last_unstale":"2023-08-11T14:03:04.341477+0000","last_undegraded":"2023-08-11T14:03:04.341477+0000","last_fullsized":"2023-08-11T14:03:04.341477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:52:32.692687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340660+0000","last_change":"2023-08-11T14:01:37.378971+0000","last_active":"2023-08-11T14:03:04.340660+0000","last_peered":"2023-08-11T14:03:04.340660+0000","last_clean":"2023-08-11T14:03:04.340660+0000","last_became_active":"2023-08-11T14:01:37.378569+0000","last_became_peered":"2023-08-11T14:01:37.378569+0000","last_unstale":"2023-08-11T14:03:04.340660+0000","last_undegraded":"2023-08-11T14:03:04.340660+0000","last_fullsized":"2023-08-11T14:03:04.340660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:05:35.403525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.490717+0000","last_change":"2023-08-11T14:01:32.309304+0000","last_active":"2023-08-11T14:03:51.490717+0000","last_peered":"2023-08-11T14:03:51.490717+0000","last_clean":"2023-08-11T14:03:51.490717+0000","last_became_active":"2023-08-11T14:01:32.308892+0000","last_became_peered":"2023-08-11T14:01:32.308892+0000","last_unstale":"2023-08-11T14:03:51.490717+0000","last_undegraded":"2023-08-11T14:03:51.490717+0000","last_fullsized":"2023-08-11T14:03:51.490717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:30.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429485+0000","last_change":"2023-08-11T14:01:33.312023+0000","last_active":"2023-08-11T14:03:04.429485+0000","last_peered":"2023-08-11T14:03:04.429485+0000","last_clean":"2023-08-11T14:03:04.429485+0000","last_became_active":"2023-08-11T14:01:33.311644+0000","last_became_peered":"2023-08-11T14:01:33.311644+0000","last_unstale":"2023-08-11T14:03:04.429485+0000","last_undegraded":"2023-08-11T14:03:04.429485+0000","last_fullsized":"2023-08-11T14:03:04.429485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:08.247797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345171+0000","last_change":"2023-08-11T14:01:37.378954+0000","last_active":"2023-08-11T14:03:04.345171+0000","last_peered":"2023-08-11T14:03:04.345171+0000","last_clean":"2023-08-11T14:03:04.345171+0000","last_became_active":"2023-08-11T14:01:37.378574+0000","last_became_peered":"2023-08-11T14:01:37.378574+0000","last_unstale":"2023-08-11T14:03:04.345171+0000","last_undegraded":"2023-08-11T14:03:04.345171+0000","last_fullsized":"2023-08-11T14:03:04.345171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:28:23.669347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429535+0000","last_change":"2023-08-11T14:01:32.310863+0000","last_active":"2023-08-11T14:03:04.429535+0000","last_peered":"2023-08-11T14:03:04.429535+0000","last_clean":"2023-08-11T14:03:04.429535+0000","last_became_active":"2023-08-11T14:01:32.310503+0000","last_became_peered":"2023-08-11T14:01:32.310503+0000","last_unstale":"2023-08-11T14:03:04.429535+0000","last_undegraded":"2023-08-11T14:03:04.429535+0000","last_fullsized":"2023-08-11T14:03:04.429535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:21:02.749730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523055+0000","last_change":"2023-08-11T14:01:33.306156+0000","last_active":"2023-08-11T14:03:04.523055+0000","last_peered":"2023-08-11T14:03:04.523055+0000","last_clean":"2023-08-11T14:03:04.523055+0000","last_became_active":"2023-08-11T14:01:33.305786+0000","last_became_peered":"2023-08-11T14:01:33.305786+0000","last_unstale":"2023-08-11T14:03:04.523055+0000","last_undegraded":"2023-08-11T14:03:04.523055+0000","last_fullsized":"2023-08-11T14:03:04.523055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:07:21.968021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345306+0000","last_change":"2023-08-11T14:01:37.374785+0000","last_active":"2023-08-11T14:03:04.345306+0000","last_peered":"2023-08-11T14:03:04.345306+0000","last_clean":"2023-08-11T14:03:04.345306+0000","last_became_active":"2023-08-11T14:01:37.374145+0000","last_became_peered":"2023-08-11T14:01:37.374145+0000","last_unstale":"2023-08-11T14:03:04.345306+0000","last_undegraded":"2023-08-11T14:03:04.345306+0000","last_fullsized":"2023-08-11T14:03:04.345306+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:27:36.143460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345867+0000","last_change":"2023-08-11T14:01:32.300870+0000","last_active":"2023-08-11T14:03:04.345867+0000","last_peered":"2023-08-11T14:03:04.345867+0000","last_clean":"2023-08-11T14:03:04.345867+0000","last_became_active":"2023-08-11T14:01:32.300571+0000","last_became_peered":"2023-08-11T14:01:32.300571+0000","last_unstale":"2023-08-11T14:03:04.345867+0000","last_undegraded":"2023-08-11T14:03:04.345867+0000","last_fullsized":"2023-08-11T14:03:04.345867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:57:28.879262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523408+0000","last_change":"2023-08-11T14:01:33.296359+0000","last_active":"2023-08-11T14:03:04.523408+0000","last_peered":"2023-08-11T14:03:04.523408+0000","last_clean":"2023-08-11T14:03:04.523408+0000","last_became_active":"2023-08-11T14:01:33.296027+0000","last_became_peered":"2023-08-11T14:01:33.296027+0000","last_unstale":"2023-08-11T14:03:04.523408+0000","last_undegraded":"2023-08-11T14:03:04.523408+0000","last_fullsized":"2023-08-11T14:03:04.523408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:25:56.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345326+0000","last_change":"2023-08-11T14:01:37.374142+0000","last_active":"2023-08-11T14:03:04.345326+0000","last_peered":"2023-08-11T14:03:04.345326+0000","last_clean":"2023-08-11T14:03:04.345326+0000","last_became_active":"2023-08-11T14:01:37.373560+0000","last_became_peered":"2023-08-11T14:01:37.373560+0000","last_unstale":"2023-08-11T14:03:04.345326+0000","last_undegraded":"2023-08-11T14:03:04.345326+0000","last_fullsized":"2023-08-11T14:03:04.345326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:34:11.483871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345233+0000","last_change":"2023-08-11T14:01:32.297032+0000","last_active":"2023-08-11T14:03:04.345233+0000","last_peered":"2023-08-11T14:03:04.345233+0000","last_clean":"2023-08-11T14:03:04.345233+0000","last_became_active":"2023-08-11T14:01:32.296673+0000","last_became_peered":"2023-08-11T14:01:32.296673+0000","last_unstale":"2023-08-11T14:03:04.345233+0000","last_undegraded":"2023-08-11T14:03:04.345233+0000","last_fullsized":"2023-08-11T14:03:04.345233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:30:08.629247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346154+0000","last_change":"2023-08-11T14:01:33.304734+0000","last_active":"2023-08-11T14:03:04.346154+0000","last_peered":"2023-08-11T14:03:04.346154+0000","last_clean":"2023-08-11T14:03:04.346154+0000","last_became_active":"2023-08-11T14:01:33.304241+0000","last_became_peered":"2023-08-11T14:01:33.304241+0000","last_unstale":"2023-08-11T14:03:04.346154+0000","last_undegraded":"2023-08-11T14:03:04.346154+0000","last_fullsized":"2023-08-11T14:03:04.346154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:05:50.190228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341191+0000","last_change":"2023-08-11T14:01:37.370136+0000","last_active":"2023-08-11T14:03:04.341191+0000","last_peered":"2023-08-11T14:03:04.341191+0000","last_clean":"2023-08-11T14:03:04.341191+0000","last_became_active":"2023-08-11T14:01:37.369701+0000","last_became_peered":"2023-08-11T14:01:37.369701+0000","last_unstale":"2023-08-11T14:03:04.341191+0000","last_undegraded":"2023-08-11T14:03:04.341191+0000","last_fullsized":"2023-08-11T14:03:04.341191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:20.949554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:50.490894+0000","last_change":"2023-08-11T14:01:32.297416+0000","last_active":"2023-08-11T14:03:50.490894+0000","last_peered":"2023-08-11T14:03:50.490894+0000","last_clean":"2023-08-11T14:03:50.490894+0000","last_became_active":"2023-08-11T14:01:32.297155+0000","last_became_peered":"2023-08-11T14:01:32.297155+0000","last_unstale":"2023-08-11T14:03:50.490894+0000","last_undegraded":"2023-08-11T14:03:50.490894+0000","last_fullsized":"2023-08-11T14:03:50.490894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:17:13.008649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345556+0000","last_change":"2023-08-11T14:01:33.301146+0000","last_active":"2023-08-11T14:03:04.345556+0000","last_peered":"2023-08-11T14:03:04.345556+0000","last_clean":"2023-08-11T14:03:04.345556+0000","last_became_active":"2023-08-11T14:01:33.300392+0000","last_became_peered":"2023-08-11T14:01:33.300392+0000","last_unstale":"2023-08-11T14:03:04.345556+0000","last_undegraded":"2023-08-11T14:03:04.345556+0000","last_fullsized":"2023-08-11T14:03:04.345556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:40:55.597917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341002+0000","last_change":"2023-08-11T14:01:37.378864+0000","last_active":"2023-08-11T14:03:04.341002+0000","last_peered":"2023-08-11T14:03:04.341002+0000","last_clean":"2023-08-11T14:03:04.341002+0000","last_became_active":"2023-08-11T14:01:37.378445+0000","last_became_peered":"2023-08-11T14:01:37.378445+0000","last_unstale":"2023-08-11T14:03:04.341002+0000","last_undegraded":"2023-08-11T14:03:04.341002+0000","last_fullsized":"2023-08-11T14:03:04.341002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:43:57.181073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.419897+0000","last_change":"2023-08-11T14:01:32.294998+0000","last_active":"2023-08-11T14:03:04.419897+0000","last_peered":"2023-08-11T14:03:04.419897+0000","last_clean":"2023-08-11T14:03:04.419897+0000","last_became_active":"2023-08-11T14:01:32.294597+0000","last_became_peered":"2023-08-11T14:01:32.294597+0000","last_unstale":"2023-08-11T14:03:04.419897+0000","last_undegraded":"2023-08-11T14:03:04.419897+0000","last_fullsized":"2023-08-11T14:03:04.419897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:40:06.849906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346383+0000","last_change":"2023-08-11T14:01:33.312058+0000","last_active":"2023-08-11T14:03:04.346383+0000","last_peered":"2023-08-11T14:03:04.346383+0000","last_clean":"2023-08-11T14:03:04.346383+0000","last_became_active":"2023-08-11T14:01:33.311742+0000","last_became_peered":"2023-08-11T14:01:33.311742+0000","last_unstale":"2023-08-11T14:03:04.346383+0000","last_undegraded":"2023-08-11T14:03:04.346383+0000","last_fullsized":"2023-08-11T14:03:04.346383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:42:53.656558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:37.377910+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:37.377498+0000","last_became_peered":"2023-08-11T14:01:37.377498+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:54:52.322697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339638+0000","last_change":"2023-08-11T14:01:32.309818+0000","last_active":"2023-08-11T14:03:04.339638+0000","last_peered":"2023-08-11T14:03:04.339638+0000","last_clean":"2023-08-11T14:03:04.339638+0000","last_became_active":"2023-08-11T14:01:32.309413+0000","last_became_peered":"2023-08-11T14:01:32.309413+0000","last_unstale":"2023-08-11T14:03:04.339638+0000","last_undegraded":"2023-08-11T14:03:04.339638+0000","last_fullsized":"2023-08-11T14:03:04.339638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:12:09.761922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429002+0000","last_change":"2023-08-11T14:01:33.312781+0000","last_active":"2023-08-11T14:03:04.429002+0000","last_peered":"2023-08-11T14:03:04.429002+0000","last_clean":"2023-08-11T14:03:04.429002+0000","last_became_active":"2023-08-11T14:01:33.312275+0000","last_became_peered":"2023-08-11T14:01:33.312275+0000","last_unstale":"2023-08-11T14:03:04.429002+0000","last_undegraded":"2023-08-11T14:03:04.429002+0000","last_fullsized":"2023-08-11T14:03:04.429002+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:23:22.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523029+0000","last_change":"2023-08-11T14:01:37.375310+0000","last_active":"2023-08-11T14:03:04.523029+0000","last_peered":"2023-08-11T14:03:04.523029+0000","last_clean":"2023-08-11T14:03:04.523029+0000","last_became_active":"2023-08-11T14:01:37.374866+0000","last_became_peered":"2023-08-11T14:01:37.374866+0000","last_unstale":"2023-08-11T14:03:04.523029+0000","last_undegraded":"2023-08-11T14:03:04.523029+0000","last_fullsized":"2023-08-11T14:03:04.523029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:43:41.402588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345435+0000","last_change":"2023-08-11T14:01:32.297918+0000","last_active":"2023-08-11T14:03:04.345435+0000","last_peered":"2023-08-11T14:03:04.345435+0000","last_clean":"2023-08-11T14:03:04.345435+0000","last_became_active":"2023-08-11T14:01:32.297601+0000","last_became_peered":"2023-08-11T14:01:32.297601+0000","last_unstale":"2023-08-11T14:03:04.345435+0000","last_undegraded":"2023-08-11T14:03:04.345435+0000","last_fullsized":"2023-08-11T14:03:04.345435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:35:33.118766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341241+0000","last_change":"2023-08-11T14:01:33.313152+0000","last_active":"2023-08-11T14:03:04.341241+0000","last_peered":"2023-08-11T14:03:04.341241+0000","last_clean":"2023-08-11T14:03:04.341241+0000","last_became_active":"2023-08-11T14:01:33.312823+0000","last_became_peered":"2023-08-11T14:01:33.312823+0000","last_unstale":"2023-08-11T14:03:04.341241+0000","last_undegraded":"2023-08-11T14:03:04.341241+0000","last_fullsized":"2023-08-11T14:03:04.341241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:53:48.999413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429059+0000","last_change":"2023-08-11T14:01:37.369746+0000","last_active":"2023-08-11T14:03:04.429059+0000","last_peered":"2023-08-11T14:03:04.429059+0000","last_clean":"2023-08-11T14:03:04.429059+0000","last_became_active":"2023-08-11T14:01:37.369355+0000","last_became_peered":"2023-08-11T14:01:37.369355+0000","last_unstale":"2023-08-11T14:03:04.429059+0000","last_undegraded":"2023-08-11T14:03:04.429059+0000","last_fullsized":"2023-08-11T14:03:04.429059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:42:25.445721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.494522+0000","last_change":"2023-08-11T14:01:32.310136+0000","last_active":"2023-08-11T14:03:53.494522+0000","last_peered":"2023-08-11T14:03:53.494522+0000","last_clean":"2023-08-11T14:03:53.494522+0000","last_became_active":"2023-08-11T14:01:32.309813+0000","last_became_peered":"2023-08-11T14:01:32.309813+0000","last_unstale":"2023-08-11T14:03:53.494522+0000","last_undegraded":"2023-08-11T14:03:53.494522+0000","last_fullsized":"2023-08-11T14:03:53.494522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:55:08.555355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523699+0000","last_change":"2023-08-11T14:01:33.298090+0000","last_active":"2023-08-11T14:03:04.523699+0000","last_peered":"2023-08-11T14:03:04.523699+0000","last_clean":"2023-08-11T14:03:04.523699+0000","last_became_active":"2023-08-11T14:01:33.297809+0000","last_became_peered":"2023-08-11T14:01:33.297809+0000","last_unstale":"2023-08-11T14:03:04.523699+0000","last_undegraded":"2023-08-11T14:03:04.523699+0000","last_fullsized":"2023-08-11T14:03:04.523699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:53:27.682173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340632+0000","last_change":"2023-08-11T14:01:37.378096+0000","last_active":"2023-08-11T14:03:04.340632+0000","last_peered":"2023-08-11T14:03:04.340632+0000","last_clean":"2023-08-11T14:03:04.340632+0000","last_became_active":"2023-08-11T14:01:37.377573+0000","last_became_peered":"2023-08-11T14:01:37.377573+0000","last_unstale":"2023-08-11T14:03:04.340632+0000","last_undegraded":"2023-08-11T14:03:04.340632+0000","last_fullsized":"2023-08-11T14:03:04.340632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:39:07.774291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427459+0000","last_change":"2023-08-11T14:01:32.299481+0000","last_active":"2023-08-11T14:03:04.427459+0000","last_peered":"2023-08-11T14:03:04.427459+0000","last_clean":"2023-08-11T14:03:04.427459+0000","last_became_active":"2023-08-11T14:01:32.298280+0000","last_became_peered":"2023-08-11T14:01:32.298280+0000","last_unstale":"2023-08-11T14:03:04.427459+0000","last_undegraded":"2023-08-11T14:03:04.427459+0000","last_fullsized":"2023-08-11T14:03:04.427459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:58:37.501349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523164+0000","last_change":"2023-08-11T14:02:03.342277+0000","last_active":"2023-08-11T14:03:04.523164+0000","last_peered":"2023-08-11T14:03:04.523164+0000","last_clean":"2023-08-11T14:03:04.523164+0000","last_became_active":"2023-08-11T14:02:03.341757+0000","last_became_peered":"2023-08-11T14:02:03.341757+0000","last_unstale":"2023-08-11T14:03:04.523164+0000","last_undegraded":"2023-08-11T14:03:04.523164+0000","last_fullsized":"2023-08-11T14:03:04.523164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:14:29.876746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345553+0000","last_change":"2023-08-11T14:01:37.373070+0000","last_active":"2023-08-11T14:03:04.345553+0000","last_peered":"2023-08-11T14:03:04.345553+0000","last_clean":"2023-08-11T14:03:04.345553+0000","last_became_active":"2023-08-11T14:01:37.372585+0000","last_became_peered":"2023-08-11T14:01:37.372585+0000","last_unstale":"2023-08-11T14:03:04.345553+0000","last_undegraded":"2023-08-11T14:03:04.345553+0000","last_fullsized":"2023-08-11T14:03:04.345553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:03:34.946284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346031+0000","last_change":"2023-08-11T14:01:32.305504+0000","last_active":"2023-08-11T14:03:04.346031+0000","last_peered":"2023-08-11T14:03:04.346031+0000","last_clean":"2023-08-11T14:03:04.346031+0000","last_became_active":"2023-08-11T14:01:32.305284+0000","last_became_peered":"2023-08-11T14:01:32.305284+0000","last_unstale":"2023-08-11T14:03:04.346031+0000","last_undegraded":"2023-08-11T14:03:04.346031+0000","last_fullsized":"2023-08-11T14:03:04.346031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:45.032323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523852+0000","last_change":"2023-08-11T14:01:33.297350+0000","last_active":"2023-08-11T14:03:04.523852+0000","last_peered":"2023-08-11T14:03:04.523852+0000","last_clean":"2023-08-11T14:03:04.523852+0000","last_became_active":"2023-08-11T14:01:33.297043+0000","last_became_peered":"2023-08-11T14:01:33.297043+0000","last_unstale":"2023-08-11T14:03:04.523852+0000","last_undegraded":"2023-08-11T14:03:04.523852+0000","last_fullsized":"2023-08-11T14:03:04.523852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:36:24.226941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339856+0000","last_change":"2023-08-11T14:01:37.379570+0000","last_active":"2023-08-11T14:03:04.339856+0000","last_peered":"2023-08-11T14:03:04.339856+0000","last_clean":"2023-08-11T14:03:04.339856+0000","last_became_active":"2023-08-11T14:01:37.378908+0000","last_became_peered":"2023-08-11T14:01:37.378908+0000","last_unstale":"2023-08-11T14:03:04.339856+0000","last_undegraded":"2023-08-11T14:03:04.339856+0000","last_fullsized":"2023-08-11T14:03:04.339856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:59:30.100680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":306,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:53.495578+0000","last_change":"2023-08-11T14:01:32.294132+0000","last_active":"2023-08-11T14:03:53.495578+0000","last_peered":"2023-08-11T14:03:53.495578+0000","last_clean":"2023-08-11T14:03:53.495578+0000","last_became_active":"2023-08-11T14:01:32.293823+0000","last_became_peered":"2023-08-11T14:01:32.293823+0000","last_unstale":"2023-08-11T14:03:53.495578+0000","last_undegraded":"2023-08-11T14:03:53.495578+0000","last_fullsized":"2023-08-11T14:03:53.495578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:23:31.930235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345541+0000","last_change":"2023-08-11T14:01:33.302510+0000","last_active":"2023-08-11T14:03:04.345541+0000","last_peered":"2023-08-11T14:03:04.345541+0000","last_clean":"2023-08-11T14:03:04.345541+0000","last_became_active":"2023-08-11T14:01:33.302156+0000","last_became_peered":"2023-08-11T14:01:33.302156+0000","last_unstale":"2023-08-11T14:03:04.345541+0000","last_undegraded":"2023-08-11T14:03:04.345541+0000","last_fullsized":"2023-08-11T14:03:04.345541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:02:34.156179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.347052+0000","last_change":"2023-08-11T14:01:37.375418+0000","last_active":"2023-08-11T14:03:04.347052+0000","last_peered":"2023-08-11T14:03:04.347052+0000","last_clean":"2023-08-11T14:03:04.347052+0000","last_became_active":"2023-08-11T14:01:37.375024+0000","last_became_peered":"2023-08-11T14:01:37.375024+0000","last_unstale":"2023-08-11T14:03:04.347052+0000","last_undegraded":"2023-08-11T14:03:04.347052+0000","last_fullsized":"2023-08-11T14:03:04.347052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:11:15.433011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342697+0000","last_change":"2023-08-11T14:02:03.350806+0000","last_active":"2023-08-11T14:03:04.342697+0000","last_peered":"2023-08-11T14:03:04.342697+0000","last_clean":"2023-08-11T14:03:04.342697+0000","last_became_active":"2023-08-11T14:02:03.339504+0000","last_became_peered":"2023-08-11T14:02:03.339504+0000","last_unstale":"2023-08-11T14:03:04.342697+0000","last_undegraded":"2023-08-11T14:03:04.342697+0000","last_fullsized":"2023-08-11T14:03:04.342697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:41.784623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340323+0000","last_change":"2023-08-11T14:01:33.308935+0000","last_active":"2023-08-11T14:03:04.340323+0000","last_peered":"2023-08-11T14:03:04.340323+0000","last_clean":"2023-08-11T14:03:04.340323+0000","last_became_active":"2023-08-11T14:01:33.308576+0000","last_became_peered":"2023-08-11T14:01:33.308576+0000","last_unstale":"2023-08-11T14:03:04.340323+0000","last_undegraded":"2023-08-11T14:03:04.340323+0000","last_fullsized":"2023-08-11T14:03:04.340323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:17:34.035219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342711+0000","last_change":"2023-08-11T14:02:03.515889+0000","last_active":"2023-08-11T14:03:04.342711+0000","last_peered":"2023-08-11T14:03:04.342711+0000","last_clean":"2023-08-11T14:03:04.342711+0000","last_became_active":"2023-08-11T14:02:03.515400+0000","last_became_peered":"2023-08-11T14:02:03.515400+0000","last_unstale":"2023-08-11T14:03:04.342711+0000","last_undegraded":"2023-08-11T14:03:04.342711+0000","last_fullsized":"2023-08-11T14:03:04.342711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:29:44.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,7,6],"acting":[11,10,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346763+0000","last_change":"2023-08-11T14:01:32.302495+0000","last_active":"2023-08-11T14:03:04.346763+0000","last_peered":"2023-08-11T14:03:04.346763+0000","last_clean":"2023-08-11T14:03:04.346763+0000","last_became_active":"2023-08-11T14:01:32.302176+0000","last_became_peered":"2023-08-11T14:01:32.302176+0000","last_unstale":"2023-08-11T14:03:04.346763+0000","last_undegraded":"2023-08-11T14:03:04.346763+0000","last_fullsized":"2023-08-11T14:03:04.346763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:19:42.194578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427345+0000","last_change":"2023-08-11T14:01:33.309869+0000","last_active":"2023-08-11T14:03:04.427345+0000","last_peered":"2023-08-11T14:03:04.427345+0000","last_clean":"2023-08-11T14:03:04.427345+0000","last_became_active":"2023-08-11T14:01:33.309474+0000","last_became_peered":"2023-08-11T14:01:33.309474+0000","last_unstale":"2023-08-11T14:03:04.427345+0000","last_undegraded":"2023-08-11T14:03:04.427345+0000","last_fullsized":"2023-08-11T14:03:04.427345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:23:29.378966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346097+0000","last_change":"2023-08-11T14:02:03.516118+0000","last_active":"2023-08-11T14:03:04.346097+0000","last_peered":"2023-08-11T14:03:04.346097+0000","last_clean":"2023-08-11T14:03:04.346097+0000","last_became_active":"2023-08-11T14:02:03.515502+0000","last_became_peered":"2023-08-11T14:02:03.515502+0000","last_unstale":"2023-08-11T14:03:04.346097+0000","last_undegraded":"2023-08-11T14:03:04.346097+0000","last_fullsized":"2023-08-11T14:03:04.346097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:56:18.989452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:51.491776+0000","last_change":"2023-08-11T14:01:32.297166+0000","last_active":"2023-08-11T14:03:51.491776+0000","last_peered":"2023-08-11T14:03:51.491776+0000","last_clean":"2023-08-11T14:03:51.491776+0000","last_became_active":"2023-08-11T14:01:32.296805+0000","last_became_peered":"2023-08-11T14:01:32.296805+0000","last_unstale":"2023-08-11T14:03:51.491776+0000","last_undegraded":"2023-08-11T14:03:51.491776+0000","last_fullsized":"2023-08-11T14:03:51.491776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:00:47.290694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426969+0000","last_change":"2023-08-11T14:01:33.306527+0000","last_active":"2023-08-11T14:03:04.426969+0000","last_peered":"2023-08-11T14:03:04.426969+0000","last_clean":"2023-08-11T14:03:04.426969+0000","last_became_active":"2023-08-11T14:01:33.306221+0000","last_became_peered":"2023-08-11T14:01:33.306221+0000","last_unstale":"2023-08-11T14:03:04.426969+0000","last_undegraded":"2023-08-11T14:03:04.426969+0000","last_fullsized":"2023-08-11T14:03:04.426969+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:35:46.867384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340459+0000","last_change":"2023-08-11T14:02:03.341361+0000","last_active":"2023-08-11T14:03:04.340459+0000","last_peered":"2023-08-11T14:03:04.340459+0000","last_clean":"2023-08-11T14:03:04.340459+0000","last_became_active":"2023-08-11T14:02:03.340905+0000","last_became_peered":"2023-08-11T14:02:03.340905+0000","last_unstale":"2023-08-11T14:03:04.340459+0000","last_undegraded":"2023-08-11T14:03:04.340459+0000","last_fullsized":"2023-08-11T14:03:04.340459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:51.746271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.344937+0000","last_change":"2023-08-11T14:01:32.301008+0000","last_active":"2023-08-11T14:03:04.344937+0000","last_peered":"2023-08-11T14:03:04.344937+0000","last_clean":"2023-08-11T14:03:04.344937+0000","last_became_active":"2023-08-11T14:01:32.300688+0000","last_became_peered":"2023-08-11T14:01:32.300688+0000","last_unstale":"2023-08-11T14:03:04.344937+0000","last_undegraded":"2023-08-11T14:03:04.344937+0000","last_fullsized":"2023-08-11T14:03:04.344937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:59:13.455539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341159+0000","last_change":"2023-08-11T14:02:03.515481+0000","last_active":"2023-08-11T14:03:04.341159+0000","last_peered":"2023-08-11T14:03:04.341159+0000","last_clean":"2023-08-11T14:03:04.341159+0000","last_became_active":"2023-08-11T14:02:03.514929+0000","last_became_peered":"2023-08-11T14:02:03.514929+0000","last_unstale":"2023-08-11T14:03:04.341159+0000","last_undegraded":"2023-08-11T14:03:04.341159+0000","last_fullsized":"2023-08-11T14:03:04.341159+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:46:34.163158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346090+0000","last_change":"2023-08-11T14:01:37.376089+0000","last_active":"2023-08-11T14:03:04.346090+0000","last_peered":"2023-08-11T14:03:04.346090+0000","last_clean":"2023-08-11T14:03:04.346090+0000","last_became_active":"2023-08-11T14:01:37.375694+0000","last_became_peered":"2023-08-11T14:01:37.375694+0000","last_unstale":"2023-08-11T14:03:04.346090+0000","last_undegraded":"2023-08-11T14:03:04.346090+0000","last_fullsized":"2023-08-11T14:03:04.346090+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:22:02.526755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.352593+0000","last_change":"2023-08-11T14:03:04.352113+0000","last_active":"2023-08-11T14:03:04.352593+0000","last_peered":"2023-08-11T14:03:04.352593+0000","last_clean":"2023-08-11T14:03:04.352593+0000","last_became_active":"2023-08-11T14:03:04.347837+0000","last_became_peered":"2023-08-11T14:03:04.347837+0000","last_unstale":"2023-08-11T14:03:04.352593+0000","last_undegraded":"2023-08-11T14:03:04.352593+0000","last_fullsized":"2023-08-11T14:03:04.352593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:47.346364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342848+0000","last_change":"2023-08-11T14:01:33.297376+0000","last_active":"2023-08-11T14:03:04.342848+0000","last_peered":"2023-08-11T14:03:04.342848+0000","last_clean":"2023-08-11T14:03:04.342848+0000","last_became_active":"2023-08-11T14:01:33.297049+0000","last_became_peered":"2023-08-11T14:01:33.297049+0000","last_unstale":"2023-08-11T14:03:04.342848+0000","last_undegraded":"2023-08-11T14:03:04.342848+0000","last_fullsized":"2023-08-11T14:03:04.342848+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:15:43.902027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345467+0000","last_change":"2023-08-11T14:01:37.586381+0000","last_active":"2023-08-11T14:03:04.345467+0000","last_peered":"2023-08-11T14:03:04.345467+0000","last_clean":"2023-08-11T14:03:04.345467+0000","last_became_active":"2023-08-11T14:01:37.585837+0000","last_became_peered":"2023-08-11T14:01:37.585837+0000","last_unstale":"2023-08-11T14:03:04.345467+0000","last_undegraded":"2023-08-11T14:03:04.345467+0000","last_fullsized":"2023-08-11T14:03:04.345467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:56:57.057812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341336+0000","last_change":"2023-08-11T14:01:32.308350+0000","last_active":"2023-08-11T14:03:04.341336+0000","last_peered":"2023-08-11T14:03:04.341336+0000","last_clean":"2023-08-11T14:03:04.341336+0000","last_became_active":"2023-08-11T14:01:32.307996+0000","last_became_peered":"2023-08-11T14:01:32.307996+0000","last_unstale":"2023-08-11T14:03:04.341336+0000","last_undegraded":"2023-08-11T14:03:04.341336+0000","last_fullsized":"2023-08-11T14:03:04.341336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:48:09.795680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420336+0000","last_change":"2023-08-11T14:01:33.309430+0000","last_active":"2023-08-11T14:03:04.420336+0000","last_peered":"2023-08-11T14:03:04.420336+0000","last_clean":"2023-08-11T14:03:04.420336+0000","last_became_active":"2023-08-11T14:01:33.309124+0000","last_became_peered":"2023-08-11T14:01:33.309124+0000","last_unstale":"2023-08-11T14:03:04.420336+0000","last_undegraded":"2023-08-11T14:03:04.420336+0000","last_fullsized":"2023-08-11T14:03:04.420336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:26:04.032679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.339836+0000","last_change":"2023-08-11T14:01:37.373827+0000","last_active":"2023-08-11T14:03:04.339836+0000","last_peered":"2023-08-11T14:03:04.339836+0000","last_clean":"2023-08-11T14:03:04.339836+0000","last_became_active":"2023-08-11T14:01:37.373311+0000","last_became_peered":"2023-08-11T14:01:37.373311+0000","last_unstale":"2023-08-11T14:03:04.339836+0000","last_undegraded":"2023-08-11T14:03:04.339836+0000","last_fullsized":"2023-08-11T14:03:04.339836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:42:17.950224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":146,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:48.487314+0000","last_change":"2023-08-11T14:01:32.313018+0000","last_active":"2023-08-11T14:03:48.487314+0000","last_peered":"2023-08-11T14:03:48.487314+0000","last_clean":"2023-08-11T14:03:48.487314+0000","last_became_active":"2023-08-11T14:01:32.312790+0000","last_became_peered":"2023-08-11T14:01:32.312790+0000","last_unstale":"2023-08-11T14:03:48.487314+0000","last_undegraded":"2023-08-11T14:03:48.487314+0000","last_fullsized":"2023-08-11T14:03:48.487314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:26:25.158801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.420203+0000","last_change":"2023-08-11T14:01:33.303578+0000","last_active":"2023-08-11T14:03:04.420203+0000","last_peered":"2023-08-11T14:03:04.420203+0000","last_clean":"2023-08-11T14:03:04.420203+0000","last_became_active":"2023-08-11T14:01:33.302983+0000","last_became_peered":"2023-08-11T14:01:33.302983+0000","last_unstale":"2023-08-11T14:03:04.420203+0000","last_undegraded":"2023-08-11T14:03:04.420203+0000","last_fullsized":"2023-08-11T14:03:04.420203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:44:57.775826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341077+0000","last_change":"2023-08-11T14:01:37.585708+0000","last_active":"2023-08-11T14:03:04.341077+0000","last_peered":"2023-08-11T14:03:04.341077+0000","last_clean":"2023-08-11T14:03:04.341077+0000","last_became_active":"2023-08-11T14:01:37.585207+0000","last_became_peered":"2023-08-11T14:01:37.585207+0000","last_unstale":"2023-08-11T14:03:04.341077+0000","last_undegraded":"2023-08-11T14:03:04.341077+0000","last_fullsized":"2023-08-11T14:03:04.341077+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:23:59.875215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345872+0000","last_change":"2023-08-11T14:01:33.307297+0000","last_active":"2023-08-11T14:03:04.345872+0000","last_peered":"2023-08-11T14:03:04.345872+0000","last_clean":"2023-08-11T14:03:04.345872+0000","last_became_active":"2023-08-11T14:01:33.307008+0000","last_became_peered":"2023-08-11T14:01:33.307008+0000","last_unstale":"2023-08-11T14:03:04.345872+0000","last_undegraded":"2023-08-11T14:03:04.345872+0000","last_fullsized":"2023-08-11T14:03:04.345872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:35:57.958298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:52.492527+0000","last_change":"2023-08-11T14:01:32.309685+0000","last_active":"2023-08-11T14:03:52.492527+0000","last_peered":"2023-08-11T14:03:52.492527+0000","last_clean":"2023-08-11T14:03:52.492527+0000","last_became_active":"2023-08-11T14:01:32.309321+0000","last_became_peered":"2023-08-11T14:01:32.309321+0000","last_unstale":"2023-08-11T14:03:52.492527+0000","last_undegraded":"2023-08-11T14:03:52.492527+0000","last_fullsized":"2023-08-11T14:03:52.492527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:07:28.567968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427398+0000","last_change":"2023-08-11T14:01:37.377301+0000","last_active":"2023-08-11T14:03:04.427398+0000","last_peered":"2023-08-11T14:03:04.427398+0000","last_clean":"2023-08-11T14:03:04.427398+0000","last_became_active":"2023-08-11T14:01:37.376885+0000","last_became_peered":"2023-08-11T14:01:37.376885+0000","last_unstale":"2023-08-11T14:03:04.427398+0000","last_undegraded":"2023-08-11T14:03:04.427398+0000","last_fullsized":"2023-08-11T14:03:04.427398+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:14:04.821908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429434+0000","last_change":"2023-08-11T14:01:33.313449+0000","last_active":"2023-08-11T14:03:04.429434+0000","last_peered":"2023-08-11T14:03:04.429434+0000","last_clean":"2023-08-11T14:03:04.429434+0000","last_became_active":"2023-08-11T14:01:33.313149+0000","last_became_peered":"2023-08-11T14:01:33.313149+0000","last_unstale":"2023-08-11T14:03:04.429434+0000","last_undegraded":"2023-08-11T14:03:04.429434+0000","last_fullsized":"2023-08-11T14:03:04.429434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:51:44.310088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345441+0000","last_change":"2023-08-11T14:01:32.308507+0000","last_active":"2023-08-11T14:03:04.345441+0000","last_peered":"2023-08-11T14:03:04.345441+0000","last_clean":"2023-08-11T14:03:04.345441+0000","last_became_active":"2023-08-11T14:01:32.308212+0000","last_became_peered":"2023-08-11T14:01:32.308212+0000","last_unstale":"2023-08-11T14:03:04.345441+0000","last_undegraded":"2023-08-11T14:03:04.345441+0000","last_fullsized":"2023-08-11T14:03:04.345441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:19:06.862831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426824+0000","last_change":"2023-08-11T14:00:27.311367+0000","last_active":"2023-08-11T14:03:04.426824+0000","last_peered":"2023-08-11T14:03:04.426824+0000","last_clean":"2023-08-11T14:03:04.426824+0000","last_became_active":"2023-08-11T14:00:27.310793+0000","last_became_peered":"2023-08-11T14:00:27.310793+0000","last_unstale":"2023-08-11T14:03:04.426824+0000","last_undegraded":"2023-08-11T14:03:04.426824+0000","last_fullsized":"2023-08-11T14:03:04.426824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T13:57:20.696426+0000","last_clean_scrub_stamp":"2023-08-11T13:57:20.696426+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:07.631487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346982+0000","last_change":"2023-08-11T14:01:37.586734+0000","last_active":"2023-08-11T14:03:04.346982+0000","last_peered":"2023-08-11T14:03:04.346982+0000","last_clean":"2023-08-11T14:03:04.346982+0000","last_became_active":"2023-08-11T14:01:37.586265+0000","last_became_peered":"2023-08-11T14:01:37.586265+0000","last_unstale":"2023-08-11T14:03:04.346982+0000","last_undegraded":"2023-08-11T14:03:04.346982+0000","last_fullsized":"2023-08-11T14:03:04.346982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:40:24.515141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340804+0000","last_change":"2023-08-11T14:02:03.335496+0000","last_active":"2023-08-11T14:03:04.340804+0000","last_peered":"2023-08-11T14:03:04.340804+0000","last_clean":"2023-08-11T14:03:04.340804+0000","last_became_active":"2023-08-11T14:02:03.335147+0000","last_became_peered":"2023-08-11T14:02:03.335147+0000","last_unstale":"2023-08-11T14:03:04.340804+0000","last_undegraded":"2023-08-11T14:03:04.340804+0000","last_fullsized":"2023-08-11T14:03:04.340804+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:12:23.071563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.426987+0000","last_change":"2023-08-11T14:01:32.298929+0000","last_active":"2023-08-11T14:03:04.426987+0000","last_peered":"2023-08-11T14:03:04.426987+0000","last_clean":"2023-08-11T14:03:04.426987+0000","last_became_active":"2023-08-11T14:01:32.298119+0000","last_became_peered":"2023-08-11T14:01:32.298119+0000","last_unstale":"2023-08-11T14:03:04.426987+0000","last_undegraded":"2023-08-11T14:03:04.426987+0000","last_fullsized":"2023-08-11T14:03:04.426987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:54:16.816623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342995+0000","last_change":"2023-08-11T14:01:37.378488+0000","last_active":"2023-08-11T14:03:04.342995+0000","last_peered":"2023-08-11T14:03:04.342995+0000","last_clean":"2023-08-11T14:03:04.342995+0000","last_became_active":"2023-08-11T14:01:37.377042+0000","last_became_peered":"2023-08-11T14:01:37.377042+0000","last_unstale":"2023-08-11T14:03:04.342995+0000","last_undegraded":"2023-08-11T14:03:04.342995+0000","last_fullsized":"2023-08-11T14:03:04.342995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:11:00.973671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:32.309408+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:32.309143+0000","last_became_peered":"2023-08-11T14:01:32.309143+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:05.194595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340208+0000","last_change":"2023-08-11T14:01:33.305357+0000","last_active":"2023-08-11T14:03:04.340208+0000","last_peered":"2023-08-11T14:03:04.340208+0000","last_clean":"2023-08-11T14:03:04.340208+0000","last_became_active":"2023-08-11T14:01:33.304933+0000","last_became_peered":"2023-08-11T14:01:33.304933+0000","last_unstale":"2023-08-11T14:03:04.340208+0000","last_undegraded":"2023-08-11T14:03:04.340208+0000","last_fullsized":"2023-08-11T14:03:04.340208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:28:36.744960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.428991+0000","last_change":"2023-08-11T14:01:37.586804+0000","last_active":"2023-08-11T14:03:04.428991+0000","last_peered":"2023-08-11T14:03:04.428991+0000","last_clean":"2023-08-11T14:03:04.428991+0000","last_became_active":"2023-08-11T14:01:37.586383+0000","last_became_peered":"2023-08-11T14:01:37.586383+0000","last_unstale":"2023-08-11T14:03:04.428991+0000","last_undegraded":"2023-08-11T14:03:04.428991+0000","last_fullsized":"2023-08-11T14:03:04.428991+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:06:57.242042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:43.252925+0000","last_change":"2023-08-11T14:01:32.311443+0000","last_active":"2023-08-11T14:03:43.252925+0000","last_peered":"2023-08-11T14:03:43.252925+0000","last_clean":"2023-08-11T14:03:43.252925+0000","last_became_active":"2023-08-11T14:01:32.311041+0000","last_became_peered":"2023-08-11T14:01:32.311041+0000","last_unstale":"2023-08-11T14:03:43.252925+0000","last_undegraded":"2023-08-11T14:03:43.252925+0000","last_fullsized":"2023-08-11T14:03:43.252925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:19:42.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429596+0000","last_change":"2023-08-11T14:01:33.313736+0000","last_active":"2023-08-11T14:03:04.429596+0000","last_peered":"2023-08-11T14:03:04.429596+0000","last_clean":"2023-08-11T14:03:04.429596+0000","last_became_active":"2023-08-11T14:01:33.313387+0000","last_became_peered":"2023-08-11T14:01:33.313387+0000","last_unstale":"2023-08-11T14:03:04.429596+0000","last_undegraded":"2023-08-11T14:03:04.429596+0000","last_fullsized":"2023-08-11T14:03:04.429596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:21:59.987965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340537+0000","last_change":"2023-08-11T14:01:37.370254+0000","last_active":"2023-08-11T14:03:04.340537+0000","last_peered":"2023-08-11T14:03:04.340537+0000","last_clean":"2023-08-11T14:03:04.340537+0000","last_became_active":"2023-08-11T14:01:37.369760+0000","last_became_peered":"2023-08-11T14:01:37.369760+0000","last_unstale":"2023-08-11T14:03:04.340537+0000","last_undegraded":"2023-08-11T14:03:04.340537+0000","last_fullsized":"2023-08-11T14:03:04.340537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:10:35.951459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346004+0000","last_change":"2023-08-11T14:01:32.295592+0000","last_active":"2023-08-11T14:03:04.346004+0000","last_peered":"2023-08-11T14:03:04.346004+0000","last_clean":"2023-08-11T14:03:04.346004+0000","last_became_active":"2023-08-11T14:01:32.295247+0000","last_became_peered":"2023-08-11T14:01:32.295247+0000","last_unstale":"2023-08-11T14:03:04.346004+0000","last_undegraded":"2023-08-11T14:03:04.346004+0000","last_fullsized":"2023-08-11T14:03:04.346004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:26:55.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523190+0000","last_change":"2023-08-11T14:01:33.313520+0000","last_active":"2023-08-11T14:03:04.523190+0000","last_peered":"2023-08-11T14:03:04.523190+0000","last_clean":"2023-08-11T14:03:04.523190+0000","last_became_active":"2023-08-11T14:01:33.313198+0000","last_became_peered":"2023-08-11T14:01:33.313198+0000","last_unstale":"2023-08-11T14:03:04.523190+0000","last_undegraded":"2023-08-11T14:03:04.523190+0000","last_fullsized":"2023-08-11T14:03:04.523190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:13:04.607004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345962+0000","last_change":"2023-08-11T14:02:03.338205+0000","last_active":"2023-08-11T14:03:04.345962+0000","last_peered":"2023-08-11T14:03:04.345962+0000","last_clean":"2023-08-11T14:03:04.345962+0000","last_became_active":"2023-08-11T14:02:03.337711+0000","last_became_peered":"2023-08-11T14:02:03.337711+0000","last_unstale":"2023-08-11T14:03:04.345962+0000","last_undegraded":"2023-08-11T14:03:04.345962+0000","last_fullsized":"2023-08-11T14:03:04.345962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:54:46.950942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523374+0000","last_change":"2023-08-11T14:01:32.294886+0000","last_active":"2023-08-11T14:03:04.523374+0000","last_peered":"2023-08-11T14:03:04.523374+0000","last_clean":"2023-08-11T14:03:04.523374+0000","last_became_active":"2023-08-11T14:01:32.294592+0000","last_became_peered":"2023-08-11T14:01:32.294592+0000","last_unstale":"2023-08-11T14:03:04.523374+0000","last_undegraded":"2023-08-11T14:03:04.523374+0000","last_fullsized":"2023-08-11T14:03:04.523374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:40:06.480953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345005+0000","last_change":"2023-08-11T14:01:33.301353+0000","last_active":"2023-08-11T14:03:04.345005+0000","last_peered":"2023-08-11T14:03:04.345005+0000","last_clean":"2023-08-11T14:03:04.345005+0000","last_became_active":"2023-08-11T14:01:33.301054+0000","last_became_peered":"2023-08-11T14:01:33.301054+0000","last_unstale":"2023-08-11T14:03:04.345005+0000","last_undegraded":"2023-08-11T14:03:04.345005+0000","last_fullsized":"2023-08-11T14:03:04.345005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:27:58.380684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341596+0000","last_change":"2023-08-11T14:01:37.379864+0000","last_active":"2023-08-11T14:03:04.341596+0000","last_peered":"2023-08-11T14:03:04.341596+0000","last_clean":"2023-08-11T14:03:04.341596+0000","last_became_active":"2023-08-11T14:01:37.379422+0000","last_became_peered":"2023-08-11T14:01:37.379422+0000","last_unstale":"2023-08-11T14:03:04.341596+0000","last_undegraded":"2023-08-11T14:03:04.341596+0000","last_fullsized":"2023-08-11T14:03:04.341596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:55:48.675324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429201+0000","last_change":"2023-08-11T14:01:32.312827+0000","last_active":"2023-08-11T14:03:04.429201+0000","last_peered":"2023-08-11T14:03:04.429201+0000","last_clean":"2023-08-11T14:03:04.429201+0000","last_became_active":"2023-08-11T14:01:32.312483+0000","last_became_peered":"2023-08-11T14:01:32.312483+0000","last_unstale":"2023-08-11T14:03:04.429201+0000","last_undegraded":"2023-08-11T14:03:04.429201+0000","last_fullsized":"2023-08-11T14:03:04.429201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:40:03.363760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342610+0000","last_change":"2023-08-11T14:01:33.307815+0000","last_active":"2023-08-11T14:03:04.342610+0000","last_peered":"2023-08-11T14:03:04.342610+0000","last_clean":"2023-08-11T14:03:04.342610+0000","last_became_active":"2023-08-11T14:01:33.307057+0000","last_became_peered":"2023-08-11T14:01:33.307057+0000","last_unstale":"2023-08-11T14:03:04.342610+0000","last_undegraded":"2023-08-11T14:03:04.342610+0000","last_fullsized":"2023-08-11T14:03:04.342610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:33:37.384946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345770+0000","last_change":"2023-08-11T14:01:37.374400+0000","last_active":"2023-08-11T14:03:04.345770+0000","last_peered":"2023-08-11T14:03:04.345770+0000","last_clean":"2023-08-11T14:03:04.345770+0000","last_became_active":"2023-08-11T14:01:37.373971+0000","last_became_peered":"2023-08-11T14:01:37.373971+0000","last_unstale":"2023-08-11T14:03:04.345770+0000","last_undegraded":"2023-08-11T14:03:04.345770+0000","last_fullsized":"2023-08-11T14:03:04.345770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:40:08.705807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340884+0000","last_change":"2023-08-11T14:01:32.302804+0000","last_active":"2023-08-11T14:03:04.340884+0000","last_peered":"2023-08-11T14:03:04.340884+0000","last_clean":"2023-08-11T14:03:04.340884+0000","last_became_active":"2023-08-11T14:01:32.302215+0000","last_became_peered":"2023-08-11T14:01:32.302215+0000","last_unstale":"2023-08-11T14:03:04.340884+0000","last_undegraded":"2023-08-11T14:03:04.340884+0000","last_fullsized":"2023-08-11T14:03:04.340884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T01:33:52.040366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429419+0000","last_change":"2023-08-11T14:01:33.315850+0000","last_active":"2023-08-11T14:03:04.429419+0000","last_peered":"2023-08-11T14:03:04.429419+0000","last_clean":"2023-08-11T14:03:04.429419+0000","last_became_active":"2023-08-11T14:01:33.315310+0000","last_became_peered":"2023-08-11T14:01:33.315310+0000","last_unstale":"2023-08-11T14:03:04.429419+0000","last_undegraded":"2023-08-11T14:03:04.429419+0000","last_fullsized":"2023-08-11T14:03:04.429419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:58:31.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345789+0000","last_change":"2023-08-11T14:01:37.585881+0000","last_active":"2023-08-11T14:03:04.345789+0000","last_peered":"2023-08-11T14:03:04.345789+0000","last_clean":"2023-08-11T14:03:04.345789+0000","last_became_active":"2023-08-11T14:01:37.585398+0000","last_became_peered":"2023-08-11T14:01:37.585398+0000","last_unstale":"2023-08-11T14:03:04.345789+0000","last_undegraded":"2023-08-11T14:03:04.345789+0000","last_fullsized":"2023-08-11T14:03:04.345789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:51:14.187908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345206+0000","last_change":"2023-08-11T14:01:32.297972+0000","last_active":"2023-08-11T14:03:04.345206+0000","last_peered":"2023-08-11T14:03:04.345206+0000","last_clean":"2023-08-11T14:03:04.345206+0000","last_became_active":"2023-08-11T14:01:32.297664+0000","last_became_peered":"2023-08-11T14:01:32.297664+0000","last_unstale":"2023-08-11T14:03:04.345206+0000","last_undegraded":"2023-08-11T14:03:04.345206+0000","last_fullsized":"2023-08-11T14:03:04.345206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T19:21:37.922008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427322+0000","last_change":"2023-08-11T14:01:33.308185+0000","last_active":"2023-08-11T14:03:04.427322+0000","last_peered":"2023-08-11T14:03:04.427322+0000","last_clean":"2023-08-11T14:03:04.427322+0000","last_became_active":"2023-08-11T14:01:33.307887+0000","last_became_peered":"2023-08-11T14:01:33.307887+0000","last_unstale":"2023-08-11T14:03:04.427322+0000","last_undegraded":"2023-08-11T14:03:04.427322+0000","last_fullsized":"2023-08-11T14:03:04.427322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:19:55.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523362+0000","last_change":"2023-08-11T14:01:37.373354+0000","last_active":"2023-08-11T14:03:04.523362+0000","last_peered":"2023-08-11T14:03:04.523362+0000","last_clean":"2023-08-11T14:03:04.523362+0000","last_became_active":"2023-08-11T14:01:37.372897+0000","last_became_peered":"2023-08-11T14:01:37.372897+0000","last_unstale":"2023-08-11T14:03:04.523362+0000","last_undegraded":"2023-08-11T14:03:04.523362+0000","last_fullsized":"2023-08-11T14:03:04.523362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T21:36:36.812617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:49.502291+0000","last_change":"2023-08-11T14:01:32.302553+0000","last_active":"2023-08-11T14:03:49.502291+0000","last_peered":"2023-08-11T14:03:49.502291+0000","last_clean":"2023-08-11T14:03:49.502291+0000","last_became_active":"2023-08-11T14:01:32.302223+0000","last_became_peered":"2023-08-11T14:01:32.302223+0000","last_unstale":"2023-08-11T14:03:49.502291+0000","last_undegraded":"2023-08-11T14:03:49.502291+0000","last_fullsized":"2023-08-11T14:03:49.502291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:16:25.101731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427708+0000","last_change":"2023-08-11T14:01:33.305893+0000","last_active":"2023-08-11T14:03:04.427708+0000","last_peered":"2023-08-11T14:03:04.427708+0000","last_clean":"2023-08-11T14:03:04.427708+0000","last_became_active":"2023-08-11T14:01:33.305559+0000","last_became_peered":"2023-08-11T14:01:33.305559+0000","last_unstale":"2023-08-11T14:03:04.427708+0000","last_undegraded":"2023-08-11T14:03:04.427708+0000","last_fullsized":"2023-08-11T14:03:04.427708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:45:18.108583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522691+0000","last_change":"2023-08-11T14:01:37.376797+0000","last_active":"2023-08-11T14:03:04.522691+0000","last_peered":"2023-08-11T14:03:04.522691+0000","last_clean":"2023-08-11T14:03:04.522691+0000","last_became_active":"2023-08-11T14:01:37.376343+0000","last_became_peered":"2023-08-11T14:01:37.376343+0000","last_unstale":"2023-08-11T14:03:04.522691+0000","last_undegraded":"2023-08-11T14:03:04.522691+0000","last_fullsized":"2023-08-11T14:03:04.522691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:36:56.757871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.346920+0000","last_change":"2023-08-11T14:01:32.306584+0000","last_active":"2023-08-11T14:03:04.346920+0000","last_peered":"2023-08-11T14:03:04.346920+0000","last_clean":"2023-08-11T14:03:04.346920+0000","last_became_active":"2023-08-11T14:01:32.306274+0000","last_became_peered":"2023-08-11T14:01:32.306274+0000","last_unstale":"2023-08-11T14:03:04.346920+0000","last_undegraded":"2023-08-11T14:03:04.346920+0000","last_fullsized":"2023-08-11T14:03:04.346920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:15:44.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427304+0000","last_change":"2023-08-11T14:01:33.308799+0000","last_active":"2023-08-11T14:03:04.427304+0000","last_peered":"2023-08-11T14:03:04.427304+0000","last_clean":"2023-08-11T14:03:04.427304+0000","last_became_active":"2023-08-11T14:01:33.308462+0000","last_became_peered":"2023-08-11T14:01:33.308462+0000","last_unstale":"2023-08-11T14:03:04.427304+0000","last_undegraded":"2023-08-11T14:03:04.427304+0000","last_fullsized":"2023-08-11T14:03:04.427304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:53:39.819984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345073+0000","last_change":"2023-08-11T14:01:37.374511+0000","last_active":"2023-08-11T14:03:04.345073+0000","last_peered":"2023-08-11T14:03:04.345073+0000","last_clean":"2023-08-11T14:03:04.345073+0000","last_became_active":"2023-08-11T14:01:37.371209+0000","last_became_peered":"2023-08-11T14:01:37.371209+0000","last_unstale":"2023-08-11T14:03:04.345073+0000","last_undegraded":"2023-08-11T14:03:04.345073+0000","last_fullsized":"2023-08-11T14:03:04.345073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:36.338435+0000","last_clean_scrub_stamp":"2023-08-11T14:01:36.338435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:02:05.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340636+0000","last_change":"2023-08-11T14:01:32.312251+0000","last_active":"2023-08-11T14:03:04.340636+0000","last_peered":"2023-08-11T14:03:04.340636+0000","last_clean":"2023-08-11T14:03:04.340636+0000","last_became_active":"2023-08-11T14:01:32.311964+0000","last_became_peered":"2023-08-11T14:01:32.311964+0000","last_unstale":"2023-08-11T14:03:04.340636+0000","last_undegraded":"2023-08-11T14:03:04.340636+0000","last_fullsized":"2023-08-11T14:03:04.340636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:31.253737+0000","last_clean_scrub_stamp":"2023-08-11T14:01:31.253737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:12:43.294958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.427646+0000","last_change":"2023-08-11T14:01:33.306004+0000","last_active":"2023-08-11T14:03:04.427646+0000","last_peered":"2023-08-11T14:03:04.427646+0000","last_clean":"2023-08-11T14:03:04.427646+0000","last_became_active":"2023-08-11T14:01:33.305690+0000","last_became_peered":"2023-08-11T14:01:33.305690+0000","last_unstale":"2023-08-11T14:03:04.427646+0000","last_undegraded":"2023-08-11T14:03:04.427646+0000","last_fullsized":"2023-08-11T14:03:04.427646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-13T00:13:53.158734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342712+0000","last_change":"2023-08-11T14:01:33.308042+0000","last_active":"2023-08-11T14:03:04.342712+0000","last_peered":"2023-08-11T14:03:04.342712+0000","last_clean":"2023-08-11T14:03:04.342712+0000","last_became_active":"2023-08-11T14:01:33.307505+0000","last_became_peered":"2023-08-11T14:01:33.307505+0000","last_unstale":"2023-08-11T14:03:04.342712+0000","last_undegraded":"2023-08-11T14:03:04.342712+0000","last_fullsized":"2023-08-11T14:03:04.342712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:13:26.491599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340809+0000","last_change":"2023-08-11T14:01:33.307198+0000","last_active":"2023-08-11T14:03:04.340809+0000","last_peered":"2023-08-11T14:03:04.340809+0000","last_clean":"2023-08-11T14:03:04.340809+0000","last_became_active":"2023-08-11T14:01:33.306898+0000","last_became_peered":"2023-08-11T14:01:33.306898+0000","last_unstale":"2023-08-11T14:03:04.340809+0000","last_undegraded":"2023-08-11T14:03:04.340809+0000","last_fullsized":"2023-08-11T14:03:04.340809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:27:39.383017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340630+0000","last_change":"2023-08-11T14:01:33.312239+0000","last_active":"2023-08-11T14:03:04.340630+0000","last_peered":"2023-08-11T14:03:04.340630+0000","last_clean":"2023-08-11T14:03:04.340630+0000","last_became_active":"2023-08-11T14:01:33.311941+0000","last_became_peered":"2023-08-11T14:01:33.311941+0000","last_unstale":"2023-08-11T14:03:04.340630+0000","last_undegraded":"2023-08-11T14:03:04.340630+0000","last_fullsized":"2023-08-11T14:03:04.340630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T15:09:51.511408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.522979+0000","last_change":"2023-08-11T14:01:33.307701+0000","last_active":"2023-08-11T14:03:04.522979+0000","last_peered":"2023-08-11T14:03:04.522979+0000","last_clean":"2023-08-11T14:03:04.522979+0000","last_became_active":"2023-08-11T14:01:33.307317+0000","last_became_peered":"2023-08-11T14:01:33.307317+0000","last_unstale":"2023-08-11T14:03:04.522979+0000","last_undegraded":"2023-08-11T14:03:04.522979+0000","last_fullsized":"2023-08-11T14:03:04.522979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T17:38:58.624580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340049+0000","last_change":"2023-08-11T14:01:33.303666+0000","last_active":"2023-08-11T14:03:04.340049+0000","last_peered":"2023-08-11T14:03:04.340049+0000","last_clean":"2023-08-11T14:03:04.340049+0000","last_became_active":"2023-08-11T14:01:33.303292+0000","last_became_peered":"2023-08-11T14:01:33.303292+0000","last_unstale":"2023-08-11T14:03:04.340049+0000","last_undegraded":"2023-08-11T14:03:04.340049+0000","last_fullsized":"2023-08-11T14:03:04.340049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:10:16.822501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.343066+0000","last_change":"2023-08-11T14:01:33.298343+0000","last_active":"2023-08-11T14:03:04.343066+0000","last_peered":"2023-08-11T14:03:04.343066+0000","last_clean":"2023-08-11T14:03:04.343066+0000","last_became_active":"2023-08-11T14:01:33.297360+0000","last_became_peered":"2023-08-11T14:01:33.297360+0000","last_unstale":"2023-08-11T14:03:04.343066+0000","last_undegraded":"2023-08-11T14:03:04.343066+0000","last_fullsized":"2023-08-11T14:03:04.343066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T16:47:34.479972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.341526+0000","last_change":"2023-08-11T14:01:33.307099+0000","last_active":"2023-08-11T14:03:04.341526+0000","last_peered":"2023-08-11T14:03:04.341526+0000","last_clean":"2023-08-11T14:03:04.341526+0000","last_became_active":"2023-08-11T14:01:33.306764+0000","last_became_peered":"2023-08-11T14:01:33.306764+0000","last_unstale":"2023-08-11T14:03:04.341526+0000","last_undegraded":"2023-08-11T14:03:04.341526+0000","last_fullsized":"2023-08-11T14:03:04.341526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:23:56.188787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429642+0000","last_change":"2023-08-11T14:01:33.306112+0000","last_active":"2023-08-11T14:03:04.429642+0000","last_peered":"2023-08-11T14:03:04.429642+0000","last_clean":"2023-08-11T14:03:04.429642+0000","last_became_active":"2023-08-11T14:01:33.305676+0000","last_became_peered":"2023-08-11T14:01:33.305676+0000","last_unstale":"2023-08-11T14:03:04.429642+0000","last_undegraded":"2023-08-11T14:03:04.429642+0000","last_fullsized":"2023-08-11T14:03:04.429642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T23:01:08.200835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.345624+0000","last_change":"2023-08-11T14:01:33.312110+0000","last_active":"2023-08-11T14:03:04.345624+0000","last_peered":"2023-08-11T14:03:04.345624+0000","last_clean":"2023-08-11T14:03:04.345624+0000","last_became_active":"2023-08-11T14:01:33.311809+0000","last_became_peered":"2023-08-11T14:01:33.311809+0000","last_unstale":"2023-08-11T14:03:04.345624+0000","last_undegraded":"2023-08-11T14:03:04.345624+0000","last_fullsized":"2023-08-11T14:03:04.345624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T22:04:08.040487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.429411+0000","last_change":"2023-08-11T14:01:33.316220+0000","last_active":"2023-08-11T14:03:04.429411+0000","last_peered":"2023-08-11T14:03:04.429411+0000","last_clean":"2023-08-11T14:03:04.429411+0000","last_became_active":"2023-08-11T14:01:33.315710+0000","last_became_peered":"2023-08-11T14:01:33.315710+0000","last_unstale":"2023-08-11T14:03:04.429411+0000","last_undegraded":"2023-08-11T14:03:04.429411+0000","last_fullsized":"2023-08-11T14:03:04.429411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T20:43:43.729999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.523915+0000","last_change":"2023-08-11T14:01:33.303801+0000","last_active":"2023-08-11T14:03:04.523915+0000","last_peered":"2023-08-11T14:03:04.523915+0000","last_clean":"2023-08-11T14:03:04.523915+0000","last_became_active":"2023-08-11T14:01:33.303462+0000","last_became_peered":"2023-08-11T14:01:33.303462+0000","last_unstale":"2023-08-11T14:03:04.523915+0000","last_undegraded":"2023-08-11T14:03:04.523915+0000","last_fullsized":"2023-08-11T14:03:04.523915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:25:07.770441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.340919+0000","last_change":"2023-08-11T14:01:33.303444+0000","last_active":"2023-08-11T14:03:04.340919+0000","last_peered":"2023-08-11T14:03:04.340919+0000","last_clean":"2023-08-11T14:03:04.340919+0000","last_became_active":"2023-08-11T14:01:33.303138+0000","last_became_peered":"2023-08-11T14:01:33.303138+0000","last_unstale":"2023-08-11T14:03:04.340919+0000","last_undegraded":"2023-08-11T14:03:04.340919+0000","last_fullsized":"2023-08-11T14:03:04.340919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T14:52:14.465240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-08-11T14:03:04.342974+0000","last_change":"2023-08-11T14:01:33.296943+0000","last_active":"2023-08-11T14:03:04.342974+0000","last_peered":"2023-08-11T14:03:04.342974+0000","last_clean":"2023-08-11T14:03:04.342974+0000","last_became_active":"2023-08-11T14:01:33.296626+0000","last_became_peered":"2023-08-11T14:01:33.296626+0000","last_unstale":"2023-08-11T14:03:04.342974+0000","last_undegraded":"2023-08-11T14:03:04.342974+0000","last_fullsized":"2023-08-11T14:03:04.342974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-11T14:01:32.273793+0000","last_clean_scrub_stamp":"2023-08-11T14:01:32.273793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-12T18:52:01.425835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73338,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1155,"num_read_kb":664,"num_write":250,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":214635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449128,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":509222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.63},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.71599999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.56999999999999995,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55100000000000005,"15min":0.55600000000000005},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79600000000000004,"5min":0.88,"15min":0.88},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.44,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79600000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92400000000000004,"5min":1.296,"15min":1.296},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79600000000000004},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92700000000000005,"5min":1.208,"15min":1.208},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.61899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.048,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.876,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84599999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80600000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.876},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.006}]},{"osd":9,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0109999999999999,"5min":1.27,"15min":1.27},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.66700000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0720000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98399999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.55600000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.97399999999999998,"5min":1.139,"15min":1.139},"last":0.93500000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678060,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1162864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.278,"5min":1.526,"15min":1.526},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52400000000000002,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.117,"5min":1.516,"15min":1.516},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.873,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92400000000000004,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.70399999999999996},"min":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.74299999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":0.33700000000000002}]},{"osd":5,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.378,"15min":0.378},"max":{"1min":0.97799999999999998,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.438},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.085,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.77800000000000002,"15min":0.81100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.012,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.77900000000000003,"15min":0.78800000000000003},"min":{"1min":0.57199999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95899999999999996,"5min":1.605,"15min":1.605},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.65000000000000002}]},{"osd":8,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81200000000000006,"15min":0.82799999999999996},"min":{"1min":0.55100000000000005,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98499999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84499999999999997,"15min":0.88400000000000001},"min":{"1min":0.52600000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.92400000000000004,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.48199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.92100000000000004,"15min":0.90300000000000002},"min":{"1min":0.67000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":6.9829999999999997,"5min":6.9829999999999997,"15min":6.9829999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.55200000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95199999999999996,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.58399999999999996}]},{"osd":11,"last update":"Fri Aug 11 14:03:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.65700000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94999999999999996,"5min":1.234,"15min":1.234},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.79900000000000004,"15min":0.78100000000000003},"min":{"1min":0.748,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0229999999999999,"5min":1.129,"15min":1.129},"last":0.54300000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":513158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.32300000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.58999999999999997,"15min":0.63200000000000001},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66800000000000004,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71999999999999997,"15min":0.748},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85199999999999998,"5min":1,"15min":1},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.53700000000000003,"5min":0.44,"15min":0.44},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.51600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69799999999999995,"15min":0.746},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89800000000000002,"5min":1.123,"15min":1.123},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.74199999999999999},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92700000000000005,"5min":1.165,"15min":1.165},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.85599999999999998,"15min":0.90000000000000002},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.2149999999999999,"5min":3.2149999999999999,"15min":3.2149999999999999},"last":0.75},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.86499999999999999,"15min":0.91900000000000004},"min":{"1min":0.53000000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.95799999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.89800000000000002,"15min":0.95299999999999996},"min":{"1min":0.60299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1040000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:01:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.93999999999999995,"15min":1.018},"min":{"1min":0.505,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.94799999999999995,"15min":1.0229999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0309999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.878}]},{"osd":8,"last update":"Fri Aug 11 14:03:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.079,"5min":1.484,"15min":1.484},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.97799999999999998,"15min":1.042},"min":{"1min":0.59699999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0580000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.629}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.94799999999999995,"15min":0.96799999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.96099999999999997,"15min":0.99099999999999999},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":0.84099999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.88600000000000001,"15min":0.86199999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.86799999999999999,"15min":0.84099999999999997},"min":{"1min":0.72299999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.68799999999999994}]}]},{"osd":8,"up_from":58,"seq":249108103223,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":509418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.625,"15min":0.623},"min":{"1min":0.47099999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.496,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":5.3159999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.53200000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.56399999999999995,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85799999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.626}]},{"osd":2,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72799999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.40899999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.39900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.60499999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":1.022,"15min":1.022},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.56399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.501,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61899999999999999,"15min":0.59699999999999998},"min":{"1min":0.46400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.91000000000000003,"5min":1.101,"15min":1.101},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.79300000000000004,"15min":0.75900000000000001},"min":{"1min":0.628,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.65300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.042,"15min":1.042},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Aug 11 14:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.67300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98599999999999999,"5min":1.083,"15min":1.083},"last":0.877},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.746},"min":{"1min":0.58199999999999996,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96999999999999997,"5min":1.29,"15min":1.29},"last":0.66600000000000004}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.78800000000000003,"15min":0.751},"min":{"1min":0.79300000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0509999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.82799999999999996,"15min":0.79800000000000004},"min":{"1min":0.79700000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73899999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.75800000000000001,"15min":0.70799999999999996},"min":{"1min":0.72799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.79800000000000004,"15min":0.754},"min":{"1min":0.78900000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77300000000000002}]},{"osd":11,"last update":"Fri Aug 11 14:03:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.84799999999999998,"15min":0.84199999999999997},"min":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.91800000000000004,"15min":0.92400000000000004},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.02,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71699999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1105804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.42399999999999999},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56799999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46600000000000003,"15min":0.442},"min":{"1min":0.34300000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66800000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82399999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.625,"15min":0.60699999999999998},"min":{"1min":0.47699999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.64400000000000002,"15min":0.59399999999999997},"min":{"1min":0.47699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.6130000000000004,"5min":4.6130000000000004,"15min":4.6130000000000004},"last":0.504},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.42999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.753,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.377},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.85699999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.56899999999999995,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83899999999999997,"5min":1.034,"15min":1.034},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":3.6539999999999999,"5min":3.6539999999999999,"15min":3.6539999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2310000000000001,"5min":0.82799999999999996,"15min":0.73399999999999999},"min":{"1min":0.505,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.798,"5min":6.798,"15min":6.798},"last":0.59899999999999998}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.0129999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.752,"15min":0.72899999999999998},"min":{"1min":0.66000000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.96899999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.6050000000000004,"5min":5.6050000000000004,"15min":5.6050000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.86099999999999999,"15min":0.83899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97399999999999998,"5min":1.095,"15min":1.095},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.60099999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.88900000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66700000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85199999999999998,"15min":0.85599999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.67600000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.95599999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77600000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":504566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.499,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.3639999999999999,"5min":3.3639999999999999,"15min":3.3639999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71499999999999997,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.73399999999999999,"15min":0.70399999999999996},"min":{"1min":0.52300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.903,"5min":3.903,"15min":3.903},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74399999999999999,"15min":0.73499999999999999},"min":{"1min":0.52800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90300000000000002,"5min":1.131,"15min":1.131},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54700000000000004,"5min":0.433,"15min":0.433},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.133,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.82299999999999995,"15min":0.80700000000000005},"min":{"1min":0.48899999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8290000000000002,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Aug 11 14:03:24 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.81399999999999995,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.70299999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1339999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.92700000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.84899999999999998,"15min":0.88200000000000001},"min":{"1min":0.54100000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.9220000000000002,"5min":3.9220000000000002,"15min":3.9220000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.55700000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9420000000000002,"5min":3.9420000000000002,"15min":3.9420000000000002},"last":0.66300000000000003}]},{"osd":9,"last update":"Fri Aug 11 14:03:05 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.93999999999999995,"15min":0.95099999999999996},"min":{"1min":0.63900000000000001,"5min":0.5,"15min":0.5},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":3.9580000000000002,"5min":3.9580000000000002,"15min":3.9580000000000002},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.86899999999999999,"15min":0.85299999999999998},"min":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.7909999999999999,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.92100000000000004,"15min":0.92600000000000005},"min":{"1min":0.77600000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.1519999999999999,"5min":1.363,"15min":1.363},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Aug 11 14:03:40 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.91400000000000003,"15min":0.89600000000000002},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.98,"5min":3.98,"15min":3.98},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.93799999999999994,"15min":0.92000000000000004},"min":{"1min":0.72399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.90800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":571708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.42799999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.054},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82299999999999995,"5min":1.234,"15min":1.234},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.63400000000000001,"15min":0.54800000000000004},"min":{"1min":0.48299999999999998,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.95899999999999996,"5min":0.995,"15min":0.995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.55900000000000005},"min":{"1min":0.50700000000000001,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.90900000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:02:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64000000000000001,"15min":0.58599999999999997},"min":{"1min":0.51300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.60599999999999998},"min":{"1min":0.55800000000000005,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0720000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.625}]},{"osd":4,"last update":"Fri Aug 11 14:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.48999999999999999,"15min":0.41799999999999998},"min":{"1min":0.36299999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80300000000000005,"5min":1.119,"15min":1.119},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.47099999999999997,"15min":0.43099999999999999},"min":{"1min":0.39500000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.65600000000000003,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96899999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.69299999999999995,"15min":0.59899999999999998},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94399999999999995,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:02:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.61799999999999999},"min":{"1min":0.54300000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99299999999999999,"5min":3.153,"15min":3.153},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.69299999999999995,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0349999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63400000000000001,"15min":0.55600000000000005},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.089,"5min":1.159,"15min":1.159},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.67500000000000004,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.115,"5min":1.345,"15min":1.345},"last":0.61299999999999999}]},{"osd":8,"last update":"Fri Aug 11 14:03:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.76700000000000002,"15min":0.70899999999999996},"min":{"1min":0.73599999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1799999999999999,"5min":1.302,"15min":1.302},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78000000000000003,"15min":0.71599999999999997},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.131,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69599999999999995}]},{"osd":9,"last update":"Fri Aug 11 14:03:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.641,"5min":3.194,"15min":3.194},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.85399999999999998,"15min":0.83999999999999997},"min":{"1min":0.63300000000000001,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1839999999999999,"5min":1.393,"15min":1.393},"last":0.751}]},{"osd":10,"last update":"Fri Aug 11 14:03:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84999999999999998,"15min":0.84799999999999998},"min":{"1min":0.59099999999999997,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.091,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.623,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.073,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91800000000000004,"15min":0.91400000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.151,"5min":1.21,"15min":1.21},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":506022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64500000000000002,"15min":0.59099999999999997},"min":{"1min":0.51900000000000002,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.002,"5min":0.70599999999999996,"15min":0.621},"min":{"1min":0.495,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66800000000000004,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.38200000000000001,"15min":0.35399999999999998},"max":{"1min":0.93700000000000006,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76000000000000001,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":1.724,"5min":6.2910000000000004,"15min":6.2910000000000004},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67300000000000004,"15min":0.68899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0049999999999999,"5min":3.3279999999999998,"15min":3.3279999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Aug 11 14:03:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.72999999999999998,"15min":0.627},"min":{"1min":0.496,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.1619999999999999,"5min":6.1619999999999999,"15min":6.1619999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.68999999999999995,"15min":0.60599999999999998},"min":{"1min":0.54900000000000004,"5min":0.317,"15min":0.317},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79100000000000004,"15min":0.72499999999999998},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.8029999999999999,"5min":6.25,"15min":6.25},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0169999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.52300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79000000000000004,"15min":0.73999999999999999},"min":{"1min":0.45600000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.1869999999999994,"5min":8.1869999999999994,"15min":8.1869999999999994},"last":0.755}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.93300000000000005,"15min":0.94799999999999995},"min":{"1min":0.68700000000000006,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.091,"5min":3.254,"15min":3.254},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.069,"5min":1.234,"15min":1.234},"last":0.67800000000000005}]},{"osd":8,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.63900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0669999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.91100000000000003,"15min":0.86099999999999999},"min":{"1min":0.55600000000000005,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.093,"5min":6.3339999999999996,"15min":6.3339999999999996},"last":0.98199999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":0.92400000000000004,"15min":0.90500000000000003},"min":{"1min":0.625,"5min":0.376,"15min":0.376},"max":{"1min":6.742,"5min":6.742,"15min":6.742},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.91500000000000004,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.871}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.93999999999999995,"15min":0.88600000000000001},"min":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.7629999999999999,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.92500000000000004,"15min":0.90000000000000002},"min":{"1min":0.69699999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81799999999999995}]},{"osd":11,"last update":"Fri Aug 11 14:03:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.77200000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.153,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.96099999999999997,"15min":0.90500000000000003},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.177,"5min":6.3739999999999997,"15min":6.3739999999999997},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":566754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.60799999999999998,"15min":0.53400000000000003},"min":{"1min":0.38200000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.0780000000000003,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33000000000000002},"max":{"1min":0.82399999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57199999999999995}]},{"osd":1,"last update":"Fri Aug 11 14:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59499999999999997,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.30099999999999999},"max":{"1min":0.92300000000000004,"5min":1.036,"15min":1.036},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.60599999999999998,"15min":0.53400000000000003},"min":{"1min":0.41899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.92500000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81899999999999995}]},{"osd":3,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.624,"15min":0.57399999999999995},"min":{"1min":0.46300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.86299999999999999,"5min":0.876,"15min":0.876},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.56799999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.41999999999999998},"min":{"1min":0.41499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":3.218,"15min":3.218},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.44},"min":{"1min":0.38100000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.72199999999999998,"15min":0.68200000000000005},"min":{"1min":0.57299999999999995,"5min":0.436,"15min":0.436},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.74099999999999999},"min":{"1min":0.5,"5min":0.439,"15min":0.439},"max":{"1min":0.90400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.80000000000000004,"15min":0.79400000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Aug 11 14:03:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.65100000000000002,"15min":0.58699999999999997},"min":{"1min":0.48799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.62},"min":{"1min":0.59799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.006,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.63}]},{"osd":9,"last update":"Fri Aug 11 14:03:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.67900000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77800000000000002,"15min":0.73199999999999998},"min":{"1min":0.57999999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1120000000000001,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.57399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":0.94999999999999996,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77900000000000003,"15min":0.745},"min":{"1min":0.60399999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Aug 11 14:03:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81299999999999994,"15min":0.80900000000000005},"min":{"1min":0.63900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.84099999999999997,"15min":0.83899999999999997},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0680000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76200000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":504274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48399999999999999,"15min":0.42199999999999999},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.02,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.35199999999999998}]},{"osd":1,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.68600000000000005,"15min":0.60499999999999998},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.423,"5min":6.423,"15min":6.423},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.61299999999999999},"min":{"1min":0.57199999999999995,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":2.548,"15min":2.548},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Aug 11 14:02:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71599999999999997,"15min":0.67000000000000004},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.753},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.72499999999999998,"15min":0.65500000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":6.5010000000000003,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69299999999999995,"15min":0.627},"min":{"1min":0.51400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2849999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.2010000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.747,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Aug 11 14:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.751,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.66300000000000003,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Aug 11 14:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.89400000000000002,"15min":0.83399999999999996},"min":{"1min":0.65900000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":8.6370000000000005,"5min":8.6370000000000005,"15min":8.6370000000000005},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Aug 11 14:00:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96199999999999997,"15min":0.996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.88200000000000001,"15min":0.92800000000000005},"min":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Aug 11 14:03:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.92200000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.179,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91500000000000004,"15min":0.92700000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.1879999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.92200000000000004}]},{"osd":11,"last update":"Fri Aug 11 14:03:38 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.91300000000000003,"15min":0.92000000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.254,"5min":1.379,"15min":1.379},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.169,"5min":0.89200000000000002,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":6.4610000000000003,"5min":6.4610000000000003,"15min":6.4610000000000003},"last":0.98799999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":509171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.374},"min":{"1min":0.33000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.61199999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.38600000000000001},"min":{"1min":0.36599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65800000000000003,"5min":0.875,"15min":0.875},"last":0.376}]},{"osd":2,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Aug 11 14:03:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64900000000000002,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79100000000000004,"5min":5.0519999999999996,"15min":5.0519999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60399999999999998},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.436},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.59899999999999998},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83499999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Aug 11 14:03:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Aug 11 14:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.36099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.89800000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.496}]},{"osd":8,"last update":"Fri Aug 11 14:03:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67200000000000004,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.98899999999999999,"5min":1.02,"15min":1.02},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Aug 11 14:03:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76600000000000001,"15min":0.752},"min":{"1min":0.65500000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71099999999999997,"15min":0.70699999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.71499999999999997}]},{"osd":10,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.63,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0409999999999999,"5min":1.075,"15min":1.075},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Aug 11 14:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.64100000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97799999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.67500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78100000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1101128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":1.46,"15min":1.46},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.38,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.52800000000000002},"min":{"1min":0.46899999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82199999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.56100000000000005,"5min":0.318,"15min":0.318},"max":{"1min":0.78000000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.39900000000000002},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.59899999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.442,"15min":0.42299999999999999},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.409,"15min":1.409},"last":0.626},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":3.4900000000000002,"15min":3.4900000000000002},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.63200000000000001,"15min":0.57699999999999996},"min":{"1min":0.5,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.61499999999999999,"15min":0.58099999999999996},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Aug 11 14:02:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59599999999999997},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":1.393,"15min":1.393},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67700000000000005,"15min":0.64300000000000002},"min":{"1min":0.50700000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89400000000000002,"5min":1.484,"15min":1.484},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Aug 11 14:03:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.46600000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.878},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.82599999999999996}]},{"osd":8,"last update":"Fri Aug 11 14:03:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.71899999999999997}]},{"osd":9,"last update":"Fri Aug 11 14:03:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.60499999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98999999999999999,"5min":1.181,"15min":1.181},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.78500000000000003,"15min":0.76100000000000001},"min":{"1min":0.58799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.89800000000000002}]},{"osd":10,"last update":"Fri Aug 11 14:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72499999999999998,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96899999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-11T14:03:55.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 -- 172.21.15.3:0/1689763367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e78101b30 msgr2=0x7f7e78101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/1689763367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e78101b30 0x7f7e78101f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009990 tx=0x7f7e64009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 -- 172.21.15.3:0/1689763367 shutdown_connections 2023-08-11T14:03:55.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/1689763367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e781024d0 0x7f7e78108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/1689763367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e78100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/1689763367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e78101b30 0x7f7e78101f90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 -- 172.21.15.3:0/1689763367 >> 172.21.15.3:0/1689763367 conn(0x7f7e780fbe90 msgr2=0x7f7e780fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:55.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 -- 172.21.15.3:0/1689763367 shutdown_connections 2023-08-11T14:03:55.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.675+0000 7f7e7ebff700 1 -- 172.21.15.3:0/1689763367 wait complete. 2023-08-11T14:03:55.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:55 smithi103 ceph-mon[130434]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:03:55.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:55 smithi103 ceph-mon[130434]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:55.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.676+0000 7f7e7ebff700 1 Processor -- start 2023-08-11T14:03:55.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 -- start start 2023-08-11T14:03:55.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e78101b30 0x7f7e78072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 0x7f7e78073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780739f0 con 0x7f7e78101b30 2023-08-11T14:03:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78073b60 con 0x7f7e78100940 2023-08-11T14:03:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7ebff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781a0620 con 0x7f7e781024d0 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7c99b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7c99b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48818/0 (socket says 172.21.15.3:48818) 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7c99b700 1 -- 172.21.15.3:0/3960364722 learned_addr learned my addr 172.21.15.3:0/3960364722 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e77fff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e78101b30 0x7f7e78072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.677+0000 7f7e7d19c700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 0x7f7e78073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 -- 172.21.15.3:0/3960364722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 msgr2=0x7f7e78073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 0x7f7e78073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 -- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e78101b30 msgr2=0x7f7e78072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e78101b30 0x7f7e78072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 -- 172.21.15.3:0/3960364722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640095f0 con 0x7f7e78100940 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7d19c700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 0x7f7e78073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7c99b700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00d980 tx=0x7f7e6c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c004020 con 0x7f7e78100940 2023-08-11T14:03:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781a0900 con 0x7f7e78100940 2023-08-11T14:03:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6c009730 con 0x7f7e78100940 2023-08-11T14:03:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003a60 con 0x7f7e78100940 2023-08-11T14:03:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.678+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e781a0e60 con 0x7f7e78100940 2023-08-11T14:03:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.679+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78107e90 con 0x7f7e78100940 2023-08-11T14:03:55.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.682+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7e6c003bc0 con 0x7f7e78100940 2023-08-11T14:03:55.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.683+0000 7f7e75ffb700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 0x7f7e680a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:55.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.683+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7e6c0caf80 con 0x7f7e78100940 2023-08-11T14:03:55.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.683+0000 7f7e77fff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 0x7f7e680a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:55.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.684+0000 7f7e77fff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 0x7f7e680a2460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b2e0 tx=0x7f7e64005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:55.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.687+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f7e6c094990 con 0x7f7e78100940 2023-08-11T14:03:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:55 smithi003 ceph-mon[97195]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:03:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:55 smithi003 ceph-mon[97195]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:55.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:55 smithi187 ceph-mon[130973]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 11 op/s 2023-08-11T14:03:55.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:55 smithi187 ceph-mon[130973]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:55.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.918+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e7810c1a0 con 0x7f7e78100940 2023-08-11T14:03:55.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.920+0000 7f7e75ffb700 1 -- 172.21.15.3:0/3960364722 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7f7e6c097fb0 con 0x7f7e78100940 2023-08-11T14:03:55.922 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:55.922 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 msgr2=0x7f7e680a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 0x7f7e680a2460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b2e0 tx=0x7f7e64005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 msgr2=0x7f7e780746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00d980 tx=0x7f7e6c00dc90 comp rx=0 tx=0).stop 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 shutdown_connections 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e781024d0 0x7f7e78073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7e6809ffa0 0x7f7e680a2460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e78100940 0x7f7e780746d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 --2- 172.21.15.3:0/3960364722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7e78101b30 0x7f7e78072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 >> 172.21.15.3:0/3960364722 conn(0x7f7e780fbe90 msgr2=0x7f7e78104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 shutdown_connections 2023-08-11T14:03:55.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:55.923+0000 7f7e7ebff700 1 -- 172.21.15.3:0/3960364722 wait complete. 2023-08-11T14:03:55.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:03:55.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 -- 172.21.15.3:0/684769473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344100910 msgr2=0x7fc344100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/684769473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344100910 0x7fc344100d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc334009990 tx=0x7fc334009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 -- 172.21.15.3:0/684769473 shutdown_connections 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/684769473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 0x7fc344108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/684769473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344100910 0x7fc344100d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/684769473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344101b00 0x7fc344101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 -- 172.21.15.3:0/684769473 >> 172.21.15.3:0/684769473 conn(0x7fc3440fbe60 msgr2=0x7fc3440fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 -- 172.21.15.3:0/684769473 shutdown_connections 2023-08-11T14:03:56.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.180+0000 7fc34a1ff700 1 -- 172.21.15.3:0/684769473 wait complete. 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.181+0000 7fc34a1ff700 1 Processor -- start 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.181+0000 7fc34a1ff700 1 -- start start 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344100910 0x7fc344198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 0x7fc34419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a03e0 con 0x7fc344100910 2023-08-11T14:03:56.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a0520 con 0x7fc3441024a0 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc34a1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441a06b0 con 0x7fc344101b00 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc3437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344100910 0x7fc344198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60716/0 (socket says 172.21.15.3:60716) 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 -- 172.21.15.3:0/2117957584 learned_addr learned my addr 172.21.15.3:0/2117957584 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc343fff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 0x7fc34419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 -- 172.21.15.3:0/2117957584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 msgr2=0x7fc34419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 0x7fc34419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 -- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344100910 msgr2=0x7fc344198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344100910 0x7fc344198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.182+0000 7fc342ffd700 1 -- 172.21.15.3:0/2117957584 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340095f0 con 0x7fc344101b00 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc342ffd700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc33800e9a0 tx=0x7fc33800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338009790 con 0x7fc344101b00 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3441a0990 con 0x7fc344101b00 2023-08-11T14:03:56.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc338010430 con 0x7fc344101b00 2023-08-11T14:03:56.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3441a0ef0 con 0x7fc344101b00 2023-08-11T14:03:56.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.183+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338018680 con 0x7fc344101b00 2023-08-11T14:03:56.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.184+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344192f60 con 0x7fc344101b00 2023-08-11T14:03:56.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.186+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc338010810 con 0x7fc344101b00 2023-08-11T14:03:56.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.188+0000 7fc340ff9700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 0x7fc32c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.188+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc338014070 con 0x7fc344101b00 2023-08-11T14:03:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.188+0000 7fc3437fe700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 0x7fc32c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.188+0000 7fc3437fe700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 0x7fc32c0a2490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b2e0 tx=0x7fc334005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:56.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.191+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fc338051c30 con 0x7fc344101b00 2023-08-11T14:03:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.422+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc34402cc80 con 0x7fc344101b00 2023-08-11T14:03:56.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.423+0000 7fc340ff9700 1 -- 172.21.15.3:0/2117957584 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc338095130 con 0x7fc344101b00 2023-08-11T14:03:56.425 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:03:56.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.425+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 msgr2=0x7fc32c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.425+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 0x7fc32c0a2490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc33400b2e0 tx=0x7fc334005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.425+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 msgr2=0x7fc344198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.425+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc33800e9a0 tx=0x7fc33800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 shutdown_connections 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc32c09ffd0 0x7fc32c0a2490 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3441024a0 0x7fc34419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc344100910 0x7fc344198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 --2- 172.21.15.3:0/2117957584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc344101b00 0x7fc344198da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 >> 172.21.15.3:0/2117957584 conn(0x7fc3440fbe60 msgr2=0x7fc344107420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:56.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 shutdown_connections 2023-08-11T14:03:56.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.426+0000 7fc34a1ff700 1 -- 172.21.15.3:0/2117957584 wait complete. 2023-08-11T14:03:56.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:03:56.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.663+0000 7f02b61ff700 1 -- 172.21.15.3:0/3156465732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b01065f0 msgr2=0x7f02b0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.663+0000 7f02b61ff700 1 --2- 172.21.15.3:0/3156465732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b01065f0 0x7f02b0108a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009f90 tx=0x7f02a800b330 comp rx=0 tx=0).stop 2023-08-11T14:03:56.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 -- 172.21.15.3:0/3156465732 shutdown_connections 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 --2- 172.21.15.3:0/3156465732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b01065f0 0x7f02b0108a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 --2- 172.21.15.3:0/3156465732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 0x7f02b0103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 --2- 172.21.15.3:0/3156465732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b0103cc0 0x7f02b01060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 -- 172.21.15.3:0/3156465732 >> 172.21.15.3:0/3156465732 conn(0x7f02b00fad80 msgr2=0x7f02b00fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 -- 172.21.15.3:0/3156465732 shutdown_connections 2023-08-11T14:03:56.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.664+0000 7f02b61ff700 1 -- 172.21.15.3:0/3156465732 wait complete. 2023-08-11T14:03:56.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.665+0000 7f02b61ff700 1 Processor -- start 2023-08-11T14:03:56.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.665+0000 7f02b61ff700 1 -- start start 2023-08-11T14:03:56.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 0x7f02b0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b01065f0 0x7f02b019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01a0480 con 0x7f02b0101390 2023-08-11T14:03:56.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01a05c0 con 0x7f02b0103cc0 2023-08-11T14:03:56.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02b61ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01a0750 con 0x7f02b01065f0 2023-08-11T14:03:56.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02af7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 0x7f02b0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48860/0 (socket says 172.21.15.3:48860) 2023-08-11T14:03:56.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02aeffd700 1 -- 172.21.15.3:0/2488129182 learned_addr learned my addr 172.21.15.3:0/2488129182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:56.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.666+0000 7f02affff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b01065f0 0x7f02b019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 -- 172.21.15.3:0/2488129182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b01065f0 msgr2=0x7f02b019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b01065f0 0x7f02b019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 -- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 msgr2=0x7f02b0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 0x7f02b0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 -- 172.21.15.3:0/2488129182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00095f0 con 0x7f02b0103cc0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02aeffd700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02a400e9a0 tx=0x7f02a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4009790 con 0x7f02b0103cc0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a4010430 con 0x7f02b0103cc0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a8009c40 con 0x7f02b0103cc0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4018680 con 0x7f02b0103cc0 2023-08-11T14:03:56.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.667+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b01a0e00 con 0x7f02b0103cc0 2023-08-11T14:03:56.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.669+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b0108170 con 0x7f02b0103cc0 2023-08-11T14:03:56.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.671+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f02a40105a0 con 0x7f02b0103cc0 2023-08-11T14:03:56.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.673+0000 7f02acff9700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 0x7f02980a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.673+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f02a4014070 con 0x7f02b0103cc0 2023-08-11T14:03:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.673+0000 7f02af7fe700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 0x7f02980a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.674+0000 7f02af7fe700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 0x7f02980a2490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005fd0 tx=0x7f02a0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:56 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3960364722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.677+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f02a4051c30 con 0x7f02b0103cc0 2023-08-11T14:03:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:56 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3960364722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:56.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='client.25538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='client.25544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:56.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:56 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3960364722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:56.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.917+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02b002ce70 con 0x7f02b0103cc0 2023-08-11T14:03:56.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.918+0000 7f02acff9700 1 -- 172.21.15.3:0/2488129182 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02a4095100 con 0x7f02b0103cc0 2023-08-11T14:03:56.919 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 msgr2=0x7f02980a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 0x7f02980a2490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005fd0 tx=0x7f02a0009310 comp rx=0 tx=0).stop 2023-08-11T14:03:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 msgr2=0x7f02b0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f02a400e9a0 tx=0x7f02a400ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 shutdown_connections 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02b01065f0 0x7f02b019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b0101390 0x7f02b0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f029809ffd0 0x7f02980a2490 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 --2- 172.21.15.3:0/2488129182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0103cc0 0x7f02b0198e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 >> 172.21.15.3:0/2488129182 conn(0x7f02b00fad80 msgr2=0x7f02b00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 shutdown_connections 2023-08-11T14:03:56.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:56.920+0000 7f02b61ff700 1 -- 172.21.15.3:0/2488129182 wait complete. 2023-08-11T14:03:56.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:03:57.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 -- 172.21.15.3:0/1749997298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac074960 msgr2=0x7fb9ac072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/1749997298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac074960 0x7fb9ac072db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009990 tx=0x7fb9a8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 -- 172.21.15.3:0/1749997298 shutdown_connections 2023-08-11T14:03:57.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/1749997298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac1087d0 0x7fb9ac10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/1749997298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac074960 0x7fb9ac072db0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/1749997298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac073380 0x7fb9ac073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 -- 172.21.15.3:0/1749997298 >> 172.21.15.3:0/1749997298 conn(0x7fb9ac0fbe60 msgr2=0x7fb9ac0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:57.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 -- 172.21.15.3:0/1749997298 shutdown_connections 2023-08-11T14:03:57.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.151+0000 7fb9b35ff700 1 -- 172.21.15.3:0/1749997298 wait complete. 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 Processor -- start 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 -- start start 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac074960 0x7fb9ac198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac1087d0 0x7fb9ac19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a03f0 con 0x7fb9ac073380 2023-08-11T14:03:57.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a0530 con 0x7fb9ac074960 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.152+0000 7fb9b35ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a06c0 con 0x7fb9ac1087d0 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59746/0 (socket says 172.21.15.3:59746) 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 -- 172.21.15.3:0/3521173867 learned_addr learned my addr 172.21.15.3:0/3521173867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b1b9c700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac1087d0 0x7fb9ac19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b0b9a700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac074960 0x7fb9ac198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 -- 172.21.15.3:0/3521173867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac1087d0 msgr2=0x7fb9ac19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac1087d0 0x7fb9ac19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 -- 172.21.15.3:0/3521173867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac074960 msgr2=0x7fb9ac198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac074960 0x7fb9ac198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 -- 172.21.15.3:0/3521173867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80095f0 con 0x7fb9ac073380 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.153+0000 7fb9b139b700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8004730 tx=0x7fb9a8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.154+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a801d070 con 0x7fb9ac073380 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.154+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac1a0940 con 0x7fb9ac073380 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.154+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a8004c90 con 0x7fb9ac073380 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.154+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80054b0 con 0x7fb9ac073380 2023-08-11T14:03:57.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.154+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac1a0d00 con 0x7fb9ac073380 2023-08-11T14:03:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.155+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac106e80 con 0x7fb9ac073380 2023-08-11T14:03:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.157+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb9a800fbc0 con 0x7fb9ac073380 2023-08-11T14:03:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.158+0000 7fb9a27fc700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 0x7fb9980a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.158+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb9a80cc840 con 0x7fb9ac073380 2023-08-11T14:03:57.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.159+0000 7fb9b0b9a700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 0x7fb9980a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.160+0000 7fb9b0b9a700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 0x7fb9980a2410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac074680 tx=0x7fb99c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:57.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.162+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb9a805ac30 con 0x7fb9ac073380 2023-08-11T14:03:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.385+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9ac02cd30 con 0x7fb9ac073380 2023-08-11T14:03:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.385+0000 7fb9a27fc700 1 -- 172.21.15.3:0/3521173867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9a80962e0 con 0x7fb9ac073380 2023-08-11T14:03:57.387 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:03:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 msgr2=0x7fb9980a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 0x7fb9980a2410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac074680 tx=0x7fb99c009260 comp rx=0 tx=0).stop 2023-08-11T14:03:57.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 msgr2=0x7fb9ac198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8004730 tx=0x7fb9a8004760 comp rx=0 tx=0).stop 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 shutdown_connections 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9ac1087d0 0x7fb9ac19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac073380 0x7fb9ac198870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb99809ff50 0x7fb9980a2410 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 --2- 172.21.15.3:0/3521173867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9ac074960 0x7fb9ac198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 >> 172.21.15.3:0/3521173867 conn(0x7fb9ac0fbe60 msgr2=0x7fb9ac0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 shutdown_connections 2023-08-11T14:03:57.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.388+0000 7fb9b35ff700 1 -- 172.21.15.3:0/3521173867 wait complete. 2023-08-11T14:03:57.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:03:57.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.629+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/3884608820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34100930 msgr2=0x7f6f34100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.629+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/3884608820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34100930 0x7f6f34100d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099c0 tx=0x7f6f24009cd0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/3884608820 shutdown_connections 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/3884608820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f341024c0 0x7f6f34108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/3884608820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34100930 0x7f6f34100d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/3884608820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f34101b20 0x7f6f34101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/3884608820 >> 172.21.15.3:0/3884608820 conn(0x7f6f340fbea0 msgr2=0x7f6f340fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/3884608820 shutdown_connections 2023-08-11T14:03:57.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.630+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/3884608820 wait complete. 2023-08-11T14:03:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 Processor -- start 2023-08-11T14:03:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 -- start start 2023-08-11T14:03:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f34100930 0x7f6f34198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 0x7f6f34198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a03f0 con 0x7f6f341024c0 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a0530 con 0x7f6f34101b20 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.631+0000 7f6f3a5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f341a06c0 con 0x7f6f34100930 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 -- 172.21.15.3:0/2568838487 learned_addr learned my addr 172.21.15.3:0/2568838487 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:57.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f337fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 0x7f6f34198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f33fff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f34100930 0x7f6f34198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 -- 172.21.15.3:0/2568838487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f34100930 msgr2=0x7f6f34198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f34100930 0x7f6f34198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 -- 172.21.15.3:0/2568838487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 msgr2=0x7f6f34198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 0x7f6f34198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 -- 172.21.15.3:0/2568838487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240095f0 con 0x7f6f341024c0 2023-08-11T14:03:57.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f337fe700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 0x7f6f34198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.632+0000 7f6f38b9c700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00ef30 tx=0x7f6f2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.633+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c018070 con 0x7f6f341024c0 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.633+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f2c00f040 con 0x7f6f341024c0 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.633+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f341a0940 con 0x7f6f341024c0 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.633+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00cd00 con 0x7f6f341024c0 2023-08-11T14:03:57.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.633+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f341a0e50 con 0x7f6f341024c0 2023-08-11T14:03:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.634+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f34192f80 con 0x7f6f341024c0 2023-08-11T14:03:57.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.636+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6f2c003a00 con 0x7f6f341024c0 2023-08-11T14:03:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.638+0000 7f6f317fa700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 0x7f6f1c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.638+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6f2c0caff0 con 0x7f6f341024c0 2023-08-11T14:03:57.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.638+0000 7f6f33fff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 0x7f6f1c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:57.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.639+0000 7f6f33fff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 0x7f6f1c0a2490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009990 tx=0x7f6f24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:57.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.642+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6f2c094af0 con 0x7f6f341024c0 2023-08-11T14:03:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:57 smithi103 ceph-mon[130434]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2117957584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:57 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2488129182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:57 smithi003 ceph-mon[97195]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2117957584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:57 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2488129182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:57 smithi187 ceph-mon[130973]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2117957584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:57.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:57 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2488129182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:03:57.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.858+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6f3402cdc0 con 0x7f6f341024c0 2023-08-11T14:03:57.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.858+0000 7f6f317fa700 1 -- 172.21.15.3:0/2568838487 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6f2c098110 con 0x7f6f341024c0 2023-08-11T14:03:57.860 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:03:57.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 msgr2=0x7f6f1c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 0x7f6f1c0a2490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009990 tx=0x7f6f24005fb0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 msgr2=0x7f6f3419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00ef30 tx=0x7f6f2c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 shutdown_connections 2023-08-11T14:03:57.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f341024c0 0x7f6f3419fe20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f34100930 0x7f6f34198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f1c09ffd0 0x7f6f1c0a2490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 --2- 172.21.15.3:0/2568838487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f34101b20 0x7f6f34198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:57.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 >> 172.21.15.3:0/2568838487 conn(0x7f6f340fbea0 msgr2=0x7f6f34107460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:57.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 shutdown_connections 2023-08-11T14:03:57.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:57.861+0000 7f6f3a5ff700 1 -- 172.21.15.3:0/2568838487 wait complete. 2023-08-11T14:03:57.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:03:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 -- 172.21.15.3:0/3792803591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94101b10 msgr2=0x7fdf94101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 --2- 172.21.15.3:0/3792803591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94101b10 0x7fdf94101f70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009920 tx=0x7fdf88009c30 comp rx=0 tx=0).stop 2023-08-11T14:03:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 -- 172.21.15.3:0/3792803591 shutdown_connections 2023-08-11T14:03:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 --2- 172.21.15.3:0/3792803591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf941024b0 0x7fdf94108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 --2- 172.21.15.3:0/3792803591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf94100920 0x7fdf94100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 --2- 172.21.15.3:0/3792803591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94101b10 0x7fdf94101f70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 -- 172.21.15.3:0/3792803591 >> 172.21.15.3:0/3792803591 conn(0x7fdf940fbe50 msgr2=0x7fdf940fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.084+0000 7fdf98bff700 1 -- 172.21.15.3:0/3792803591 shutdown_connections 2023-08-11T14:03:58.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.085+0000 7fdf98bff700 1 -- 172.21.15.3:0/3792803591 wait complete. 2023-08-11T14:03:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.085+0000 7fdf98bff700 1 Processor -- start 2023-08-11T14:03:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 -- start start 2023-08-11T14:03:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 0x7fdf94072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 0x7fdf94073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf940739f0 con 0x7fdf94100920 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94073b60 con 0x7fdf941024b0 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf98bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf941a0710 con 0x7fdf94101b10 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf9259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf9259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59782/0 (socket says 172.21.15.3:59782) 2023-08-11T14:03:58.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf9259c700 1 -- 172.21.15.3:0/2065636754 learned_addr learned my addr 172.21.15.3:0/2065636754 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf91d9b700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 0x7fdf94072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.086+0000 7fdf92d9d700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 0x7fdf94073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 -- 172.21.15.3:0/2065636754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 msgr2=0x7fdf94072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 0x7fdf94072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 -- 172.21.15.3:0/2065636754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 msgr2=0x7fdf94073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 0x7fdf94073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 -- 172.21.15.3:0/2065636754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840095f0 con 0x7fdf94100920 2023-08-11T14:03:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf91d9b700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 0x7fdf94072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf92d9d700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 0x7fdf94073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf9259c700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400ebf0 tx=0x7fdf8400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84009a10 con 0x7fdf94100920 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8400ce10 con 0x7fdf94100920 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf88009520 con 0x7fdf94100920 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84018890 con 0x7fdf94100920 2023-08-11T14:03:58.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.087+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf941a0dc0 con 0x7fdf94100920 2023-08-11T14:03:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.088+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf94107fb0 con 0x7fdf94100920 2023-08-11T14:03:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.090+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fdf84010ad0 con 0x7fdf94100920 2023-08-11T14:03:58.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.092+0000 7fdf837fe700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 0x7fdf7c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.092+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fdf84014070 con 0x7fdf94100920 2023-08-11T14:03:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.092+0000 7fdf91d9b700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 0x7fdf7c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.092+0000 7fdf91d9b700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 0x7fdf7c0a24e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005e90 tx=0x7fdf88019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:58.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.095+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fdf84095610 con 0x7fdf94100920 2023-08-11T14:03:58.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.351+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf9410c130 con 0x7fdf94100920 2023-08-11T14:03:58.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.352+0000 7fdf837fe700 1 -- 172.21.15.3:0/2065636754 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7fdf9410c130 con 0x7fdf94100920 2023-08-11T14:03:58.353 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:58.353 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 msgr2=0x7fdf7c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 0x7fdf7c0a24e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005e90 tx=0x7fdf88019040 comp rx=0 tx=0).stop 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 msgr2=0x7fdf940746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400ebf0 tx=0x7fdf8400c2d0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 shutdown_connections 2023-08-11T14:03:58.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdf7c0a0020 0x7fdf7c0a24e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf941024b0 0x7fdf94073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf94100920 0x7fdf940746d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 --2- 172.21.15.3:0/2065636754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf94101b10 0x7fdf94072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 >> 172.21.15.3:0/2065636754 conn(0x7fdf940fbe50 msgr2=0x7fdf941050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.354+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 shutdown_connections 2023-08-11T14:03:58.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.355+0000 7fdf98bff700 1 -- 172.21.15.3:0/2065636754 wait complete. 2023-08-11T14:03:58.358 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:03:58.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:03:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 -- 172.21.15.3:0/526804359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4103d50 msgr2=0x7ff1a4106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 --2- 172.21.15.3:0/526804359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4103d50 0x7ff1a4106140 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff194009990 tx=0x7ff194009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 -- 172.21.15.3:0/526804359 shutdown_connections 2023-08-11T14:03:58.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 --2- 172.21.15.3:0/526804359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4106680 0x7ff1a4108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 --2- 172.21.15.3:0/526804359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4101420 0x7ff1a4103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 --2- 172.21.15.3:0/526804359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4103d50 0x7ff1a4106140 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 -- 172.21.15.3:0/526804359 >> 172.21.15.3:0/526804359 conn(0x7ff1a40fad50 msgr2=0x7ff1a40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 -- 172.21.15.3:0/526804359 shutdown_connections 2023-08-11T14:03:58.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.578+0000 7ff1abbff700 1 -- 172.21.15.3:0/526804359 wait complete. 2023-08-11T14:03:58.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.579+0000 7ff1abbff700 1 Processor -- start 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.579+0000 7ff1abbff700 1 -- start start 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4101420 0x7ff1a4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4103d50 0x7ff1a4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a0300 con 0x7ff1a4106680 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a0440 con 0x7ff1a4103d50 2023-08-11T14:03:58.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1abbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a05d0 con 0x7ff1a4101420 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1a919a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4103d50 0x7ff1a4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1a999b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4101420 0x7ff1a4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59790/0 (socket says 172.21.15.3:59790) 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 -- 172.21.15.3:0/3565547921 learned_addr learned my addr 172.21.15.3:0/3565547921 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 -- 172.21.15.3:0/3565547921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4101420 msgr2=0x7ff1a4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4101420 0x7ff1a4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 -- 172.21.15.3:0/3565547921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4103d50 msgr2=0x7ff1a4198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4103d50 0x7ff1a4198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.580+0000 7ff1aa19c700 1 -- 172.21.15.3:0/3565547921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940095f0 con 0x7ff1a4106680 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff1aa19c700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ef30 tx=0x7ff19c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c018070 con 0x7ff1a4106680 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c00f040 con 0x7ff1a4106680 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a41a0850 con 0x7ff1a4106680 2023-08-11T14:03:58.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00cd00 con 0x7ff1a4106680 2023-08-11T14:03:58.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.581+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41a0d60 con 0x7ff1a4106680 2023-08-11T14:03:58.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.582+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a410c130 con 0x7ff1a4106680 2023-08-11T14:03:58.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.584+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff19c003a00 con 0x7ff1a4106680 2023-08-11T14:03:58.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.585+0000 7ff19affd700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 0x7ff1900a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:58.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.586+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff19c0cb0a0 con 0x7ff1a4106680 2023-08-11T14:03:58.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.586+0000 7ff1a999b700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 0x7ff1900a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:58.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.586+0000 7ff1a999b700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 0x7ff1900a24e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009c60 tx=0x7ff1a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.589+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff19c094b40 con 0x7ff1a4106680 2023-08-11T14:03:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3521173867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:03:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:03:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:58.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:58 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2568838487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3521173867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:03:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:03:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:58.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:58 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2568838487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3521173867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:03:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:03:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:03:58.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:58 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2568838487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:03:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.802+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a41a0f80 con 0x7ff1a4106680 2023-08-11T14:03:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.803+0000 7ff19affd700 1 -- 172.21.15.3:0/3565547921 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7ff19c098160 con 0x7ff1a4106680 2023-08-11T14:03:58.805 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:03:58.806 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:03:58.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 msgr2=0x7ff1900a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 0x7ff1900a24e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009c60 tx=0x7ff1a0009260 comp rx=0 tx=0).stop 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 msgr2=0x7ff1a419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff19c00ef30 tx=0x7ff19c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 shutdown_connections 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1a4106680 0x7ff1a419fd30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1a4101420 0x7ff1a4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff1900a0020 0x7ff1900a24e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 --2- 172.21.15.3:0/3565547921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1a4103d50 0x7ff1a4198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 >> 172.21.15.3:0/3565547921 conn(0x7ff1a40fad50 msgr2=0x7ff1a4102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 shutdown_connections 2023-08-11T14:03:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:58.806+0000 7ff1abbff700 1 -- 172.21.15.3:0/3565547921 wait complete. 2023-08-11T14:03:58.827 DEBUG:tasks.fwd_scrub:Yielding 2023-08-11T14:03:58.828 INFO:teuthology.run_tasks:Running task background_exec... 2023-08-11T14:03:58.845 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:03:58.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-08-11T14:03:58.846 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-08-11T14:03:58.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-08-11T14:03:58.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-08-11T14:03:58.853 INFO:teuthology.run_tasks:Running task exec... 2023-08-11T14:03:58.886 INFO:teuthology.task.exec:Executing custom commands... 2023-08-11T14:03:58.887 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:03:58.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-08-11T14:03:59.030 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:03:59.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 -- 172.21.15.3:0/1369567583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c106660 msgr2=0x7fba1c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 --2- 172.21.15.3:0/1369567583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c106660 0x7fba1c108ad0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fba1400b0a0 tx=0x7fba1400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 -- 172.21.15.3:0/1369567583 shutdown_connections 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 --2- 172.21.15.3:0/1369567583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c106660 0x7fba1c108ad0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 --2- 172.21.15.3:0/1369567583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c101400 0x7fba1c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 --2- 172.21.15.3:0/1369567583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c103d30 0x7fba1c106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 -- 172.21.15.3:0/1369567583 >> 172.21.15.3:0/1369567583 conn(0x7fba1c0fadf0 msgr2=0x7fba1c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 -- 172.21.15.3:0/1369567583 shutdown_connections 2023-08-11T14:03:59.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.067+0000 7fba21dff700 1 -- 172.21.15.3:0/1369567583 wait complete. 2023-08-11T14:03:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.068+0000 7fba21dff700 1 Processor -- start 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.068+0000 7fba21dff700 1 -- start start 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c103d30 0x7fba1c0ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 0x7fba1c0ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c100090 con 0x7fba1c101400 2023-08-11T14:03:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c100200 con 0x7fba1c106660 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba21dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1a0750 con 0x7fba1c103d30 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59818/0 (socket says 172.21.15.3:59818) 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 -- 172.21.15.3:0/616233282 learned_addr learned my addr 172.21.15.3:0/616233282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1affd700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c103d30 0x7fba1c0ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1bfff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 0x7fba1c0ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 -- 172.21.15.3:0/616233282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c103d30 msgr2=0x7fba1c0ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c103d30 0x7fba1c0ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 -- 172.21.15.3:0/616233282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 msgr2=0x7fba1c0ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 0x7fba1c0ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1b7fe700 1 -- 172.21.15.3:0/616233282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009520 con 0x7fba1c101400 2023-08-11T14:03:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.069+0000 7fba1bfff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 0x7fba1c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e9a0 tx=0x7fba0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba18ff9700 1 -- 172.21.15.3:0/616233282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c009790 con 0x7fba1c101400 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba18ff9700 1 -- 172.21.15.3:0/616233282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0c00cbd0 con 0x7fba1c101400 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba14009d00 con 0x7fba1c101400 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba18ff9700 1 -- 172.21.15.3:0/616233282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c018680 con 0x7fba1c101400 2023-08-11T14:03:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.070+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c1a0e00 con 0x7fba1c101400 2023-08-11T14:03:59.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.075+0000 7fba18ff9700 1 -- 172.21.15.3:0/616233282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fba0c00cd40 con 0x7fba1c101400 2023-08-11T14:03:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.076+0000 7fba35810b80 1 Processor -- start 2023-08-11T14:03:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.076+0000 7fba35810b80 1 -- start start 2023-08-11T14:03:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c3145f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c3146930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c309e6b0 con 0x55b5c3146e70 2023-08-11T14:03:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c309bfd0 con 0x55b5c3145b50 2023-08-11T14:03:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba35810b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c30978c0 con 0x55b5c31464b0 2023-08-11T14:03:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba18ff9700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 0x7fba040a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c3146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba18ff9700 1 -- 172.21.15.3:0/616233282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba0c014070 con 0x7fba1c101400 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59830/0 (socket says 172.21.15.3:59830) 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1affd700 1 -- 172.21.15.3:0/2996068848 learned_addr learned my addr 172.21.15.3:0/2996068848 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c3145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1bfff700 1 -- 172.21.15.3:0/2996068848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 msgr2=0x55b5c3146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.077+0000 7fba1bfff700 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c3146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1bfff700 1 -- 172.21.15.3:0/2996068848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 msgr2=0x55b5c31472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1bfff700 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1bfff700 1 -- 172.21.15.3:0/2996068848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c2f176b0 con 0x55b5c3145b50 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1affd700 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1bfff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 0x7fba040a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba1bfff700 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c3145f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba140093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f06f587d7a7b958a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba01ffb700 1 -- 172.21.15.3:0/2996068848 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e040 con 0x55b5c3145b50 2023-08-11T14:03:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba01ffb700 1 -- 172.21.15.3:0/2996068848 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1400be90 con 0x55b5c3145b50 2023-08-11T14:03:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.078+0000 7fba01ffb700 1 -- 172.21.15.3:0/2996068848 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14027cf0 con 0x55b5c3145b50 2023-08-11T14:03:59.084 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem Size Used Avail Use% Mounted on 2023-08-11T14:03:59.084 INFO:teuthology.orchestra.run.smithi003.stdout:0@69ee59aa-384e-11ee-9b37-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1 1.1T 332M 1.1T 1% /tmp/cephfs 2023-08-11T14:03:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 -- 172.21.15.3:0/2996068848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 msgr2=0x55b5c3145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c3145f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba140093c0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba1bfff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 0x7fba040a2490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b5c31478f0 tx=0x7fba1402c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 -- 172.21.15.3:0/2996068848 shutdown_connections 2023-08-11T14:03:59.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c31472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c3145f70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 --2- 172.21.15.3:0/2996068848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c3146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.079+0000 7fba35810b80 1 -- 172.21.15.3:0/2996068848 >> 172.21.15.3:0/2996068848 conn(0x55b5c30729d0 msgr2=0x55b5c31404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.080+0000 7fba35810b80 1 -- 172.21.15.3:0/2996068848 shutdown_connections 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.080+0000 7fba35810b80 1 -- 172.21.15.3:0/2996068848 wait complete. 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.080+0000 7fba35810b80 1 -- start start 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.080+0000 7fba35810b80 1 Processor -- start 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c314f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c3156760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3035b40 con 0x55b5c3146e70 2023-08-11T14:03:59.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3035840 con 0x55b5c3145b50 2023-08-11T14:03:59.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c2f199a0 con 0x55b5c31464b0 2023-08-11T14:03:59.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba35810b80 10 client.0 _ll_register_callbacks cb 0x55b5c3038b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:59.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48992/0 (socket says 172.21.15.3:48992) 2023-08-11T14:03:59.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1980634429 learned_addr learned my addr 172.21.15.3:0/1980634429 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.081+0000 7fba1bfff700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c3156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1980634429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 msgr2=0x55b5c314f720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:03:59.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c314f720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1980634429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 msgr2=0x55b5c3156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c3156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1980634429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba10009b70 con 0x55b5c3145b50 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1bfff700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c3156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00c5b0 tx=0x7fba0c098940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba34fcfcf0e4931 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fb9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:59.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0b6ea0 con 0x55b5c3145b50 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba0c09cce0 con 0x55b5c3145b50 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.082+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5c3138b10 con 0x55b5c3145b50 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba0c0a0cc0 con 0x55b5c3145b50 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 10 client.25598 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fba1c1096c0 con 0x55b5c3145b50 2023-08-11T14:03:59.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5c3138b10 con 0x55b5c3145b50 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba0c093ac0 con 0x55b5c3145b50 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 10 client.25598 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 20 client.25598 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c315af50 0x55b5c315d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 4 client.25598 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.083+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5c3114560 con 0x55b5c315af50 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.084+0000 7fba1affd700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c315af50 0x55b5c315d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.084+0000 7fba1affd700 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c315af50 0x55b5c315d370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44018230ee0fbfdf server_cookie=172a0839345c2689 in_seq=0 out_seq=0 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.084+0000 7fb9faffd700 10 client.25598 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.085+0000 7fb9faffd700 1 -- 172.21.15.3:0/1980634429 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b5c3114560 con 0x55b5c315af50 2023-08-11T14:03:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.085+0000 7fb9faffd700 10 client.25598 handle_command_reply: tid=0 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.085+0000 7fba35810b80 1 client.25598 shutdown 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.085+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 msgr2=0x55b5c314f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.085+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00c5b0 tx=0x7fba0c098940 comp rx=0 tx=0).stop 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 shutdown_connections 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3146e70 0x55b5c3156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c315af50 0x55b5c315d370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c314f1e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 --2- 172.21.15.3:0/1980634429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c31464b0 0x55b5c314f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 >> 172.21.15.3:0/1980634429 conn(0x55b5c30729d0 msgr2=0x55b5c313ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fb9faffd700 0 client.25598 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 shutdown_connections 2023-08-11T14:03:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 1 -- 172.21.15.3:0/1980634429 wait complete. 2023-08-11T14:03:59.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-08-11T14:03:59.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.086+0000 7fba35810b80 20 client.25598 trim_cache size 0 max 16384 2023-08-11T14:03:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 Processor -- start 2023-08-11T14:03:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 -- start start 2023-08-11T14:03:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c31464b0 0x55b5c3156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c313a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c314cfc0 con 0x55b5c31464b0 2023-08-11T14:03:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c314a870 con 0x55b5c3145b50 2023-08-11T14:03:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba35810b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3141540 con 0x55b5c3146e70 2023-08-11T14:03:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba1affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c31464b0 0x55b5c3156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2023-08-11T14:03:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1306324390 learned_addr learned my addr 172.21.15.3:0/1306324390 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.104+0000 7fba1bfff700 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c313a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1306324390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 msgr2=0x55b5c313a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c313a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1306324390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c31464b0 msgr2=0x55b5c3156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c31464b0 0x55b5c3156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 -- 172.21.15.3:0/1306324390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c314e2b0 con 0x55b5c3145b50 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0b11c0 tx=0x7fba0c010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc130ec9de17169f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba027fc700 1 -- 172.21.15.3:0/1306324390 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0b6ea0 con 0x55b5c3145b50 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba027fc700 1 -- 172.21.15.3:0/1306324390 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0c09cce0 con 0x55b5c3145b50 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.105+0000 7fba027fc700 1 -- 172.21.15.3:0/1306324390 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0b8310 con 0x55b5c3145b50 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- 172.21.15.3:0/1306324390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 msgr2=0x55b5c313a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0b11c0 tx=0x7fba0c010b30 comp rx=0 tx=0).stop 2023-08-11T14:03:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- 172.21.15.3:0/1306324390 shutdown_connections 2023-08-11T14:03:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c313a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c3145b50 0x55b5c313a4a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 --2- 172.21.15.3:0/1306324390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c31464b0 0x55b5c3156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- 172.21.15.3:0/1306324390 >> 172.21.15.3:0/1306324390 conn(0x55b5c30729d0 msgr2=0x55b5c3140430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- 172.21.15.3:0/1306324390 shutdown_connections 2023-08-11T14:03:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- 172.21.15.3:0/1306324390 wait complete. 2023-08-11T14:03:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.106+0000 7fba35810b80 1 -- start start 2023-08-11T14:03:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 Processor -- start 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3145b50 0x55b5c3153230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c31aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3156790 con 0x55b5c3145b50 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3156900 con 0x55b5c31464b0 2023-08-11T14:03:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c3156ac0 con 0x55b5c3146e70 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba35810b80 10 client.0 _ll_register_callbacks cb 0x7fba0c0989f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3145b50 0x55b5c3153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba1affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba1affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49020/0 (socket says 172.21.15.3:49020) 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.107+0000 7fba1affd700 1 -- 172.21.15.3:0/709451888 learned_addr learned my addr 172.21.15.3:0/709451888 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1bfff700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c31aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 -- 172.21.15.3:0/709451888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 msgr2=0x55b5c31aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c31aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 -- 172.21.15.3:0/709451888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3145b50 msgr2=0x55b5c3153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3145b50 0x55b5c3153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 -- 172.21.15.3:0/709451888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c1096c0 con 0x55b5c31464b0 2023-08-11T14:03:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fba1affd700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b5c31418e0 tx=0x7fba1001e390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6e10267e16bc45 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fb9faffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:03:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.108+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10017040 con 0x55b5c31464b0 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba1400be90 con 0x55b5c31464b0 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba1001ed20 con 0x55b5c31464b0 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba10005020 con 0x55b5c31464b0 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 10 client.25604 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5c2ec3e60 con 0x55b5c31464b0 2023-08-11T14:03:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1400be90 con 0x55b5c31464b0 2023-08-11T14:03:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fba10012550 con 0x55b5c31464b0 2023-08-11T14:03:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:03:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 20 client.25604 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c31b0aa0 0x55b5c31b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 4 client.25604 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.109+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b5c30908a0 con 0x55b5c31b0aa0 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.110+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c31b0aa0 0x55b5c31b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.110+0000 7fba1b7fe700 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c31b0aa0 0x55b5c31b2ec0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d689cbfc073a5765 server_cookie=531eb4b23d58eef5 in_seq=0 out_seq=0 2023-08-11T14:03:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.110+0000 7fb9faffd700 10 client.25604 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.111+0000 7fb9faffd700 1 -- 172.21.15.3:0/709451888 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b5c30908a0 con 0x55b5c31b0aa0 2023-08-11T14:03:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.111+0000 7fb9faffd700 10 client.25604 handle_command_reply: tid=0 2023-08-11T14:03:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.111+0000 7fba35810b80 1 client.25604 shutdown 2023-08-11T14:03:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 msgr2=0x55b5c3153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b5c31418e0 tx=0x7fba1001e390 comp rx=0 tx=0).stop 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 shutdown_connections 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5c3146e70 0x55b5c31aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5c3145b50 0x55b5c3153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b5c31b0aa0 0x55b5c31b2ec0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 --2- 172.21.15.3:0/709451888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5c31464b0 0x55b5c3153770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 >> 172.21.15.3:0/709451888 conn(0x55b5c30729d0 msgr2=0x55b5c3144950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fb9faffd700 0 client.25604 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 shutdown_connections 2023-08-11T14:03:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 1 -- 172.21.15.3:0/709451888 wait complete. 2023-08-11T14:03:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.112+0000 7fba35810b80 20 client.25604 trim_cache size 0 max 16384 2023-08-11T14:03:59.129 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2023-08-11T14:03:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "92be14a8-cc6f-4450-88c9-7768dd83eae8", 2023-08-11T14:03:59.129 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2023-08-11T14:03:59.130 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 msgr2=0x7fba040a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 0x7fba040a2490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55b5c31478f0 tx=0x7fba1402c000 comp rx=0 tx=0).stop 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 msgr2=0x7fba1c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e9a0 tx=0x7fba0c00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 shutdown_connections 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba0409ffd0 0x7fba040a2490 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba1c106660 0x7fba1c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba1c101400 0x7fba1c100d70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 --2- 172.21.15.3:0/616233282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c103d30 0x7fba1c0ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 >> 172.21.15.3:0/616233282 conn(0x7fba1c0fadf0 msgr2=0x7fba1c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 shutdown_connections 2023-08-11T14:03:59.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.113+0000 7fba21dff700 1 -- 172.21.15.3:0/616233282 wait complete. 2023-08-11T14:03:59.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-11T14:03:59.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-08-11T14:03:59.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-08-11T14:03:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 -- 172.21.15.3:0/125415564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 msgr2=0x7fc040103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 --2- 172.21.15.3:0/125415564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040103820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc030009990 tx=0x7fc030009ca0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 -- 172.21.15.3:0/125415564 shutdown_connections 2023-08-11T14:03:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 --2- 172.21.15.3:0/125415564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc040108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 --2- 172.21.15.3:0/125415564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040103820 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 --2- 172.21.15.3:0/125415564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc040106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 -- 172.21.15.3:0/125415564 >> 172.21.15.3:0/125415564 conn(0x7fc0400fad80 msgr2=0x7fc0400fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 -- 172.21.15.3:0/125415564 shutdown_connections 2023-08-11T14:03:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.395+0000 7fc0465ff700 1 -- 172.21.15.3:0/125415564 wait complete. 2023-08-11T14:03:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.396+0000 7fc0465ff700 1 Processor -- start 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.396+0000 7fc0465ff700 1 -- start start 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.396+0000 7fc0465ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.396+0000 7fc0465ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc0400ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc0465ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc0400ff920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc0465ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401000c0 con 0x7fc040101430 2023-08-11T14:03:59.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc0465ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040100230 con 0x7fc040103d60 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc0465ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401a06c0 con 0x7fc040106690 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc0400ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc0400ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59870/0 (socket says 172.21.15.3:59870) 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc03ffff700 1 -- 172.21.15.3:0/1764421158 learned_addr learned my addr 172.21.15.3:0/1764421158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc0400ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49044/0 (socket says 172.21.15.3:49044) 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 -- 172.21.15.3:0/1764421158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 msgr2=0x7fc0400ff3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc0400ff3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 -- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 msgr2=0x7fc040100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.397+0000 7fc044b9c700 1 -- 172.21.15.3:0/1764421158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300095f0 con 0x7fc040106690 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc0400ff920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc03800efc0 tx=0x7fc03800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc03d7fa700 1 -- 172.21.15.3:0/1764421158 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038018070 con 0x7fc040106690 2023-08-11T14:03:59.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc03d7fa700 1 -- 172.21.15.3:0/1764421158 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03800f040 con 0x7fc040106690 2023-08-11T14:03:59.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0401a0940 con 0x7fc040106690 2023-08-11T14:03:59.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc03d7fa700 1 -- 172.21.15.3:0/1764421158 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03800cd40 con 0x7fc040106690 2023-08-11T14:03:59.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.398+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0401a0e00 con 0x7fc040106690 2023-08-11T14:03:59.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.403+0000 7fc059f82b80 1 Processor -- start 2023-08-11T14:03:59.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.403+0000 7fc03d7fa700 1 -- 172.21.15.3:0/1764421158 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc038003a00 con 0x7fc040106690 2023-08-11T14:03:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.403+0000 7fc059f82b80 1 -- start start 2023-08-11T14:03:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f9af630 con 0x557a2fa56ad0 2023-08-11T14:03:59.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f9acf50 con 0x557a2fa57430 2023-08-11T14:03:59.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc059f82b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f9a8840 con 0x557a2fa57df0 2023-08-11T14:03:59.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49052/0 (socket says 172.21.15.3:49052) 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 -- 172.21.15.3:0/3393374358 learned_addr learned my addr 172.21.15.3:0/3393374358 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 -- 172.21.15.3:0/3393374358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 msgr2=0x557a2fa58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa58270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 -- 172.21.15.3:0/3393374358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 msgr2=0x557a2fa56ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa56ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03ffff700 1 -- 172.21.15.3:0/3393374358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a2f828630 con 0x557a2fa57430 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.404+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc03ffff700 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc030005f30 tx=0x7fc030005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe57e417ff5757b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc03d7fa700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 0x7fc0280a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc03d7fa700 1 -- 172.21.15.3:0/1764421158 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc0380cb120 con 0x7fc040106690 2023-08-11T14:03:59.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc0267fc700 1 -- 172.21.15.3:0/3393374358 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03001d070 con 0x557a2fa57430 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc0267fc700 1 -- 172.21.15.3:0/3393374358 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03001e040 con 0x557a2fa57430 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc0267fc700 1 -- 172.21.15.3:0/3393374358 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030004aa0 con 0x557a2fa57430 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.405+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 0x7fc0280a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 -- 172.21.15.3:0/3393374358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 msgr2=0x557a2fa578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 0x7fc0280a2410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x557a2fa52400 tx=0x7fc0340058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc030005f30 tx=0x7fc030005bc0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 -- 172.21.15.3:0/3393374358 shutdown_connections 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 --2- 172.21.15.3:0/3393374358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa578b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 -- 172.21.15.3:0/3393374358 >> 172.21.15.3:0/3393374358 conn(0x557a2f983950 msgr2=0x557a2fa51420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 -- 172.21.15.3:0/3393374358 shutdown_connections 2023-08-11T14:03:59.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.406+0000 7fc059f82b80 1 -- 172.21.15.3:0/3393374358 wait complete. 2023-08-11T14:03:59.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 -- start start 2023-08-11T14:03:59.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 Processor -- start 2023-08-11T14:03:59.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa56ad0 0x557a2fa60150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 0x557a2fa676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f946ac0 con 0x557a2fa57df0 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f9467c0 con 0x557a2fa57430 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.407+0000 7fc059f82b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2f82a920 con 0x557a2fa56ad0 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc059f82b80 10 client.0 _ll_register_callbacks cb 0x557a2f949ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:59.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 0x557a2fa676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa60690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49068/0 (socket says 172.21.15.3:49068) 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc03f7fe700 1 -- 172.21.15.3:0/1751738440 learned_addr learned my addr 172.21.15.3:0/1751738440 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 -- 172.21.15.3:0/1751738440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa56ad0 msgr2=0x557a2fa60150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa56ad0 0x557a2fa60150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 -- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 msgr2=0x557a2fa60690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa60690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 -- 172.21.15.3:0/1751738440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc034009d50 con 0x557a2fa57df0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 0x557a2fa676d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc0380c8950 tx=0x7fc038098340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2c1c52d7426d62 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.408+0000 7fc01f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038083070 con 0x557a2fa57df0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a2fa49a90 con 0x557a2fa57df0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc0380988e0 con 0x557a2fa57df0 2023-08-11T14:03:59.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03809c710 con 0x557a2fa57df0 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc059f82b80 10 client.15582 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc040109650 con 0x557a2fa57df0 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a2fa49a90 con 0x557a2fa57df0 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.409+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0380a01d0 con 0x557a2fa57df0 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:03:59.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 20 client.15582 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fa6bf00 0x557a2fa6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 4 client.15582 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a2fa254e0 con 0x557a2fa6bf00 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.410+0000 7fc03ffff700 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fa6bf00 0x557a2fa6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.411+0000 7fc03ffff700 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fa6bf00 0x557a2fa6e340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ec918132d8ce5d server_cookie=724640a97b672af5 in_seq=0 out_seq=0 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.411+0000 7fc01f7fe700 10 client.15582 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.411+0000 7fc01f7fe700 1 -- 172.21.15.3:0/1751738440 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x557a2fa254e0 con 0x557a2fa6bf00 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.411+0000 7fc01f7fe700 10 client.15582 handle_command_reply: tid=0 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 client.15582 shutdown 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 msgr2=0x557a2fa676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 0x557a2fa676d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc0380c8950 tx=0x7fc038098340 comp rx=0 tx=0).stop 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 shutdown_connections 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57df0 0x557a2fa676d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa56ad0 0x557a2fa60150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fa6bf00 0x557a2fa6e340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1751738440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57430 0x557a2fa60690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 >> 172.21.15.3:0/1751738440 conn(0x557a2f983950 msgr2=0x557a2fa4bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc01f7fe700 0 client.15582 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 shutdown_connections 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 1 -- 172.21.15.3:0/1751738440 wait complete. 2023-08-11T14:03:59.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.412+0000 7fc059f82b80 20 client.15582 trim_cache size 0 max 16384 2023-08-11T14:03:59.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.435+0000 7fc059f82b80 1 Processor -- start 2023-08-11T14:03:59.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 -- start start 2023-08-11T14:03:59.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 0x557a2fa5ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa5b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa5f530 con 0x557a2fa57430 2023-08-11T14:03:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa693c0 con 0x557a2fa56ad0 2023-08-11T14:03:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc059f82b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa5ff20 con 0x557a2fa57df0 2023-08-11T14:03:59.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc03f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59896/0 (socket says 172.21.15.3:59896) 2023-08-11T14:03:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc03f7fe700 1 -- 172.21.15.3:0/1721362045 learned_addr learned my addr 172.21.15.3:0/1721362045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc03ffff700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 0x557a2fa5ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.436+0000 7fc044b9c700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa5b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 -- 172.21.15.3:0/1721362045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 msgr2=0x557a2fa5b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa5b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 -- 172.21.15.3:0/1721362045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 msgr2=0x557a2fa5ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 0x557a2fa5ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 -- 172.21.15.3:0/1721362045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a2fa584a0 con 0x557a2fa57430 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03ffff700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 0x557a2fa5ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc034004420 tx=0x7fc034015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c09128bff1dc159 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc026ffd700 1 -- 172.21.15.3:0/1721362045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034015070 con 0x557a2fa57430 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc026ffd700 1 -- 172.21.15.3:0/1721362045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0340248c0 con 0x557a2fa57430 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.437+0000 7fc026ffd700 1 -- 172.21.15.3:0/1721362045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034020670 con 0x557a2fa57430 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 -- 172.21.15.3:0/1721362045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 msgr2=0x557a2fa67690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc034004420 tx=0x7fc034015000 comp rx=0 tx=0).stop 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 -- 172.21.15.3:0/1721362045 shutdown_connections 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57df0 0x557a2fa5b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa56ad0 0x557a2fa5ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 --2- 172.21.15.3:0/1721362045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa57430 0x557a2fa67690 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 -- 172.21.15.3:0/1721362045 >> 172.21.15.3:0/1721362045 conn(0x557a2f983950 msgr2=0x557a2fa6df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 -- 172.21.15.3:0/1721362045 shutdown_connections 2023-08-11T14:03:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.438+0000 7fc059f82b80 1 -- 172.21.15.3:0/1721362045 wait complete. 2023-08-11T14:03:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.439+0000 7fc059f82b80 1 -- start start 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.439+0000 7fc059f82b80 1 Processor -- start 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.439+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.439+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57430 0x557a2fa63e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.439+0000 7fc059f82b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57df0 0x557a2fa66e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc059f82b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa67340 con 0x557a2fa56ad0 2023-08-11T14:03:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc059f82b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa674b0 con 0x557a2fa57df0 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc059f82b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2fa67670 con 0x557a2fa57430 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc059f82b80 10 client.0 _ll_register_callbacks cb 0x7fc0380986d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59902/0 (socket says 172.21.15.3:59902) 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 -- 172.21.15.3:0/628023986 learned_addr learned my addr 172.21.15.3:0/628023986 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57430 0x557a2fa63e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc044b9c700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57df0 0x557a2fa66e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 -- 172.21.15.3:0/628023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57430 msgr2=0x557a2fa63e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57430 0x557a2fa63e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 -- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57df0 msgr2=0x557a2fa66e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57df0 0x557a2fa66e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.440+0000 7fc03ffff700 1 -- 172.21.15.3:0/628023986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a2f828630 con 0x557a2fa56ad0 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc01f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:03:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc03ffff700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc030005f30 tx=0x7fc030009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf7f909b0218bfd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030031070 con 0x557a2fa56ad0 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc03001e040 con 0x557a2fa56ad0 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc030005250 con 0x557a2fa56ad0 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03001d040 con 0x557a2fa56ad0 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc059f82b80 10 client.15588 fetch_fsmap learned FSMap version 27 2023-08-11T14:03:59.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.441+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc03401c070 con 0x557a2fa56ad0 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03001e040 con 0x557a2fa56ad0 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc03000f460 con 0x557a2fa56ad0 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 20 client.15588 populate_metadata read hostname 'smithi003' 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fac0bf0 0x557a2fac3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:03:59.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 4 client.15588 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a2f9a1820 con 0x557a2fac0bf0 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.442+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fac0bf0 0x557a2fac3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc03f7fe700 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fac0bf0 0x557a2fac3010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8955c33c95b4bd67 server_cookie=a24d0d0957f41345 in_seq=0 out_seq=0 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc01f7fe700 10 client.15588 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc01f7fe700 1 -- 172.21.15.3:0/628023986 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557a2f9a1820 con 0x557a2fac0bf0 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc01f7fe700 10 client.15588 handle_command_reply: tid=0 2023-08-11T14:03:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc059f82b80 1 client.15588 shutdown 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 msgr2=0x557a2fa63910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc030005f30 tx=0x7fc030009310 comp rx=0 tx=0).stop 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.443+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 shutdown_connections 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a2fac0bf0 0x557a2fac3010 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a2fa57df0 0x557a2fa66e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a2fa56ad0 0x557a2fa63910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 --2- 172.21.15.3:0/628023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a2fa57430 0x557a2fa63e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc01f7fe700 0 client.15588 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 >> 172.21.15.3:0/628023986 conn(0x557a2f983950 msgr2=0x557a2fa5ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 shutdown_connections 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 1 -- 172.21.15.3:0/628023986 wait complete. 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.444+0000 7fc059f82b80 20 client.15588 trim_cache size 0 max 16384 2023-08-11T14:03:59.453 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "92be14a8-cc6f-4450-88c9-7768dd83eae8": { 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "92be14a8-cc6f-4450-88c9-7768dd83eae8", 2023-08-11T14:03:59.454 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2023-08-11T14:03:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:03:59.455 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:03:59.455 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:03:59.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 msgr2=0x7fc0280a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 0x7fc0280a2410 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x557a2fa52400 tx=0x7fc0340058e0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 msgr2=0x7fc0400ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc0400ff920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc03800efc0 tx=0x7fc03800c3a0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 shutdown_connections 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc040106690 0x7fc0400ff920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc040101430 0x7fc040100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc02809ff50 0x7fc0280a2410 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 --2- 172.21.15.3:0/1764421158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc040103d60 0x7fc0400ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:03:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 >> 172.21.15.3:0/1764421158 conn(0x7fc0400fad80 msgr2=0x7fc0401020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:03:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 shutdown_connections 2023-08-11T14:03:59.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:03:59.445+0000 7fc0465ff700 1 -- 172.21.15.3:0/1764421158 wait complete. 2023-08-11T14:03:59.469 INFO:tasks.cephfs.filesystem:scrub status for tag:92be14a8-cc6f-4450-88c9-7768dd83eae8 - {'path': '/', 'tag': '92be14a8-cc6f-4450-88c9-7768dd83eae8', 'options': 'recursive,force'} 2023-08-11T14:03:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2065636754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3565547921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: scrub queued for path: / 2023-08-11T14:03:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:03:59.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:03:59 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:03:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2065636754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3565547921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: scrub queued for path: / 2023-08-11T14:03:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:03:59.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:03:59 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: pgmap v349: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2065636754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3565547921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: scrub queued for path: / 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:03:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:03:59 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:00 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:00 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:00.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:00 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:01 smithi103 ceph-mon[130434]: pgmap v350: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-08-11T14:04:01.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:01 smithi003 ceph-mon[97195]: pgmap v350: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-08-11T14:04:01.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:01 smithi187 ceph-mon[130973]: pgmap v350: 129 pgs: 129 active+clean; 648 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 11 op/s 2023-08-11T14:04:02.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-08-11T14:04:02.808 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-08-11T14:04:02.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-08-11T14:04:02.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-08-11T14:04:03.014 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/: 2023-08-11T14:04:03.014 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:03.014 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 . 2023-08-11T14:04:03.014 INFO:teuthology.orchestra.run.smithi003.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:03 data 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data: 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:03 . 2023-08-11T14:04:03.015 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:04:03.019 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-08-11T14:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout:[Service] 2023-08-11T14:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-08-11T14:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-08-11T14:04:03.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-08-11T14:04:03.350 INFO:teuthology.orchestra.run.smithi003.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-08-11T14:04:03.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:03 smithi003 ceph-mon[97195]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:04:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:03 smithi103 ceph-mon[130434]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:04:03.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:03 smithi187 ceph-mon[130973]: pgmap v351: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2023-08-11T14:04:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:05 smithi103 ceph-mon[130434]: pgmap v352: 129 pgs: 129 active+clean; 1.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-08-11T14:04:05.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:05 smithi003 ceph-mon[97195]: pgmap v352: 129 pgs: 129 active+clean; 1.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-08-11T14:04:05.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:05 smithi187 ceph-mon[130973]: pgmap v352: 129 pgs: 129 active+clean; 1.1 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 14 op/s 2023-08-11T14:04:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:07 smithi103 ceph-mon[130434]: pgmap v353: 129 pgs: 129 active+clean; 11 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 865 KiB/s wr, 17 op/s 2023-08-11T14:04:07.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:07 smithi003 ceph-mon[97195]: pgmap v353: 129 pgs: 129 active+clean; 11 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 865 KiB/s wr, 17 op/s 2023-08-11T14:04:07.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:07 smithi187 ceph-mon[130973]: pgmap v353: 129 pgs: 129 active+clean; 11 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 865 KiB/s wr, 17 op/s 2023-08-11T14:04:08.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:08 smithi103 ceph-mon[130434]: pgmap v354: 129 pgs: 129 active+clean; 14 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 28 op/s 2023-08-11T14:04:08.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:08 smithi003 ceph-mon[97195]: pgmap v354: 129 pgs: 129 active+clean; 14 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 28 op/s 2023-08-11T14:04:08.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:08 smithi187 ceph-mon[130973]: pgmap v354: 129 pgs: 129 active+clean; 14 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 28 op/s 2023-08-11T14:04:11.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:11 smithi003 ceph-mon[97195]: pgmap v355: 129 pgs: 129 active+clean; 25 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:11.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:11 smithi187 ceph-mon[130973]: pgmap v355: 129 pgs: 129 active+clean; 25 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:11 smithi103 ceph-mon[130434]: pgmap v355: 129 pgs: 129 active+clean; 25 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 -- 172.21.15.3:0/902583918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514101b40 msgr2=0x7f2514101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 --2- 172.21.15.3:0/902583918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514101b40 0x7f2514101fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25080098f0 tx=0x7f2508009c00 comp rx=0 tx=0).stop 2023-08-11T14:04:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 -- 172.21.15.3:0/902583918 shutdown_connections 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 --2- 172.21.15.3:0/902583918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25141024e0 0x7f2514108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 --2- 172.21.15.3:0/902583918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2514100950 0x7f2514100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 --2- 172.21.15.3:0/902583918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514101b40 0x7f2514101fa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 -- 172.21.15.3:0/902583918 >> 172.21.15.3:0/902583918 conn(0x7f25140fbea0 msgr2=0x7f25140fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 -- 172.21.15.3:0/902583918 shutdown_connections 2023-08-11T14:04:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.110+0000 7f251a9ff700 1 -- 172.21.15.3:0/902583918 wait complete. 2023-08-11T14:04:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.111+0000 7f251a9ff700 1 Processor -- start 2023-08-11T14:04:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 -- start start 2023-08-11T14:04:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514100950 0x7f2514198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 0x7f2514198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419a450 con 0x7f2514101b40 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419a5c0 con 0x7f25141024e0 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f251a9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251419a780 con 0x7f2514100950 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f2518f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f25137fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 0x7f2514198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f2513fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514100950 0x7f2514198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f2518f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39828/0 (socket says 172.21.15.3:39828) 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.112+0000 7f2518f9c700 1 -- 172.21.15.3:0/2985901724 learned_addr learned my addr 172.21.15.3:0/2985901724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 -- 172.21.15.3:0/2985901724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514100950 msgr2=0x7f2514198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514100950 0x7f2514198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 -- 172.21.15.3:0/2985901724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 msgr2=0x7f2514199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 -- 172.21.15.3:0/2985901724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508009520 con 0x7f2514101b40 2023-08-11T14:04:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25137fe700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 0x7f2514198f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2508009230 tx=0x7f250800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25117fa700 1 -- 172.21.15.3:0/2985901724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250801c070 con 0x7f2514101b40 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251419aa00 con 0x7f2514101b40 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25117fa700 1 -- 172.21.15.3:0/2985901724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2508005040 con 0x7f2514101b40 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f25117fa700 1 -- 172.21.15.3:0/2985901724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508003990 con 0x7f2514101b40 2023-08-11T14:04:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.113+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141a1220 con 0x7f2514101b40 2023-08-11T14:04:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.117+0000 7f25117fa700 1 -- 172.21.15.3:0/2985901724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2508020030 con 0x7f2514101b40 2023-08-11T14:04:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 Processor -- start 2023-08-11T14:04:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 -- start start 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f25117fa700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 0x7f24fc0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 0x55b6af4611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af461c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f25117fa700 1 -- 172.21.15.3:0/2985901724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f25080cc490 con 0x7f2514101b40 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af3b95d0 con 0x55b6af4608a0 2023-08-11T14:04:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af3b6ef0 con 0x55b6af45ff40 2023-08-11T14:04:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.119+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af3b27e0 con 0x55b6af461790 2023-08-11T14:04:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 0x7f24fc0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39838/0 (socket says 172.21.15.3:39838) 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 -- 172.21.15.3:0/2932269592 learned_addr learned my addr 172.21.15.3:0/2932269592 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f25137fe700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 0x55b6af4611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af461c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 -- 172.21.15.3:0/2932269592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 msgr2=0x55b6af461c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af461c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 -- 172.21.15.3:0/2932269592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 msgr2=0x55b6af4611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 0x55b6af4611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 -- 172.21.15.3:0/2932269592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6af2325d0 con 0x55b6af45ff40 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af461c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f25137fe700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 0x55b6af4611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.120+0000 7f2513fff700 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2504009d30 tx=0x7f2504011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4aba96511506cb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 0x7f24fc0a2410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b6af460f70 tx=0x7f250c009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f24fa7fc700 1 -- 172.21.15.3:0/2932269592 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504004220 con 0x55b6af45ff40 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f24fa7fc700 1 -- 172.21.15.3:0/2932269592 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2504004cf0 con 0x55b6af45ff40 2023-08-11T14:04:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f24fa7fc700 1 -- 172.21.15.3:0/2932269592 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504005540 con 0x55b6af45ff40 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2932269592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 msgr2=0x55b6af460360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.121+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2504009d30 tx=0x7f2504011720 comp rx=0 tx=0).stop 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2932269592 shutdown_connections 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af461c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af460360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2932269592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af4608a0 0x55b6af4611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2932269592 >> 172.21.15.3:0/2932269592 conn(0x55b6af38d8f0 msgr2=0x55b6af45c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2932269592 shutdown_connections 2023-08-11T14:04:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2932269592 wait complete. 2023-08-11T14:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.122+0000 7f252e2c4b80 1 -- start start 2023-08-11T14:04:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 Processor -- start 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af469bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af471130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af350a60 con 0x55b6af45ff40 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af350760 con 0x55b6af4608a0 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af2348c0 con 0x55b6af461790 2023-08-11T14:04:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f252e2c4b80 10 client.0 _ll_register_callbacks cb 0x55b6af353a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f25137fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f25137fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39850/0 (socket says 172.21.15.3:39850) 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f25137fe700 1 -- 172.21.15.3:0/3728404147 learned_addr learned my addr 172.21.15.3:0/3728404147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.123+0000 7f2513fff700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af469bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f2518f9c700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af471130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 -- 172.21.15.3:0/3728404147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 msgr2=0x55b6af471130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af471130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 -- 172.21.15.3:0/3728404147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 msgr2=0x55b6af469bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af469bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 -- 172.21.15.3:0/3728404147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141a1220 con 0x55b6af4608a0 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f2518f9c700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af471130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f25137fe700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25080960a0 tx=0x7f25080995d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d65d82add28e49 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f24f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508099bf0 con 0x55b6af4608a0 2023-08-11T14:04:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6af4537e0 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.124+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f250809da20 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25080a1970 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 10 client.25634 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f2504009960 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6af4537e0 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2508099e50 con 0x55b6af4608a0 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 10 client.25634 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 20 client.25634 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af475660 0x55b6af477a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 4 client.25634 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6af42f480 con 0x55b6af475660 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.125+0000 7f2513fff700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af475660 0x55b6af477a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.126+0000 7f2513fff700 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af475660 0x55b6af477a50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=856ab214a9fa4bd2 server_cookie=47027b20cbba8854 in_seq=0 out_seq=0 2023-08-11T14:04:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.126+0000 7f24f37fe700 10 client.25634 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.127+0000 7f24f37fe700 1 -- 172.21.15.3:0/3728404147 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b6af42f480 con 0x55b6af475660 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.127+0000 7f24f37fe700 10 client.25634 handle_command_reply: tid=0 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.127+0000 7f252e2c4b80 1 client.25634 shutdown 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.127+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 msgr2=0x55b6af46a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.127+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25080960a0 tx=0x7f25080995d0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 shutdown_connections 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af461790 0x55b6af471130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af469bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af475660 0x55b6af477a50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/3728404147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af4608a0 0x55b6af46a0f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f24f37fe700 0 client.25634 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 >> 172.21.15.3:0/3728404147 conn(0x55b6af38d8f0 msgr2=0x55b6af45b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 shutdown_connections 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 1 -- 172.21.15.3:0/3728404147 wait complete. 2023-08-11T14:04:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.128+0000 7f252e2c4b80 20 client.25634 trim_cache size 0 max 16384 2023-08-11T14:04:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.143+0000 7f252e2c4b80 1 Processor -- start 2023-08-11T14:04:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.143+0000 7f252e2c4b80 1 -- start start 2023-08-11T14:04:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af461790 0x55b6af469bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af469980 con 0x55b6af45ff40 2023-08-11T14:04:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af468350 con 0x55b6af461790 2023-08-11T14:04:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af460fa0 con 0x55b6af4608a0 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38624/0 (socket says 172.21.15.3:38624) 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 -- 172.21.15.3:0/978338762 learned_addr learned my addr 172.21.15.3:0/978338762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2518f9c700 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af461790 0x55b6af469bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f25137fe700 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 -- 172.21.15.3:0/978338762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 msgr2=0x55b6af4710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 -- 172.21.15.3:0/978338762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af461790 msgr2=0x55b6af469bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af461790 0x55b6af469bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.144+0000 7f2513fff700 1 -- 172.21.15.3:0/978338762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6af1ded80 con 0x55b6af45ff40 2023-08-11T14:04:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.145+0000 7f2513fff700 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b6af45c2f0 tx=0x7f2504009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7174405748bd6cc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.145+0000 7f24faffd700 1 -- 172.21.15.3:0/978338762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250402b070 con 0x55b6af45ff40 2023-08-11T14:04:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.145+0000 7f24faffd700 1 -- 172.21.15.3:0/978338762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2504030070 con 0x55b6af45ff40 2023-08-11T14:04:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.145+0000 7f24faffd700 1 -- 172.21.15.3:0/978338762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504004850 con 0x55b6af45ff40 2023-08-11T14:04:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- 172.21.15.3:0/978338762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 msgr2=0x55b6af461b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55b6af45c2f0 tx=0x7f2504009790 comp rx=0 tx=0).stop 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- 172.21.15.3:0/978338762 shutdown_connections 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af461790 0x55b6af469bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af45ff40 0x55b6af461b70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/978338762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- 172.21.15.3:0/978338762 >> 172.21.15.3:0/978338762 conn(0x55b6af38d8f0 msgr2=0x55b6af467c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- 172.21.15.3:0/978338762 shutdown_connections 2023-08-11T14:04:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- 172.21.15.3:0/978338762 wait complete. 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 -- start start 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.146+0000 7f252e2c4b80 1 Processor -- start 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 0x55b6af46f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af46f8d0 con 0x55b6af461790 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af46fa40 con 0x55b6af45ff40 2023-08-11T14:04:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6af46fc00 con 0x55b6af4608a0 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f252e2c4b80 10 client.0 _ll_register_callbacks cb 0x55b6af353a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f2513fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f2513fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39870/0 (socket says 172.21.15.3:39870) 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f2513fff700 1 -- 172.21.15.3:0/2483155412 learned_addr learned my addr 172.21.15.3:0/2483155412 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 0x55b6af46f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.147+0000 7f25137fe700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 -- 172.21.15.3:0/2483155412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 msgr2=0x55b6af4710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 -- 172.21.15.3:0/2483155412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 msgr2=0x55b6af46f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 0x55b6af46f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 -- 172.21.15.3:0/2483155412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6af2325d0 con 0x55b6af45ff40 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2518f9c700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 0x55b6af46f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f2513fff700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2504005950 tx=0x7f250402c190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31605eea166de396 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f24f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250402b050 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2504004cf0 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f250401f040 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2504004a80 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f252e2c4b80 10 client.25640 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.148+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b6af1c7c00 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2504004cf0 con 0x55b6af45ff40 2023-08-11T14:04:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f25040358b0 con 0x55b6af45ff40 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 20 client.25640 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af4ca450 0x55b6af4cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 4 client.25640 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b6af3ab7c0 con 0x55b6af4ca450 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.149+0000 7f25137fe700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af4ca450 0x55b6af4cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:12.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.150+0000 7f25137fe700 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af4ca450 0x55b6af4cc870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21976a66b137dae2 server_cookie=726d007edac834d in_seq=0 out_seq=0 2023-08-11T14:04:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.150+0000 7f24f37fe700 10 client.25640 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f24f37fe700 1 -- 172.21.15.3:0/2483155412 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b6af3ab7c0 con 0x55b6af4ca450 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f24f37fe700 10 client.25640 handle_command_reply: tid=0 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 client.25640 shutdown 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 msgr2=0x55b6af470b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2504005950 tx=0x7f250402c190 comp rx=0 tx=0).stop 2023-08-11T14:04:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 shutdown_connections 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6af461790 0x55b6af46f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b6af4ca450 0x55b6af4cc870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6af45ff40 0x55b6af470b70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 --2- 172.21.15.3:0/2483155412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6af4608a0 0x55b6af4710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f24f37fe700 0 client.25640 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 >> 172.21.15.3:0/2483155412 conn(0x55b6af38d8f0 msgr2=0x55b6af464790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 shutdown_connections 2023-08-11T14:04:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.478+0000 7f252e2c4b80 1 -- 172.21.15.3:0/2483155412 wait complete. 2023-08-11T14:04:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f252e2c4b80 20 client.25640 trim_cache size 0 max 16384 2023-08-11T14:04:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:04:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:04:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:04:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:04:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 msgr2=0x7f24fc0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 0x7f24fc0a2410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55b6af460f70 tx=0x7f250c009920 comp rx=0 tx=0).stop 2023-08-11T14:04:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 msgr2=0x7f2514198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 0x7f2514198f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2508009230 tx=0x7f250800b560 comp rx=0 tx=0).stop 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 shutdown_connections 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24fc09ff50 0x7f24fc0a2410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25141024e0 0x7f2514199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2514100950 0x7f2514198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.479+0000 7f251a9ff700 1 --2- 172.21.15.3:0/2985901724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2514101b40 0x7f2514198f70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.480+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 >> 172.21.15.3:0/2985901724 conn(0x7f25140fbea0 msgr2=0x7f25141074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.480+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 shutdown_connections 2023-08-11T14:04:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:12.480+0000 7f251a9ff700 1 -- 172.21.15.3:0/2985901724 wait complete. 2023-08-11T14:04:12.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:12 smithi103 ceph-mon[130434]: scrub summary: idle 2023-08-11T14:04:12.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:12 smithi003 ceph-mon[97195]: scrub summary: idle 2023-08-11T14:04:12.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:12 smithi187 ceph-mon[130973]: scrub summary: idle 2023-08-11T14:04:13.519 INFO:teuthology.orchestra.run.smithi003.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-08-11T14:04:13.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-08-11T14:04:13.612 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:13 smithi003 ceph-mon[97195]: pgmap v356: 129 pgs: 129 active+clean; 25 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:13.640 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/: 2023-08-11T14:04:13.641 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.641 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 . 2023-08-11T14:04:13.641 INFO:teuthology.orchestra.run.smithi003.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:04:13.641 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 data 2023-08-11T14:04:13.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.642 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data: 2023-08-11T14:04:13.642 INFO:teuthology.orchestra.run.smithi003.stdout:total 30 2023-08-11T14:04:13.642 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 . 2023-08-11T14:04:13.643 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:04:13.643 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 base 2023-08-11T14:04:13.644 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 11 14:04 global 2023-08-11T14:04:13.644 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 log 2023-08-11T14:04:13.644 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_commit_ts 2023-08-11T14:04:13.644 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_dynshmem 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Aug 11 14:04 pg_hba.conf 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 11 14:04 pg_ident.conf 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 pg_logical 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 pg_multixact 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 pg_notify 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_replslot 2023-08-11T14:04:13.645 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_serial 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_snapshots 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_stat 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_stat_tmp 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 pg_subtrans 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_tblspc 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_twophase 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:04:13.646 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 pg_wal 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 pg_xact 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 11 14:04 postgresql.auto.conf 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Aug 11 14:04 postgresql.conf 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base: 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 . 2023-08-11T14:04:13.647 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 1 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 12346 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 12347 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/1: 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:total 7209 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 . 2023-08-11T14:04:13.648 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 .. 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:04:13.649 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:04:13.650 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:04:13.651 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:04:13.652 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:04:13.653 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:04 1249 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:04 1259 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:04:13.654 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:04:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:04:13.656 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:04:13.657 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:04:13.658 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606_vm 2023-08-11T14:04:13.659 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 11 14:04 2608 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2608_fsm 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2608_vm 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:04:13.660 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610_fsm 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2610_vm 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:04:13.661 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:04:13.662 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:04:13.663 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 11 14:04 2619 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2619_fsm 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2619_vm 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:04:13.664 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 11 14:04 2658 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2659 2023-08-11T14:04:13.665 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2662 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2663 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2664 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2665 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2666 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2667 2023-08-11T14:04:13.666 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:04 2673 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:04 2674 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2678 2023-08-11T14:04:13.667 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2679 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:04:13.668 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2696 2023-08-11T14:04:13.669 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:04:13.670 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:04:13.671 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:04:13.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:04:13.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:04:13.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:04:13.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:04:13.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:04:13.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 3455 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:04:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:04:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:04:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:04:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:04:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:04:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:04:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:04:13.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:13 smithi103 ceph-mon[130434]: pgmap v356: 129 pgs: 129 active+clean; 25 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:13.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:04:13.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:04:13.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:04:13.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:04:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:04:13.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-08-11T14:04:13.690 INFO:teuthology.orchestra.run.smithi003.stdout:total 7209 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 . 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 .. 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:04:13.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:04:13.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:04:13.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:04:13.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:04:13.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:04 1249 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:04:13.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:04 1259 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:04:13.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:04:13.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:04:13.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:04:13.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:04:13.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606_vm 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 11 14:04 2608 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2608_fsm 2023-08-11T14:04:13.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2608_vm 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610_fsm 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2610_vm 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:04:13.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:04:13.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:04:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 11 14:04 2619 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2619_fsm 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2619_vm 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:04:13.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:04:13.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 11 14:04 2658 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2659 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2662 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2663 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2664 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2665 2023-08-11T14:04:13.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2666 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2667 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:04 2673 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:04 2674 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:04:13.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2678 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2679 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:04:13.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:04:13.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2696 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:04:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:04:13.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:04:13.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:04:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:04:13.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:04:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:04:13.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:04:13.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:04:13.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:04:13.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:04:13.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:04:13.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:04:13.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:04:13.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:04:13.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:04:13.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 3455 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:04:13.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:04:13.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:04:13.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:04:13.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:04:13.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:04:13.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:04:13.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:04:13.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:04:13.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:04:13.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:total 7209 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 . 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 .. 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:04:13.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:04:13.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:04:13.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:04:13.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:04:13.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:04 1249 2023-08-11T14:04:13.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:04 1259 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:04:13.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:04:13.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:04:13.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:04:13.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:04:13.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606_vm 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:04:13.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 11 14:04 2608 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2608_fsm 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2608_vm 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610 2023-08-11T14:04:13.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610_fsm 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2610_vm 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:04:13.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:04:13.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 11 14:04 2619 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2619_fsm 2023-08-11T14:04:13.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2619_vm 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:04:13.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 11 14:04 2658 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2659 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:04:13.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2662 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2663 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2664 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2665 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2666 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2667 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:04:13.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:04 2673 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:04 2674 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2678 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2679 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:04:13.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:04:13.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2696 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:04:13.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:04:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:04:13.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:04:13.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:04:13.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:04:13.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:04:13.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:04:13.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:04:13.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 3455 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:04:13.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:04:13.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:04:13.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:04:13.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:04:13.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:04:13.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:04:13.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:04:13.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:04:13.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/global: 2023-08-11T14:04:13.770 INFO:teuthology.orchestra.run.smithi003.stdout:total 553 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 11 14:04 . 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1136 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1136_fsm 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1136_vm 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1137 2023-08-11T14:04:13.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1213 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1213_fsm 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1213_vm 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1214 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1214_fsm 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1214_vm 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1232 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1233 2023-08-11T14:04:13.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1260 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1260_fsm 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1260_vm 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1261 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1261_fsm 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1261_vm 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1262 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1262_fsm 2023-08-11T14:04:13.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1262_vm 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2396 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2396_fsm 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2396_vm 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2397 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2671 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2672 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2676 2023-08-11T14:04:13.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2677 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2694 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2695 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2697 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2698 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2846 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2846_vm 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2847 2023-08-11T14:04:13.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2964 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2964_vm 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2965 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2966 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2966_vm 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2967 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3592 2023-08-11T14:04:13.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3592_vm 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3593 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 4060 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 4060_vm 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 4061 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6000 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6000_vm 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6001 2023-08-11T14:04:13.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6002 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6100 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6100_vm 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6114 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6115 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 pg_control 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/log: 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-08-11T14:04:13.779 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.780 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:total 1 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 . 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 mappings 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 11 14:04 replorigin_checkpoint 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 snapshots 2023-08-11T14:04:13.781 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 .. 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 .. 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 . 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 members 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 offsets 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 .. 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:04:13.784 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 .. 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-08-11T14:04:13.785 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-08-11T14:04:13.788 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.789 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:04:13.790 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.791 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:total 16384 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 . 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:04 000000010000000000000001 2023-08-11T14:04:13.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 archive_status 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 .. 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:13.793 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-08-11T14:04:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:04:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:04:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 11 14:04 .. 2023-08-11T14:04:13.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:04:13.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-08-11T14:04:13.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:13 smithi187 ceph-mon[130973]: pgmap v356: 129 pgs: 129 active+clean; 25 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.1 MiB/s wr, 36 op/s 2023-08-11T14:04:14.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-08-11T14:04:14.156 INFO:teuthology.orchestra.run.smithi003.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-08-11T14:04:14.163 INFO:teuthology.orchestra.run.smithi003.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-08-11T14:04:14.168 INFO:teuthology.orchestra.run.smithi003.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-08-11T14:04:14.174 INFO:teuthology.orchestra.run.smithi003.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-08-11T14:04:14.498 INFO:teuthology.orchestra.run.smithi003.stderr:creating tables... 2023-08-11T14:04:14.609 INFO:teuthology.orchestra.run.smithi003.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.11 s, remaining 53.08 s) 2023-08-11T14:04:14.878 INFO:teuthology.orchestra.run.smithi003.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.38 s, remaining 94.25 s) 2023-08-11T14:04:15.262 INFO:teuthology.orchestra.run.smithi003.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.76 s, remaining 126.21 s) 2023-08-11T14:04:15.365 INFO:teuthology.orchestra.run.smithi003.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.87 s, remaining 107.27 s) 2023-08-11T14:04:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:15 smithi103 ceph-mon[130434]: pgmap v357: 129 pgs: 129 active+clean; 30 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 176 op/s 2023-08-11T14:04:15.682 INFO:teuthology.orchestra.run.smithi003.stderr:500000 of 50000000 tuples (1%) done (elapsed 1.18 s, remaining 116.99 s) 2023-08-11T14:04:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:15 smithi003 ceph-mon[97195]: pgmap v357: 129 pgs: 129 active+clean; 30 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 176 op/s 2023-08-11T14:04:15.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:15 smithi187 ceph-mon[130973]: pgmap v357: 129 pgs: 129 active+clean; 30 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 2.8 MiB/s wr, 176 op/s 2023-08-11T14:04:15.885 INFO:teuthology.orchestra.run.smithi003.stderr:600000 of 50000000 tuples (1%) done (elapsed 1.38 s, remaining 114.02 s) 2023-08-11T14:04:16.182 INFO:teuthology.orchestra.run.smithi003.stderr:700000 of 50000000 tuples (1%) done (elapsed 1.68 s, remaining 118.50 s) 2023-08-11T14:04:16.473 INFO:teuthology.orchestra.run.smithi003.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.97 s, remaining 121.34 s) 2023-08-11T14:04:16.626 INFO:teuthology.orchestra.run.smithi003.stderr:900000 of 50000000 tuples (1%) done (elapsed 2.13 s, remaining 115.97 s) 2023-08-11T14:04:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:1000000 of 50000000 tuples (2%) done (elapsed 2.41 s, remaining 117.90 s) 2023-08-11T14:04:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:1100000 of 50000000 tuples (2%) done (elapsed 2.72 s, remaining 120.82 s) 2023-08-11T14:04:17.411 INFO:teuthology.orchestra.run.smithi003.stderr:1200000 of 50000000 tuples (2%) done (elapsed 2.91 s, remaining 118.36 s) 2023-08-11T14:04:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:17 smithi103 ceph-mon[130434]: pgmap v358: 129 pgs: 129 active+clean; 40 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2023-08-11T14:04:17.692 INFO:teuthology.orchestra.run.smithi003.stderr:1300000 of 50000000 tuples (2%) done (elapsed 3.19 s, remaining 119.57 s) 2023-08-11T14:04:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:17 smithi003 ceph-mon[97195]: pgmap v358: 129 pgs: 129 active+clean; 40 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2023-08-11T14:04:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:17 smithi187 ceph-mon[130973]: pgmap v358: 129 pgs: 129 active+clean; 40 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 4.3 MiB/s wr, 261 op/s 2023-08-11T14:04:17.994 INFO:teuthology.orchestra.run.smithi003.stderr:1400000 of 50000000 tuples (2%) done (elapsed 3.49 s, remaining 121.30 s) 2023-08-11T14:04:18.172 INFO:teuthology.orchestra.run.smithi003.stderr:1500000 of 50000000 tuples (3%) done (elapsed 3.67 s, remaining 118.72 s) 2023-08-11T14:04:18.480 INFO:teuthology.orchestra.run.smithi003.stderr:1600000 of 50000000 tuples (3%) done (elapsed 3.98 s, remaining 120.40 s) 2023-08-11T14:04:18.791 INFO:teuthology.orchestra.run.smithi003.stderr:1700000 of 50000000 tuples (3%) done (elapsed 4.29 s, remaining 121.91 s) 2023-08-11T14:04:18.967 INFO:teuthology.orchestra.run.smithi003.stderr:1800000 of 50000000 tuples (3%) done (elapsed 4.47 s, remaining 119.61 s) 2023-08-11T14:04:19.228 INFO:teuthology.orchestra.run.smithi003.stderr:1900000 of 50000000 tuples (3%) done (elapsed 4.73 s, remaining 119.68 s) 2023-08-11T14:04:19.498 INFO:teuthology.orchestra.run.smithi003.stderr:2000000 of 50000000 tuples (4%) done (elapsed 5.00 s, remaining 119.91 s) 2023-08-11T14:04:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2100000 of 50000000 tuples (4%) done (elapsed 5.13 s, remaining 116.92 s) 2023-08-11T14:04:19.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:19 smithi103 ceph-mon[130434]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 12 MiB/s wr, 372 op/s 2023-08-11T14:04:19.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:19 smithi003 ceph-mon[97195]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 12 MiB/s wr, 372 op/s 2023-08-11T14:04:19.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:19 smithi187 ceph-mon[130973]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 12 MiB/s wr, 372 op/s 2023-08-11T14:04:19.963 INFO:teuthology.orchestra.run.smithi003.stderr:2200000 of 50000000 tuples (4%) done (elapsed 5.46 s, remaining 118.66 s) 2023-08-11T14:04:20.225 INFO:teuthology.orchestra.run.smithi003.stderr:2300000 of 50000000 tuples (4%) done (elapsed 5.72 s, remaining 118.73 s) 2023-08-11T14:04:20.341 INFO:teuthology.orchestra.run.smithi003.stderr:2400000 of 50000000 tuples (4%) done (elapsed 5.84 s, remaining 115.84 s) 2023-08-11T14:04:20.740 INFO:teuthology.orchestra.run.smithi003.stderr:2500000 of 50000000 tuples (5%) done (elapsed 6.24 s, remaining 118.56 s) 2023-08-11T14:04:20.869 INFO:teuthology.orchestra.run.smithi003.stderr:2600000 of 50000000 tuples (5%) done (elapsed 6.37 s, remaining 116.11 s) 2023-08-11T14:04:21.190 INFO:teuthology.orchestra.run.smithi003.stderr:2700000 of 50000000 tuples (5%) done (elapsed 6.69 s, remaining 117.19 s) 2023-08-11T14:04:21.385 INFO:teuthology.orchestra.run.smithi003.stderr:2800000 of 50000000 tuples (5%) done (elapsed 6.88 s, remaining 116.06 s) 2023-08-11T14:04:21.660 INFO:teuthology.orchestra.run.smithi003.stderr:2900000 of 50000000 tuples (5%) done (elapsed 7.16 s, remaining 116.28 s) 2023-08-11T14:04:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:21 smithi103 ceph-mon[130434]: pgmap v360: 129 pgs: 129 active+clean; 135 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 367 op/s 2023-08-11T14:04:21.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:21 smithi003 ceph-mon[97195]: pgmap v360: 129 pgs: 129 active+clean; 135 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 367 op/s 2023-08-11T14:04:21.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:21 smithi187 ceph-mon[130973]: pgmap v360: 129 pgs: 129 active+clean; 135 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 19 MiB/s wr, 367 op/s 2023-08-11T14:04:21.998 INFO:teuthology.orchestra.run.smithi003.stderr:3000000 of 50000000 tuples (6%) done (elapsed 7.50 s, remaining 117.47 s) 2023-08-11T14:04:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:3100000 of 50000000 tuples (6%) done (elapsed 7.67 s, remaining 115.99 s) 2023-08-11T14:04:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:3200000 of 50000000 tuples (6%) done (elapsed 7.95 s, remaining 116.21 s) 2023-08-11T14:04:22.715 INFO:teuthology.orchestra.run.smithi003.stderr:3300000 of 50000000 tuples (6%) done (elapsed 8.22 s, remaining 116.25 s) 2023-08-11T14:04:22.886 INFO:teuthology.orchestra.run.smithi003.stderr:3400000 of 50000000 tuples (6%) done (elapsed 8.39 s, remaining 114.94 s) 2023-08-11T14:04:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:3500000 of 50000000 tuples (7%) done (elapsed 8.67 s, remaining 115.20 s) 2023-08-11T14:04:23.446 INFO:teuthology.orchestra.run.smithi003.stderr:3600000 of 50000000 tuples (7%) done (elapsed 8.95 s, remaining 115.31 s) 2023-08-11T14:04:23.645 INFO:teuthology.orchestra.run.smithi003.stderr:3700000 of 50000000 tuples (7%) done (elapsed 9.15 s, remaining 114.44 s) 2023-08-11T14:04:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:23 smithi103 ceph-mon[130434]: pgmap v361: 129 pgs: 129 active+clean; 291 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 44 MiB/s wr, 366 op/s 2023-08-11T14:04:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:23 smithi003 ceph-mon[97195]: pgmap v361: 129 pgs: 129 active+clean; 291 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 44 MiB/s wr, 366 op/s 2023-08-11T14:04:23.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:23 smithi187 ceph-mon[130973]: pgmap v361: 129 pgs: 129 active+clean; 291 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 44 MiB/s wr, 366 op/s 2023-08-11T14:04:23.898 INFO:teuthology.orchestra.run.smithi003.stderr:3800000 of 50000000 tuples (7%) done (elapsed 9.40 s, remaining 114.26 s) 2023-08-11T14:04:24.255 INFO:teuthology.orchestra.run.smithi003.stderr:3900000 of 50000000 tuples (7%) done (elapsed 9.76 s, remaining 115.31 s) 2023-08-11T14:04:24.446 INFO:teuthology.orchestra.run.smithi003.stderr:4000000 of 50000000 tuples (8%) done (elapsed 9.95 s, remaining 114.38 s) 2023-08-11T14:04:24.742 INFO:teuthology.orchestra.run.smithi003.stderr:4100000 of 50000000 tuples (8%) done (elapsed 10.24 s, remaining 114.65 s) 2023-08-11T14:04:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:4200000 of 50000000 tuples (8%) done (elapsed 10.54 s, remaining 114.96 s) 2023-08-11T14:04:25.149 INFO:teuthology.orchestra.run.smithi003.stderr:4300000 of 50000000 tuples (8%) done (elapsed 10.65 s, remaining 113.18 s) 2023-08-11T14:04:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:4400000 of 50000000 tuples (8%) done (elapsed 11.00 s, remaining 114.03 s) 2023-08-11T14:04:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 -- 172.21.15.3:0/2830104786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c40ff820 msgr2=0x7f83c40ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2830104786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c40ff820 0x7f83c40ffc40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83b40099c0 tx=0x7f83b4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 -- 172.21.15.3:0/2830104786 shutdown_connections 2023-08-11T14:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2830104786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c41013b0 0x7f83c4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2830104786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c40ff820 0x7f83c40ffc40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2830104786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 -- 172.21.15.3:0/2830104786 >> 172.21.15.3:0/2830104786 conn(0x7f83c40fad70 msgr2=0x7f83c40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 -- 172.21.15.3:0/2830104786 shutdown_connections 2023-08-11T14:04:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.634+0000 7f83c91ff700 1 -- 172.21.15.3:0/2830104786 wait complete. 2023-08-11T14:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.635+0000 7f83c91ff700 1 Processor -- start 2023-08-11T14:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.635+0000 7f83c91ff700 1 -- start start 2023-08-11T14:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.635+0000 7f83c91ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c40ff820 0x7f83c4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.635+0000 7f83c91ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c91ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c91ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419a370 con 0x7f83c40ff820 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c91ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419a4e0 con 0x7f83c41013b0 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c91ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419a6a0 con 0x7f83c4100a10 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c40ff820 0x7f83c4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60962/0 (socket says 172.21.15.3:60962) 2023-08-11T14:04:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 -- 172.21.15.3:0/2871520075 learned_addr learned my addr 172.21.15.3:0/2871520075 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 -- 172.21.15.3:0/2871520075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 msgr2=0x7f83c4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 -- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c40ff820 msgr2=0x7f83c4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c40ff820 0x7f83c4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c359e700 1 -- 172.21.15.3:0/2871520075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40095f0 con 0x7f83c41013b0 2023-08-11T14:04:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.636+0000 7f83c259c700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83c359e700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00ef30 tx=0x7f83bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83abfff700 1 -- 172.21.15.3:0/2871520075 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc018070 con 0x7f83c41013b0 2023-08-11T14:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83abfff700 1 -- 172.21.15.3:0/2871520075 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83bc00f040 con 0x7f83c41013b0 2023-08-11T14:04:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c419a920 con 0x7f83c41013b0 2023-08-11T14:04:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83abfff700 1 -- 172.21.15.3:0/2871520075 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc00cd00 con 0x7f83c41013b0 2023-08-11T14:04:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.637+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c41a1140 con 0x7f83c41013b0 2023-08-11T14:04:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.640+0000 7f83abfff700 1 -- 172.21.15.3:0/2871520075 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f83bc003a00 con 0x7f83c41013b0 2023-08-11T14:04:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.642+0000 7f83abfff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 0x7f83ac0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.642+0000 7f83abfff700 1 -- 172.21.15.3:0/2871520075 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83bc0cb0f0 con 0x7f83c41013b0 2023-08-11T14:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.642+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 0x7f83ac0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.643+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 0x7f83ac0a2410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009990 tx=0x7f83b4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 Processor -- start 2023-08-11T14:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 -- start start 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8984360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8985790 0x5626a8985c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a88dd5d0 con 0x5626a89848a0 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a88daef0 con 0x5626a8983f40 2023-08-11T14:04:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.644+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a88d67e0 con 0x5626a8985790 2023-08-11T14:04:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.645+0000 7f83c259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55948/0 (socket says 172.21.15.3:55948) 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 -- 172.21.15.3:0/2931911359 learned_addr learned my addr 172.21.15.3:0/2931911359 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8984360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 -- 172.21.15.3:0/2931911359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8985790 msgr2=0x5626a8985c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8985790 0x5626a8985c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 -- 172.21.15.3:0/2931911359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 msgr2=0x5626a8984360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.646+0000 7f83c259c700 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8984360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.647+0000 7f83c259c700 1 -- 172.21.15.3:0/2931911359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a87565d0 con 0x5626a89848a0 2023-08-11T14:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.647+0000 7f83c259c700 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83b80115f0 tx=0x7f83b8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335b04516b85ac09 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.648+0000 7f83a97fa700 1 -- 172.21.15.3:0/2931911359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800f680 con 0x5626a89848a0 2023-08-11T14:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.648+0000 7f83a97fa700 1 -- 172.21.15.3:0/2931911359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b800fca0 con 0x5626a89848a0 2023-08-11T14:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.648+0000 7f83a97fa700 1 -- 172.21.15.3:0/2931911359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8019d90 con 0x5626a89848a0 2023-08-11T14:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.649+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2931911359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 msgr2=0x5626a89851b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.649+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83b80115f0 tx=0x7f83b8011900 comp rx=0 tx=0).stop 2023-08-11T14:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.651+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2931911359 shutdown_connections 2023-08-11T14:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.651+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8985790 0x5626a8985c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.651+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8984360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.651+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2931911359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a89851b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.651+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2931911359 >> 172.21.15.3:0/2931911359 conn(0x5626a88b18f0 msgr2=0x5626a8980920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.652+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2931911359 shutdown_connections 2023-08-11T14:04:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.652+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2931911359 wait complete. 2023-08-11T14:04:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.652+0000 7f83dca9fb80 1 -- start start 2023-08-11T14:04:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.653+0000 7f83dca9fb80 1 Processor -- start 2023-08-11T14:04:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a89969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a8874a60 con 0x5626a8983f40 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a8874760 con 0x5626a8985790 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a87588c0 con 0x5626a89848a0 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.654+0000 7f83dca9fb80 10 client.0 _ll_register_callbacks cb 0x5626a8877a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55950/0 (socket says 172.21.15.3:55950) 2023-08-11T14:04:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 -- 172.21.15.3:0/1568506821 learned_addr learned my addr 172.21.15.3:0/1568506821 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c259c700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a89969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 -- 172.21.15.3:0/1568506821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 msgr2=0x5626a89969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a89969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 -- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 msgr2=0x5626a899da00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.655+0000 7f83c2d9d700 1 -- 172.21.15.3:0/1568506821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b8009dd0 con 0x5626a8983f40 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.656+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f83b4015000 tx=0x7f83b4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4581209360fb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.656+0000 7f83a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.656+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400f460 con 0x5626a8983f40 2023-08-11T14:04:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.656+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83b400f5c0 con 0x5626a8983f40 2023-08-11T14:04:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.657+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626a89777e0 con 0x5626a8983f40 2023-08-11T14:04:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.657+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b400f780 con 0x5626a8983f40 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.657+0000 7f83dca9fb80 10 client.15618 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.657+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626a8702d80 con 0x5626a8983f40 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626a89777e0 con 0x5626a8983f40 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f83b4036070 con 0x5626a8983f40 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 20 client.15618 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89a22d0 0x5626a89a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 4 client.15618 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.658+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626a8953480 con 0x5626a89a22d0 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.659+0000 7f83c259c700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89a22d0 0x5626a89a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.659+0000 7f83c259c700 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89a22d0 0x5626a89a46c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd58ca51cb3b792 server_cookie=37db9d70870cd09d in_seq=0 out_seq=0 2023-08-11T14:04:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.659+0000 7f83a6ff5700 10 client.15618 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.660+0000 7f83a6ff5700 1 -- 172.21.15.3:0/1568506821 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5626a8953480 con 0x5626a89a22d0 2023-08-11T14:04:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83a6ff5700 10 client.15618 handle_command_reply: tid=0 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 client.15618 shutdown 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 msgr2=0x5626a8996480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f83b4015000 tx=0x7f83b4009310 comp rx=0 tx=0).stop 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 shutdown_connections 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89a22d0 0x5626a89a46c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8983f40 0x5626a8996480 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/1568506821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a89969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 >> 172.21.15.3:0/1568506821 conn(0x5626a88b18f0 msgr2=0x5626a897f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83a6ff5700 0 client.15618 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 shutdown_connections 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.661+0000 7f83dca9fb80 1 -- 172.21.15.3:0/1568506821 wait complete. 2023-08-11T14:04:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.662+0000 7f83dca9fb80 20 client.15618 trim_cache size 0 max 16384 2023-08-11T14:04:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:25 smithi103 ceph-mon[130434]: pgmap v362: 129 pgs: 129 active+clean; 339 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 52 MiB/s wr, 380 op/s 2023-08-11T14:04:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 Processor -- start 2023-08-11T14:04:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 -- start start 2023-08-11T14:04:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8988670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8985790 0x5626a8988bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a897ffc0 con 0x5626a8985790 2023-08-11T14:04:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a8985610 con 0x5626a8983f40 2023-08-11T14:04:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a89777c0 con 0x5626a89848a0 2023-08-11T14:04:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.678+0000 7f83c259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40368/0 (socket says 172.21.15.3:40368) 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8985790 0x5626a8988bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 -- 172.21.15.3:0/2723734695 learned_addr learned my addr 172.21.15.3:0/2723734695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8988670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 -- 172.21.15.3:0/2723734695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 msgr2=0x5626a8988670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8988670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 -- 172.21.15.3:0/2723734695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8985790 msgr2=0x5626a8988bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8985790 0x5626a8988bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 -- 172.21.15.3:0/2723734695 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a86ebc00 con 0x5626a89848a0 2023-08-11T14:04:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83c259c700 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83b801c040 tx=0x7f83b8009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e4757b1c2b4208 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83a9ffb700 1 -- 172.21.15.3:0/2723734695 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8019770 con 0x5626a89848a0 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83a9ffb700 1 -- 172.21.15.3:0/2723734695 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b800fe90 con 0x5626a89848a0 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.679+0000 7f83a9ffb700 1 -- 172.21.15.3:0/2723734695 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80052a0 con 0x5626a89848a0 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2723734695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 msgr2=0x5626a899d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83b801c040 tx=0x7f83b8009f60 comp rx=0 tx=0).stop 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2723734695 shutdown_connections 2023-08-11T14:04:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a8985790 0x5626a8988bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8983f40 0x5626a8988670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/2723734695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a89848a0 0x5626a899d9c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2723734695 >> 172.21.15.3:0/2723734695 conn(0x5626a88b18f0 msgr2=0x5626a898bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2723734695 shutdown_connections 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.680+0000 7f83dca9fb80 1 -- 172.21.15.3:0/2723734695 wait complete. 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 -- start start 2023-08-11T14:04:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 Processor -- start 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a8999820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a899cd80 con 0x5626a89848a0 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a899cef0 con 0x5626a8985790 2023-08-11T14:04:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.681+0000 7f83dca9fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a899d0b0 con 0x5626a8983f40 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83dca9fb80 10 client.0 _ll_register_callbacks cb 0x5626a8877a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a8999820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40384/0 (socket says 172.21.15.3:40384) 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 -- 172.21.15.3:0/3092561144 learned_addr learned my addr 172.21.15.3:0/3092561144 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c359e700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 -- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 msgr2=0x5626a899c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 -- 172.21.15.3:0/3092561144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 msgr2=0x5626a8999820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a8999820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 -- 172.21.15.3:0/3092561144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626a87565d0 con 0x5626a8983f40 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83c2d9d700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f83b40058e0 tx=0x7f83b4022800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199e28e656a6220f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.682+0000 7f83a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400f660 con 0x5626a8983f40 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83b400b970 con 0x5626a8983f40 2023-08-11T14:04:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83b800fca0 con 0x5626a8983f40 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b400f7c0 con 0x5626a8983f40 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83dca9fb80 10 client.25651 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626a87291f0 con 0x5626a8983f40 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b800fca0 con 0x5626a8983f40 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.683+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f83b4036050 con 0x5626a8983f40 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 10 client.25651 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 20 client.25651 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89f66b0 0x5626a89f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 4 client.25651 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626a88cf7c0 con 0x5626a89f66b0 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83c259c700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89f66b0 0x5626a89f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83c259c700 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89f66b0 0x5626a89f8ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5ae1f8bfeb7f9 server_cookie=6d4814b0796e04dc in_seq=0 out_seq=0 2023-08-11T14:04:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.684+0000 7f83a6ff5700 10 client.25651 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:25 smithi003 ceph-mon[97195]: pgmap v362: 129 pgs: 129 active+clean; 339 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 52 MiB/s wr, 380 op/s 2023-08-11T14:04:25.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:25 smithi187 ceph-mon[130973]: pgmap v362: 129 pgs: 129 active+clean; 339 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 52 MiB/s wr, 380 op/s 2023-08-11T14:04:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83a6ff5700 1 -- 172.21.15.3:0/3092561144 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626a88cf7c0 con 0x5626a89f66b0 2023-08-11T14:04:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83a6ff5700 10 client.25651 handle_command_reply: tid=0 2023-08-11T14:04:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 client.25651 shutdown 2023-08-11T14:04:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 msgr2=0x5626a89992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f83b40058e0 tx=0x7f83b4022800 comp rx=0 tx=0).stop 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 shutdown_connections 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626a89f66b0 0x5626a89f8ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626a8985790 0x5626a899c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626a8983f40 0x5626a89992e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 --2- 172.21.15.3:0/3092561144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626a89848a0 0x5626a8999820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83a6ff5700 0 client.25651 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.799+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 >> 172.21.15.3:0/3092561144 conn(0x5626a88b18f0 msgr2=0x5626a8988670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.800+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 shutdown_connections 2023-08-11T14:04:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.800+0000 7f83dca9fb80 1 -- 172.21.15.3:0/3092561144 wait complete. 2023-08-11T14:04:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.800+0000 7f83dca9fb80 20 client.25651 trim_cache size 0 max 16384 2023-08-11T14:04:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:04:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:04:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:04:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:04:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 msgr2=0x7f83ac0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 0x7f83ac0a2410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009990 tx=0x7f83b4005fb0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 msgr2=0x7f83c4199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00ef30 tx=0x7f83bc00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 shutdown_connections 2023-08-11T14:04:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83ac09ff50 0x7f83ac0a2410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83c41013b0 0x7f83c4199d10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83c40ff820 0x7f83c4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 --2- 172.21.15.3:0/2871520075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83c4100a10 0x7f83c4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 >> 172.21.15.3:0/2871520075 conn(0x7f83c40fad70 msgr2=0x7f83c4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 shutdown_connections 2023-08-11T14:04:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:25.801+0000 7f83c91ff700 1 -- 172.21.15.3:0/2871520075 wait complete. 2023-08-11T14:04:25.872 INFO:teuthology.orchestra.run.smithi003.stderr:4500000 of 50000000 tuples (9%) done (elapsed 11.37 s, remaining 114.99 s) 2023-08-11T14:04:25.989 INFO:teuthology.orchestra.run.smithi003.stderr:4600000 of 50000000 tuples (9%) done (elapsed 11.49 s, remaining 113.39 s) 2023-08-11T14:04:26.332 INFO:teuthology.orchestra.run.smithi003.stderr:4700000 of 50000000 tuples (9%) done (elapsed 11.83 s, remaining 114.04 s) 2023-08-11T14:04:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:4800000 of 50000000 tuples (9%) done (elapsed 12.18 s, remaining 114.72 s) 2023-08-11T14:04:26.797 INFO:teuthology.orchestra.run.smithi003.stderr:4900000 of 50000000 tuples (9%) done (elapsed 12.30 s, remaining 113.19 s) 2023-08-11T14:04:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:5000000 of 50000000 tuples (10%) done (elapsed 12.64 s, remaining 113.80 s) 2023-08-11T14:04:27.252 INFO:teuthology.orchestra.run.smithi003.stderr:5100000 of 50000000 tuples (10%) done (elapsed 12.75 s, remaining 112.27 s) 2023-08-11T14:04:27.597 INFO:teuthology.orchestra.run.smithi003.stderr:5200000 of 50000000 tuples (10%) done (elapsed 13.10 s, remaining 112.84 s) 2023-08-11T14:04:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:27 smithi103 ceph-mon[130434]: pgmap v363: 129 pgs: 129 active+clean; 387 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 243 op/s 2023-08-11T14:04:27.781 INFO:teuthology.orchestra.run.smithi003.stderr:5300000 of 50000000 tuples (10%) done (elapsed 13.28 s, remaining 112.01 s) 2023-08-11T14:04:27.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:27 smithi003 ceph-mon[97195]: pgmap v363: 129 pgs: 129 active+clean; 387 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 243 op/s 2023-08-11T14:04:27.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:27 smithi187 ceph-mon[130973]: pgmap v363: 129 pgs: 129 active+clean; 387 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 60 MiB/s wr, 243 op/s 2023-08-11T14:04:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:5400000 of 50000000 tuples (10%) done (elapsed 13.55 s, remaining 111.92 s) 2023-08-11T14:04:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:5500000 of 50000000 tuples (11%) done (elapsed 13.86 s, remaining 112.11 s) 2023-08-11T14:04:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:5600000 of 50000000 tuples (11%) done (elapsed 14.03 s, remaining 111.20 s) 2023-08-11T14:04:28.850 INFO:teuthology.orchestra.run.smithi003.stderr:5700000 of 50000000 tuples (11%) done (elapsed 14.35 s, remaining 111.53 s) 2023-08-11T14:04:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:5800000 of 50000000 tuples (11%) done (elapsed 14.68 s, remaining 111.84 s) 2023-08-11T14:04:29.389 INFO:teuthology.orchestra.run.smithi003.stderr:5900000 of 50000000 tuples (11%) done (elapsed 14.89 s, remaining 111.29 s) 2023-08-11T14:04:29.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-11T14:04:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:29 smithi103 ceph-mon[130434]: pgmap v364: 129 pgs: 129 active+clean; 499 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 76 MiB/s wr, 196 op/s 2023-08-11T14:04:29.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.755+0000 7f52f89ff700 1 -- 172.21.15.3:0/3596258857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4101390 msgr2=0x7f52f4103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.755+0000 7f52f89ff700 1 --2- 172.21.15.3:0/3596258857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4101390 0x7f52f4103780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009990 tx=0x7f52e4009ca0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 -- 172.21.15.3:0/3596258857 shutdown_connections 2023-08-11T14:04:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 --2- 172.21.15.3:0/3596258857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 0x7f52f4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 --2- 172.21.15.3:0/3596258857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4101390 0x7f52f4103780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 --2- 172.21.15.3:0/3596258857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4103cc0 0x7f52f41060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 -- 172.21.15.3:0/3596258857 >> 172.21.15.3:0/3596258857 conn(0x7f52f40fad80 msgr2=0x7f52f40fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 -- 172.21.15.3:0/3596258857 shutdown_connections 2023-08-11T14:04:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.756+0000 7f52f89ff700 1 -- 172.21.15.3:0/3596258857 wait complete. 2023-08-11T14:04:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.757+0000 7f52f89ff700 1 Processor -- start 2023-08-11T14:04:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 -- start start 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 0x7f52f4198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 0x7f52f419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41a03e0 con 0x7f52f4103cc0 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41a0520 con 0x7f52f41065f0 2023-08-11T14:04:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f89ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f41a06b0 con 0x7f52f4101390 2023-08-11T14:04:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.758+0000 7f52f259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 0x7f52f419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40398/0 (socket says 172.21.15.3:40398) 2023-08-11T14:04:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 -- 172.21.15.3:0/634333068 learned_addr learned my addr 172.21.15.3:0/634333068 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 0x7f52f4198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 -- 172.21.15.3:0/634333068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 msgr2=0x7f52f419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 0x7f52f419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 -- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 msgr2=0x7f52f4198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 0x7f52f4198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 -- 172.21.15.3:0/634333068 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8009520 con 0x7f52f4101390 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 0x7f52f4198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52f259c700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52e4004730 tx=0x7f52e4004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.759+0000 7f52e37fe700 1 -- 172.21.15.3:0/634333068 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401d070 con 0x7f52f4101390 2023-08-11T14:04:29.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.760+0000 7f52e37fe700 1 -- 172.21.15.3:0/634333068 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e4004c90 con 0x7f52f4101390 2023-08-11T14:04:29.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.760+0000 7f52e37fe700 1 -- 172.21.15.3:0/634333068 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4005570 con 0x7f52f4101390 2023-08-11T14:04:29.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.760+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e40095f0 con 0x7f52f4101390 2023-08-11T14:04:29.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.760+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f41a0b10 con 0x7f52f4101390 2023-08-11T14:04:29.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.764+0000 7f52e37fe700 1 -- 172.21.15.3:0/634333068 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f52e400fc70 con 0x7f52f4101390 2023-08-11T14:04:29.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.764+0000 7f530c345b80 1 Processor -- start 2023-08-11T14:04:29.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.764+0000 7f530c345b80 1 -- start start 2023-08-11T14:04:29.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635df0 0x55d723636270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72358d630 con 0x55d723635430 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72358af50 con 0x55d723634ad0 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f530c345b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d723586840 con 0x55d723635df0 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32790/0 (socket says 172.21.15.3:32790) 2023-08-11T14:04:29.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3337800903 learned_addr learned my addr 172.21.15.3:0/3337800903 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:29.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635df0 0x55d723636270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.765+0000 7f52f259c700 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3337800903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635df0 msgr2=0x55d723636270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635df0 0x55d723636270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3337800903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 msgr2=0x55d7236358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3337800903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d723406630 con 0x55d723634ad0 2023-08-11T14:04:29.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0098f0 tx=0x7f52ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c726de1074e1ff0a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52e37fe700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 0x7f52dc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52e0ff9700 1 -- 172.21.15.3:0/3337800903 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec019070 con 0x55d723634ad0 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52e37fe700 1 -- 172.21.15.3:0/634333068 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f52e40ccaa0 con 0x7f52f4101390 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52e0ff9700 1 -- 172.21.15.3:0/3337800903 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52ec00f040 con 0x55d723634ad0 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.766+0000 7f52e0ff9700 1 -- 172.21.15.3:0/3337800903 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec007a60 con 0x55d723634ad0 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.767+0000 7f52f259c700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 0x7f52dc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.767+0000 7f530c345b80 1 -- 172.21.15.3:0/3337800903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 msgr2=0x55d723634ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.767+0000 7f52f259c700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 0x7f52dc0aaca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52e40a1a50 tx=0x7f52e4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.767+0000 7f530c345b80 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0098f0 tx=0x7f52ec009c00 comp rx=0 tx=0).stop 2023-08-11T14:04:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 -- 172.21.15.3:0/3337800903 shutdown_connections 2023-08-11T14:04:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635df0 0x55d723636270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d723634ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 --2- 172.21.15.3:0/3337800903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 -- 172.21.15.3:0/3337800903 >> 172.21.15.3:0/3337800903 conn(0x55d723561950 msgr2=0x55d72362f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.768+0000 7f530c345b80 1 -- 172.21.15.3:0/3337800903 shutdown_connections 2023-08-11T14:04:29.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.769+0000 7f530c345b80 1 -- 172.21.15.3:0/3337800903 wait complete. 2023-08-11T14:04:29.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.769+0000 7f530c345b80 1 -- start start 2023-08-11T14:04:29.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.770+0000 7f530c345b80 1 Processor -- start 2023-08-11T14:04:29.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.770+0000 7f530c345b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72363e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635430 0x55d72363e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d723524ac0 con 0x55d723635df0 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7235247c0 con 0x55d723635430 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d723408920 con 0x55d723634ad0 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f530c345b80 10 client.0 _ll_register_callbacks cb 0x55d723527ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56006/0 (socket says 172.21.15.3:56006) 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3531542643 learned_addr learned my addr 172.21.15.3:0/3531542643 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.771+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72363e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3531542643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 msgr2=0x55d72363e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72363e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635430 msgr2=0x55d72363e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635430 0x55d72363e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 -- 172.21.15.3:0/3531542643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec003c50 con 0x55d723635df0 2023-08-11T14:04:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f52ec007ed0 tx=0x7f52ec007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a30ca53e9cb83f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:04:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec019040 con 0x55d723635df0 2023-08-11T14:04:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f52ec015680 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.772+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d723627a90 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52ec0048b0 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 10 client.15636 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7233b2de0 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d723627a90 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52ec020420 con 0x55d723635df0 2023-08-11T14:04:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 20 client.15636 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d723649e40 0x55d72364c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 4 client.15636 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.773+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7236034e0 con 0x55d723649e40 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.774+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d723649e40 0x55d72364c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.774+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d723649e40 0x55d72364c260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370c89c5d420e100 server_cookie=e89afe128cd4a9a in_seq=0 out_seq=0 2023-08-11T14:04:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.774+0000 7f52d9ffb700 10 client.15636 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f52d9ffb700 1 -- 172.21.15.3:0/3531542643 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d7236034e0 con 0x55d723649e40 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f52d9ffb700 10 client.15636 handle_command_reply: tid=0 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 client.15636 shutdown 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 msgr2=0x55d7236456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f52ec007ed0 tx=0x7f52ec007500 comp rx=0 tx=0).stop 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 shutdown_connections 2023-08-11T14:04:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d7236456e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72363e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d723649e40 0x55d72364c260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 --2- 172.21.15.3:0/3531542643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635430 0x55d72363e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f52d9ffb700 0 client.15636 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 >> 172.21.15.3:0/3531542643 conn(0x55d723561950 msgr2=0x55d723629cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 shutdown_connections 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.775+0000 7f530c345b80 1 -- 172.21.15.3:0/3531542643 wait complete. 2023-08-11T14:04:29.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.776+0000 7f530c345b80 20 client.15636 trim_cache size 0 max 16384 2023-08-11T14:04:29.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:29 smithi003 ceph-mon[97195]: pgmap v364: 129 pgs: 129 active+clean; 499 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 76 MiB/s wr, 196 op/s 2023-08-11T14:04:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:29 smithi187 ceph-mon[130973]: pgmap v364: 129 pgs: 129 active+clean; 499 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 76 MiB/s wr, 196 op/s 2023-08-11T14:04:29.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.798+0000 7f530c345b80 1 Processor -- start 2023-08-11T14:04:29.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.798+0000 7f530c345b80 1 -- start start 2023-08-11T14:04:29.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.798+0000 7f530c345b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72362fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.798+0000 7f530c345b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.798+0000 7f530c345b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 0x55d72363c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f530c345b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72362be20 con 0x55d723635430 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f530c345b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72363f2e0 con 0x55d723635df0 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f530c345b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d72363ede0 con 0x55d723634ad0 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 0x55d72363c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56014/0 (socket says 172.21.15.3:56014) 2023-08-11T14:04:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2883422525 learned_addr learned my addr 172.21.15.3:0/2883422525 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72362fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2883422525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 msgr2=0x55d72362fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72362fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2883422525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 msgr2=0x55d72363c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 0x55d72363c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2883422525 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7236364a0 con 0x55d723635430 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.799+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 0x55d72363c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:29.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.800+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f52e80097c0 tx=0x7f52e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d6ca47a19a0f71 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.800+0000 7f52e17fa700 1 -- 172.21.15.3:0/2883422525 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8018040 con 0x55d723635430 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.800+0000 7f52e17fa700 1 -- 172.21.15.3:0/2883422525 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e801e070 con 0x55d723635430 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.800+0000 7f52e17fa700 1 -- 172.21.15.3:0/2883422525 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8005020 con 0x55d723635430 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 -- 172.21.15.3:0/2883422525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 msgr2=0x55d7236456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f52e80097c0 tx=0x7f52e8009750 comp rx=0 tx=0).stop 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 -- 172.21.15.3:0/2883422525 shutdown_connections 2023-08-11T14:04:29.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723635df0 0x55d72363c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723634ad0 0x55d72362fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 --2- 172.21.15.3:0/2883422525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635430 0x55d7236456a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 -- 172.21.15.3:0/2883422525 >> 172.21.15.3:0/2883422525 conn(0x55d723561950 msgr2=0x55d72362f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.801+0000 7f530c345b80 1 -- 172.21.15.3:0/2883422525 shutdown_connections 2023-08-11T14:04:29.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.802+0000 7f530c345b80 1 -- 172.21.15.3:0/2883422525 wait complete. 2023-08-11T14:04:29.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.802+0000 7f530c345b80 1 -- start start 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.802+0000 7f530c345b80 1 Processor -- start 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d7236416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d723644c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d723645190 con 0x55d723635df0 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d723645300 con 0x55d723634ad0 2023-08-11T14:04:29.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7236454c0 con 0x55d723635430 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f530c345b80 10 client.0 _ll_register_callbacks cb 0x7f52ec004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f52f1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f52f1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40422/0 (socket says 172.21.15.3:40422) 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.803+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2012956888 learned_addr learned my addr 172.21.15.3:0/2012956888 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d7236416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 msgr2=0x55d7236416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d7236416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2012956888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 msgr2=0x55d723644c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d723644c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 -- 172.21.15.3:0/2012956888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7233b2de0 con 0x55d723635430 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52f1d9b700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52e8004a10 tx=0x7f52e8004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e704830c1685f3a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.804+0000 7f52d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.805+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e801a570 con 0x55d723635430 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.805+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f52e8004fa0 con 0x55d723635430 2023-08-11T14:04:29.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.805+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52ec00f040 con 0x55d723635430 2023-08-11T14:04:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.807+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e8005160 con 0x55d723635430 2023-08-11T14:04:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.807+0000 7f530c345b80 10 client.25669 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.807+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f52e8009c40 con 0x55d723635430 2023-08-11T14:04:29.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52ec00f040 con 0x55d723635430 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f52e8023b80 con 0x55d723635430 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 10 client.25669 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 10 client.25669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 20 client.25669 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d72369ebd0 0x55d7236a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 4 client.25669 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2023-08-11T14:04:29.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.808+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d72357f820 con 0x55d72369ebd0 2023-08-11T14:04:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.810+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d72369ebd0 0x55d7236a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:29.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.810+0000 7f52f2d9d700 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d72369ebd0 0x55d7236a0ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82854f3ff24c855 server_cookie=2a103888759edf67 in_seq=0 out_seq=0 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.810+0000 7f52d9ffb700 10 client.25669 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f52d9ffb700 1 -- 172.21.15.3:0/2012956888 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d72357f820 con 0x55d72369ebd0 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f52d9ffb700 10 client.25669 handle_command_reply: tid=0 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 client.25669 shutdown 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 msgr2=0x55d723641c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52e8004a10 tx=0x7f52e8004cd0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 shutdown_connections 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d723635df0 0x55d723644c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d72369ebd0 0x55d7236a0ff0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d723634ad0 0x55d7236416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 --2- 172.21.15.3:0/2012956888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d723635430 0x55d723641c30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f52d9ffb700 0 client.25669 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:29.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 >> 172.21.15.3:0/2012956888 conn(0x55d723561950 msgr2=0x55d7236338d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 shutdown_connections 2023-08-11T14:04:29.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.811+0000 7f530c345b80 1 -- 172.21.15.3:0/2012956888 wait complete. 2023-08-11T14:04:29.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f530c345b80 20 client.25669 trim_cache size 0 max 16384 2023-08-11T14:04:29.815 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:04:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2023-08-11T14:04:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2023-08-11T14:04:29.816 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:04:29.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 msgr2=0x7f52dc0aaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 0x7f52dc0aaca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52e40a1a50 tx=0x7f52e4009310 comp rx=0 tx=0).stop 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 msgr2=0x7f52f4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52e4004730 tx=0x7f52e4004760 comp rx=0 tx=0).stop 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 shutdown_connections 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.812+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f52dc0a87e0 0x7f52dc0aaca0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52f41065f0 0x7f52f419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52f4101390 0x7f52f4198860 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 --2- 172.21.15.3:0/634333068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52f4103cc0 0x7f52f4198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 >> 172.21.15.3:0/634333068 conn(0x7f52f40fad80 msgr2=0x7f52f40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 shutdown_connections 2023-08-11T14:04:29.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:29.813+0000 7f52f89ff700 1 -- 172.21.15.3:0/634333068 wait complete. 2023-08-11T14:04:29.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-08-11T14:04:29.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:04:29.909 INFO:teuthology.orchestra.run.smithi003.stderr:6000000 of 50000000 tuples (12%) done (elapsed 15.41 s, remaining 113.00 s) 2023-08-11T14:04:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 -- 172.21.15.3:0/999470560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10cc60 msgr2=0x7f77cc10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 --2- 172.21.15.3:0/999470560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10cc60 0x7f77cc10d0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b0a0 tx=0x7f77c400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 -- 172.21.15.3:0/999470560 shutdown_connections 2023-08-11T14:04:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 --2- 172.21.15.3:0/999470560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10cc60 0x7f77cc10d0e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 --2- 172.21.15.3:0/999470560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc107920 0x7f77cc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 --2- 172.21.15.3:0/999470560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc10a290 0x7f77cc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 -- 172.21.15.3:0/999470560 >> 172.21.15.3:0/999470560 conn(0x7f77cc06d720 msgr2=0x7f77cc06fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:30.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 -- 172.21.15.3:0/999470560 shutdown_connections 2023-08-11T14:04:30.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.160+0000 7f77d2bff700 1 -- 172.21.15.3:0/999470560 wait complete. 2023-08-11T14:04:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 Processor -- start 2023-08-11T14:04:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 -- start start 2023-08-11T14:04:30.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 0x7f77cc1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 0x7f77cc1a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc10cc60 0x7f77cc1ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1acef0 con 0x7f77cc107920 2023-08-11T14:04:30.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1ad030 con 0x7f77cc10a290 2023-08-11T14:04:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.162+0000 7f77d2bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1ad1c0 con 0x7f77cc10cc60 2023-08-11T14:04:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77c3fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 0x7f77cc1a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 0x7f77cc1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77c3fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 0x7f77cc1a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32844/0 (socket says 172.21.15.3:32844) 2023-08-11T14:04:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77c3fff700 1 -- 172.21.15.3:0/2580398633 learned_addr learned my addr 172.21.15.3:0/2580398633 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:30.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d119c700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc10cc60 0x7f77cc1ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 -- 172.21.15.3:0/2580398633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc10cc60 msgr2=0x7f77cc1ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc10cc60 0x7f77cc1ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 -- 172.21.15.3:0/2580398633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 msgr2=0x7f77cc1a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 0x7f77cc1a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.163+0000 7f77d099b700 1 -- 172.21.15.3:0/2580398633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009d00 con 0x7f77cc107920 2023-08-11T14:04:30.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.164+0000 7f77d099b700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 0x7f77cc1a5370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00d950 tx=0x7f77bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:30.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.164+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc009930 con 0x7f77cc107920 2023-08-11T14:04:30.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.164+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77bc010460 con 0x7f77cc107920 2023-08-11T14:04:30.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.164+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00b5d0 con 0x7f77cc107920 2023-08-11T14:04:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.164+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc1ad4a0 con 0x7f77cc107920 2023-08-11T14:04:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.165+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc1ad960 con 0x7f77cc107920 2023-08-11T14:04:30.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.169+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc06f2a0 con 0x7f77cc107920 2023-08-11T14:04:30.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.172+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f77bc010820 con 0x7f77cc107920 2023-08-11T14:04:30.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.174+0000 7f77c1ffb700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 0x7f77b40a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.175+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f77bc0cace0 con 0x7f77cc107920 2023-08-11T14:04:30.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.177+0000 7f77c3fff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 0x7f77b40a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.178+0000 7f77c3fff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 0x7f77b40a2590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009d20 tx=0x7f77c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:30.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.179+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f77bc0946f0 con 0x7f77cc107920 2023-08-11T14:04:30.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.584+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77cc11aff0 con 0x7f77cc107920 2023-08-11T14:04:30.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.585+0000 7f77c1ffb700 1 -- 172.21.15.3:0/2580398633 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f77bc017090 con 0x7f77cc107920 2023-08-11T14:04:30.587 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:04:30.587 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:02:01.312375+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 msgr2=0x7f77b40a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 0x7f77b40a2590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009d20 tx=0x7f77c8009260 comp rx=0 tx=0).stop 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 msgr2=0x7f77cc1a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 0x7f77cc1a5370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00d950 tx=0x7f77bc00dc60 comp rx=0 tx=0).stop 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 shutdown_connections 2023-08-11T14:04:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77cc10cc60 0x7f77cc1ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77cc107920 0x7f77cc1a5370 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f77b40a00d0 0x7f77b40a2590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 --2- 172.21.15.3:0/2580398633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77cc10a290 0x7f77cc1a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 >> 172.21.15.3:0/2580398633 conn(0x7f77cc06d720 msgr2=0x7f77cc10afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 shutdown_connections 2023-08-11T14:04:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.588+0000 7f77d2bff700 1 -- 172.21.15.3:0/2580398633 wait complete. 2023-08-11T14:04:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2023-08-11T14:04:30.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-08-11T14:04:30.828 INFO:teuthology.orchestra.run.smithi003.stderr:6100000 of 50000000 tuples (12%) done (elapsed 16.32 s, remaining 117.48 s) 2023-08-11T14:04:30.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.976+0000 7f99641ff700 1 -- 172.21.15.3:0/4110203847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c103cc0 msgr2=0x7f995c1060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.976+0000 7f99641ff700 1 --2- 172.21.15.3:0/4110203847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c103cc0 0x7f995c1060b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99480098f0 tx=0x7f9948009c00 comp rx=0 tx=0).stop 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 -- 172.21.15.3:0/4110203847 shutdown_connections 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 --2- 172.21.15.3:0/4110203847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 --2- 172.21.15.3:0/4110203847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c101390 0x7f995c103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 --2- 172.21.15.3:0/4110203847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c103cc0 0x7f995c1060b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 -- 172.21.15.3:0/4110203847 >> 172.21.15.3:0/4110203847 conn(0x7f995c0fad80 msgr2=0x7f995c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:30.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 -- 172.21.15.3:0/4110203847 shutdown_connections 2023-08-11T14:04:30.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.977+0000 7f99641ff700 1 -- 172.21.15.3:0/4110203847 wait complete. 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.978+0000 7f99641ff700 1 Processor -- start 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 -- start start 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c101390 0x7f995c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c1a03e0 con 0x7f995c1065f0 2023-08-11T14:04:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c1a0520 con 0x7f995c103cc0 2023-08-11T14:04:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f99641ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c1a06b0 con 0x7f995c101390 2023-08-11T14:04:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f996179a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f996179a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32852/0 (socket says 172.21.15.3:32852) 2023-08-11T14:04:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.979+0000 7f996179a700 1 -- 172.21.15.3:0/21506846 learned_addr learned my addr 172.21.15.3:0/21506846 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f9961f9b700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c101390 0x7f995c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996279c700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 -- 172.21.15.3:0/21506846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c101390 msgr2=0x7f995c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c101390 0x7f995c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 -- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 msgr2=0x7f995c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 -- 172.21.15.3:0/21506846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950009d00 con 0x7f995c103cc0 2023-08-11T14:04:30.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996279c700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.980+0000 7f996179a700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fd0 tx=0x7f994800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.981+0000 7f995affd700 1 -- 172.21.15.3:0/21506846 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994801c070 con 0x7f995c103cc0 2023-08-11T14:04:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.981+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9948009520 con 0x7f995c103cc0 2023-08-11T14:04:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.981+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c1a0c20 con 0x7f995c103cc0 2023-08-11T14:04:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.982+0000 7f995affd700 1 -- 172.21.15.3:0/21506846 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f994800f820 con 0x7f995c103cc0 2023-08-11T14:04:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.982+0000 7f995affd700 1 -- 172.21.15.3:0/21506846 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948004bb0 con 0x7f995c103cc0 2023-08-11T14:04:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.984+0000 7f995affd700 1 -- 172.21.15.3:0/21506846 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9948020070 con 0x7f995c103cc0 2023-08-11T14:04:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.986+0000 7f995affd700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 0x7f994c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.986+0000 7f995affd700 1 -- 172.21.15.3:0/21506846 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f99480cd290 con 0x7f995c103cc0 2023-08-11T14:04:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.986+0000 7f9961f9b700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 0x7f994c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.987+0000 7f9961f9b700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 0x7f994c0a2490 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9950009cd0 tx=0x7f995000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.988+0000 7f9977bc4b80 1 Processor -- start 2023-08-11T14:04:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.988+0000 7f9977bc4b80 1 -- start start 2023-08-11T14:04:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776c5e630 con 0x556776d05ad0 2023-08-11T14:04:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776c5bf50 con 0x556776d06df0 2023-08-11T14:04:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.989+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776c57840 con 0x556776d06430 2023-08-11T14:04:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f9961f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40482/0 (socket says 172.21.15.3:40482) 2023-08-11T14:04:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f9961f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56050/0 (socket says 172.21.15.3:56050) 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f996179a700 1 -- 172.21.15.3:0/1613606035 learned_addr learned my addr 172.21.15.3:0/1613606035 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.990+0000 7f996279c700 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 -- 172.21.15.3:0/1613606035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 msgr2=0x556776d068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 -- 172.21.15.3:0/1613606035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 msgr2=0x556776d07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 -- 172.21.15.3:0/1613606035 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556776ad7630 con 0x556776d05ad0 2023-08-11T14:04:30.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99500198a0 tx=0x7f9950009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448f7e813f3a71c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9947fff700 1 -- 172.21.15.3:0/1613606035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000e030 con 0x556776d05ad0 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9947fff700 1 -- 172.21.15.3:0/1613606035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995000f040 con 0x556776d05ad0 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.991+0000 7f9947fff700 1 -- 172.21.15.3:0/1613606035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995001e070 con 0x556776d05ad0 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.992+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1613606035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 msgr2=0x556776d05ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.992+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99500198a0 tx=0x7f9950009050 comp rx=0 tx=0).stop 2023-08-11T14:04:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1613606035 shutdown_connections 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d05ef0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1613606035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1613606035 >> 172.21.15.3:0/1613606035 conn(0x556776c32950 msgr2=0x556776d00420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1613606035 shutdown_connections 2023-08-11T14:04:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.993+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1613606035 wait complete. 2023-08-11T14:04:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.994+0000 7f9977bc4b80 1 -- start start 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.994+0000 7f9977bc4b80 1 Processor -- start 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776bf5ac0 con 0x556776d05ad0 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776bf57c0 con 0x556776d06df0 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776ad9920 con 0x556776d06430 2023-08-11T14:04:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9977bc4b80 10 client.0 _ll_register_callbacks cb 0x556776bf8ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40498/0 (socket says 172.21.15.3:40498) 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f996179a700 1 -- 172.21.15.3:0/1466275653 learned_addr learned my addr 172.21.15.3:0/1466275653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.995+0000 7f996279c700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 -- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 msgr2=0x556776d166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d166a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 -- 172.21.15.3:0/1466275653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 msgr2=0x556776d0f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d0f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 -- 172.21.15.3:0/1466275653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9950019580 con 0x556776d06430 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f996179a700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99480caac0 tx=0x7f994809b220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca3144e1a8b2af1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.996+0000 7f99457fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.997+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480a27d0 con 0x556776d06430 2023-08-11T14:04:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.997+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f994809acb0 con 0x556776d06430 2023-08-11T14:04:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.997+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556776cf8a90 con 0x556776d06430 2023-08-11T14:04:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994809ae70 con 0x556776d06430 2023-08-11T14:04:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 10 client.25687 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556776a83de0 con 0x556776d06430 2023-08-11T14:04:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556776cf8a90 con 0x556776d06430 2023-08-11T14:04:31.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f99480c4bb0 con 0x556776d06430 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 10 client.25687 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 10 client.25687 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 20 client.25687 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d1cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 4 client.25687 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.998+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556776cd44e0 con 0x556776d1ab40 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.999+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d1cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.999+0000 7f9961f9b700 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d1cf60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a80db99d9a10e6b server_cookie=c912c7343b608fc5 in_seq=0 out_seq=0 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:30.999+0000 7f99457fa700 10 client.25687 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f99457fa700 1 -- 172.21.15.3:0/1466275653 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556776cd44e0 con 0x556776d1ab40 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f99457fa700 10 client.25687 handle_command_reply: tid=0 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 client.25687 shutdown 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 msgr2=0x556776d0f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99480caac0 tx=0x7f994809b220 comp rx=0 tx=0).stop 2023-08-11T14:04:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 shutdown_connections 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d1cf60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f99457fa700 0 client.25687 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d0f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/1466275653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d0f660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.000+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 >> 172.21.15.3:0/1466275653 conn(0x556776c32950 msgr2=0x556776cfacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.001+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 shutdown_connections 2023-08-11T14:04:31.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.001+0000 7f9977bc4b80 1 -- 172.21.15.3:0/1466275653 wait complete. 2023-08-11T14:04:31.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.001+0000 7f9977bc4b80 20 client.25687 trim_cache size 0 max 16384 2023-08-11T14:04:31.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.015+0000 7f9977bc4b80 1 Processor -- start 2023-08-11T14:04:31.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.015+0000 7f9977bc4b80 1 -- start start 2023-08-11T14:04:31.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.015+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.015+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06430 0x556776d16660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776cfcb40 con 0x556776d05ad0 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776d0e6a0 con 0x556776d06430 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776d01ae0 con 0x556776d06df0 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9961f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40514/0 (socket says 172.21.15.3:40514) 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 -- 172.21.15.3:0/719906833 learned_addr learned my addr 172.21.15.3:0/719906833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f9961f9b700 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d218c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56064/0 (socket says 172.21.15.3:56064) 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996179a700 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06430 0x556776d16660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 -- 172.21.15.3:0/719906833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06430 msgr2=0x556776d16660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06430 0x556776d16660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 -- 172.21.15.3:0/719906833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 msgr2=0x556776d218c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d218c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.016+0000 7f996279c700 1 -- 172.21.15.3:0/719906833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556776a6cc60 con 0x556776d06df0 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.017+0000 7f996279c700 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f995400be80 tx=0x7f9954012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c417cad8dd1f3a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.017+0000 7f9958ff9700 1 -- 172.21.15.3:0/719906833 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954017070 con 0x556776d06df0 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.017+0000 7f9958ff9700 1 -- 172.21.15.3:0/719906833 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9954012b90 con 0x556776d06df0 2023-08-11T14:04:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.017+0000 7f9958ff9700 1 -- 172.21.15.3:0/719906833 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954010630 con 0x556776d06df0 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- 172.21.15.3:0/719906833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 msgr2=0x556776d09cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f995400be80 tx=0x7f9954012730 comp rx=0 tx=0).stop 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- 172.21.15.3:0/719906833 shutdown_connections 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06df0 0x556776d09cb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/719906833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06430 0x556776d16660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- 172.21.15.3:0/719906833 >> 172.21.15.3:0/719906833 conn(0x556776c32950 msgr2=0x556776d0ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- 172.21.15.3:0/719906833 shutdown_connections 2023-08-11T14:04:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- 172.21.15.3:0/719906833 wait complete. 2023-08-11T14:04:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.018+0000 7f9977bc4b80 1 -- start start 2023-08-11T14:04:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 Processor -- start 2023-08-11T14:04:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d12440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776d15ee0 con 0x556776d05ad0 2023-08-11T14:04:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776d16050 con 0x556776d06df0 2023-08-11T14:04:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556776d16210 con 0x556776d06430 2023-08-11T14:04:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9977bc4b80 10 client.0 _ll_register_callbacks cb 0x556776bf8ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.019+0000 7f9961f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d12440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40520/0 (socket says 172.21.15.3:40520) 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 -- 172.21.15.3:0/4269541822 learned_addr learned my addr 172.21.15.3:0/4269541822 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996279c700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 -- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 msgr2=0x556776d159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 -- 172.21.15.3:0/4269541822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 msgr2=0x556776d12440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d12440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 -- 172.21.15.3:0/4269541822 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995400bb30 con 0x556776d06430 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996279c700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f996179a700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99480cab50 tx=0x7f9948099000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b734e98408ccd545 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f99457fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99480a2520 con 0x556776d06430 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f995000f040 con 0x556776d06430 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.020+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f99480a27d0 con 0x556776d06430 2023-08-11T14:04:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99480a2340 con 0x556776d06430 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 10 client.25693 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556776aaa250 con 0x556776d06430 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f994807d080 con 0x556776d06430 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f99480999b0 con 0x556776d06430 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 10 client.25693 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 20 client.25693 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d71210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 4 client.25693 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556776c50820 con 0x556776d1ab40 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.021+0000 7f9961f9b700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d71210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9961f9b700 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d71210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3d006d92da66f7b server_cookie=d5ec5287c48edf3c in_seq=0 out_seq=0 2023-08-11T14:04:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f99457fa700 10 client.25693 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f99457fa700 1 -- 172.21.15.3:0/4269541822 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556776c50820 con 0x556776d1ab40 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f99457fa700 10 client.25693 handle_command_reply: tid=0 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 client.25693 shutdown 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 msgr2=0x556776d12980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99480cab50 tx=0x7f9948099000 comp rx=0 tx=0).stop 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 shutdown_connections 2023-08-11T14:04:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x556776d1ab40 0x556776d71210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556776d06df0 0x556776d159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556776d05ad0 0x556776d12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 --2- 172.21.15.3:0/4269541822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556776d06430 0x556776d12980 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 >> 172.21.15.3:0/4269541822 conn(0x556776c32950 msgr2=0x556776d0f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f99457fa700 0 client.25693 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 shutdown_connections 2023-08-11T14:04:31.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.022+0000 7f9977bc4b80 1 -- 172.21.15.3:0/4269541822 wait complete. 2023-08-11T14:04:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f9977bc4b80 20 client.25693 trim_cache size 0 max 16384 2023-08-11T14:04:31.036 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:04:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 msgr2=0x7f994c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 0x7f994c0a2490 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9950009cd0 tx=0x7f995000b040 comp rx=0 tx=0).stop 2023-08-11T14:04:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 msgr2=0x7f995c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fd0 tx=0x7f994800b790 comp rx=0 tx=0).stop 2023-08-11T14:04:31.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 shutdown_connections 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f995c1065f0 0x7f995c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f995c101390 0x7f995c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f994c09ffd0 0x7f994c0a2490 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 --2- 172.21.15.3:0/21506846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f995c103cc0 0x7f995c198da0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.023+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 >> 172.21.15.3:0/21506846 conn(0x7f995c0fad80 msgr2=0x7f995c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.024+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 shutdown_connections 2023-08-11T14:04:31.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.024+0000 7f99641ff700 1 -- 172.21.15.3:0/21506846 wait complete. 2023-08-11T14:04:31.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-08-11T14:04:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:6200000 of 50000000 tuples (12%) done (elapsed 16.68 s, remaining 117.86 s) 2023-08-11T14:04:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.365+0000 7f52335ff700 1 -- 172.21.15.3:0/2371454839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 msgr2=0x7f522c100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.365+0000 7f52335ff700 1 --2- 172.21.15.3:0/2371454839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c100980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5224009f90 tx=0x7f522400b330 comp rx=0 tx=0).stop 2023-08-11T14:04:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.365+0000 7f52335ff700 1 -- 172.21.15.3:0/2371454839 shutdown_connections 2023-08-11T14:04:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.365+0000 7f52335ff700 1 --2- 172.21.15.3:0/2371454839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c100980 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.365+0000 7f52335ff700 1 --2- 172.21.15.3:0/2371454839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ff1e0 0x7f522c0ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.366+0000 7f52335ff700 1 --2- 172.21.15.3:0/2371454839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ffb40 0x7f522c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.366+0000 7f52335ff700 1 -- 172.21.15.3:0/2371454839 >> 172.21.15.3:0/2371454839 conn(0x7f522c0fad80 msgr2=0x7f522c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.366+0000 7f52335ff700 1 -- 172.21.15.3:0/2371454839 shutdown_connections 2023-08-11T14:04:31.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.366+0000 7f52335ff700 1 -- 172.21.15.3:0/2371454839 wait complete. 2023-08-11T14:04:31.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.367+0000 7f52335ff700 1 Processor -- start 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 -- start start 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 0x7f522c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1a04f0 con 0x7f522c0ffb40 2023-08-11T14:04:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1a0630 con 0x7f522c0ff1e0 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f52335ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1a07c0 con 0x7f522c100500 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32896/0 (socket says 172.21.15.3:32896) 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f523139b700 1 -- 172.21.15.3:0/1393122188 learned_addr learned my addr 172.21.15.3:0/1393122188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.368+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f5230b9a700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 0x7f522c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 -- 172.21.15.3:0/1393122188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 msgr2=0x7f522c19ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c19ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 -- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 msgr2=0x7f522c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 0x7f522c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 -- 172.21.15.3:0/1393122188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280095f0 con 0x7f522c0ff1e0 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f5230b9a700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 0x7f522c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f523139b700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f522800ec10 tx=0x7f522800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f52227fc700 1 -- 172.21.15.3:0/1393122188 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800cd40 con 0x7f522c0ff1e0 2023-08-11T14:04:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f52227fc700 1 -- 172.21.15.3:0/1393122188 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52280044e0 con 0x7f522c0ff1e0 2023-08-11T14:04:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c009520 con 0x7f522c0ff1e0 2023-08-11T14:04:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.369+0000 7f52227fc700 1 -- 172.21.15.3:0/1393122188 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228010690 con 0x7f522c0ff1e0 2023-08-11T14:04:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.370+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5224009c40 con 0x7f522c0ff1e0 2023-08-11T14:04:31.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.373+0000 7f52227fc700 1 -- 172.21.15.3:0/1393122188 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f5228003710 con 0x7f522c0ff1e0 2023-08-11T14:04:31.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.374+0000 7f52227fc700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 0x7f52180a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.375+0000 7f52227fc700 1 -- 172.21.15.3:0/1393122188 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5228014070 con 0x7f522c0ff1e0 2023-08-11T14:04:31.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.375+0000 7f5230b9a700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 0x7f52180a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.376+0000 7f5230b9a700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 0x7f52180a2410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d70 tx=0x7f521c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.376+0000 7f5246fb9b80 1 Processor -- start 2023-08-11T14:04:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 -- start start 2023-08-11T14:04:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d2368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236df0 0x559d1d237270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d18e630 con 0x559d1d236430 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d18bf50 con 0x559d1d236df0 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d187840 con 0x559d1d235ad0 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f523139b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f523139b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40548/0 (socket says 172.21.15.3:40548) 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.377+0000 7f523139b700 1 -- 172.21.15.3:0/838903665 learned_addr learned my addr 172.21.15.3:0/838903665 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 -- 172.21.15.3:0/838903665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236df0 msgr2=0x559d1d237270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236df0 0x559d1d237270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 -- 172.21.15.3:0/838903665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 msgr2=0x559d1d2368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d2368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 -- 172.21.15.3:0/838903665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d1d007630 con 0x559d1d235ad0 2023-08-11T14:04:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f523139b700 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5228095740 tx=0x7f522800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba72be261c7b7712 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f52177fe700 1 -- 172.21.15.3:0/838903665 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228099240 con 0x559d1d235ad0 2023-08-11T14:04:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f52177fe700 1 -- 172.21.15.3:0/838903665 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522809d070 con 0x559d1d235ad0 2023-08-11T14:04:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.378+0000 7f52177fe700 1 -- 172.21.15.3:0/838903665 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280994a0 con 0x559d1d235ad0 2023-08-11T14:04:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 -- 172.21.15.3:0/838903665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 msgr2=0x559d1d235ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5228095740 tx=0x7f522800ef90 comp rx=0 tx=0).stop 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 -- 172.21.15.3:0/838903665 shutdown_connections 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236df0 0x559d1d237270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d235ad0 0x559d1d235ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/838903665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d2368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 -- 172.21.15.3:0/838903665 >> 172.21.15.3:0/838903665 conn(0x559d1d162950 msgr2=0x559d1d230420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.379+0000 7f5246fb9b80 1 -- 172.21.15.3:0/838903665 shutdown_connections 2023-08-11T14:04:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.380+0000 7f5246fb9b80 1 -- 172.21.15.3:0/838903665 wait complete. 2023-08-11T14:04:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.380+0000 7f5246fb9b80 1 -- start start 2023-08-11T14:04:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 Processor -- start 2023-08-11T14:04:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d247a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d247f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d125ac0 con 0x559d1d236430 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d1257c0 con 0x559d1d235ad0 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.381+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d009920 con 0x559d1d236df0 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f5246fb9b80 10 client.0 _ll_register_callbacks cb 0x559d1d128ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d247a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d247a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32924/0 (socket says 172.21.15.3:32924) 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f523139b700 1 -- 172.21.15.3:0/1083286765 learned_addr learned my addr 172.21.15.3:0/1083286765 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f5230b9a700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d247f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f5231b9c700 1 -- 172.21.15.3:0/1083286765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 msgr2=0x559d1d247a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.382+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d247a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5231b9c700 1 -- 172.21.15.3:0/1083286765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 msgr2=0x559d1d247f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d247f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5231b9c700 1 -- 172.21.15.3:0/1083286765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5224009c40 con 0x559d1d236df0 2023-08-11T14:04:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24efb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5224007930 tx=0x7f5224007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f538f74709049d32 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5214ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:31.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400e040 con 0x559d1d236df0 2023-08-11T14:04:31.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.383+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f52240093c0 con 0x559d1d236df0 2023-08-11T14:04:31.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.384+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d1d228a90 con 0x559d1d236df0 2023-08-11T14:04:31.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.390+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5224003e10 con 0x559d1d236df0 2023-08-11T14:04:31.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.390+0000 7f5246fb9b80 10 client.25705 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.390+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d1cfb3de0 con 0x559d1d236df0 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d1d228a90 con 0x559d1d236df0 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f522402daa0 con 0x559d1d236df0 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 10 client.25705 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 10 client.25705 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 20 client.25705 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d253830 0x559d1d255c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 4 client.25705 mds_command: new command op to 24511 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:31.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d1d2044e0 con 0x559d1d253830 2023-08-11T14:04:31.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.391+0000 7f523139b700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d253830 0x559d1d255c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.392+0000 7f523139b700 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d253830 0x559d1d255c50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a27b72d7a3762ca6 server_cookie=13236dd3a9714eeb in_seq=0 out_seq=0 2023-08-11T14:04:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.392+0000 7f5214ff9700 10 client.25705 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:04:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5214ff9700 1 -- 172.21.15.3:0/1083286765 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559d1d2044e0 con 0x559d1d253830 2023-08-11T14:04:31.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5214ff9700 10 client.25705 handle_command_reply: tid=0 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 client.25705 shutdown 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 msgr2=0x559d1d24efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24efb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5224007930 tx=0x7f5224007d30 comp rx=0 tx=0).stop 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 shutdown_connections 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d253830 0x559d1d255c50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24efb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5214ff9700 0 client.25705 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d247a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1083286765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236430 0x559d1d247f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 >> 172.21.15.3:0/1083286765 conn(0x559d1d162950 msgr2=0x559d1d22acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 shutdown_connections 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.393+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1083286765 wait complete. 2023-08-11T14:04:31.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.394+0000 7f5246fb9b80 20 client.25705 trim_cache size 0 max 16384 2023-08-11T14:04:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.421+0000 7f5246fb9b80 1 Processor -- start 2023-08-11T14:04:31.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.421+0000 7f5246fb9b80 1 -- start start 2023-08-11T14:04:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 0x559d1d24ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 0x559d1d23d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d22cb10 con 0x559d1d236df0 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d23cff0 con 0x559d1d235ad0 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d23a900 con 0x559d1d236430 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5230b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 0x559d1d24ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5230b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 0x559d1d24ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40580/0 (socket says 172.21.15.3:40580) 2023-08-11T14:04:31.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5230b9a700 1 -- 172.21.15.3:0/1783752067 learned_addr learned my addr 172.21.15.3:0/1783752067 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:32928/0 (socket says 172.21.15.3:32928) 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 0x559d1d23d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 -- 172.21.15.3:0/1783752067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 msgr2=0x559d1d24ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 0x559d1d24ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 -- 172.21.15.3:0/1783752067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 msgr2=0x559d1d23d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 0x559d1d23d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f523139b700 1 -- 172.21.15.3:0/1783752067 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d1cf9cc60 con 0x559d1d235ad0 2023-08-11T14:04:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.422+0000 7f5231b9c700 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 0x559d1d23d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.423+0000 7f523139b700 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5228097620 tx=0x7f52280a11d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e9f26b44af24bb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.423+0000 7f5217fff700 1 -- 172.21.15.3:0/1783752067 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228099240 con 0x559d1d235ad0 2023-08-11T14:04:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.423+0000 7f5217fff700 1 -- 172.21.15.3:0/1783752067 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5228097650 con 0x559d1d235ad0 2023-08-11T14:04:31.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.423+0000 7f5217fff700 1 -- 172.21.15.3:0/1783752067 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280b1220 con 0x559d1d235ad0 2023-08-11T14:04:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1783752067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 msgr2=0x559d1d235eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5228097620 tx=0x7f52280a11d0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1783752067 shutdown_connections 2023-08-11T14:04:31.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d236df0 0x559d1d23d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d235ad0 0x559d1d235eb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/1783752067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236430 0x559d1d24ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1783752067 >> 172.21.15.3:0/1783752067 conn(0x559d1d162950 msgr2=0x559d1d246790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1783752067 shutdown_connections 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- 172.21.15.3:0/1783752067 wait complete. 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.424+0000 7f5246fb9b80 1 -- start start 2023-08-11T14:04:31.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.425+0000 7f5246fb9b80 1 Processor -- start 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.425+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.425+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236430 0x559d1d24b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.425+0000 7f5246fb9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d24ea70 con 0x559d1d235ad0 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d24ebe0 con 0x559d1d236430 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f5246fb9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1d24eda0 con 0x559d1d236df0 2023-08-11T14:04:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f5246fb9b80 10 client.0 _ll_register_callbacks cb 0x559d1d128ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56130/0 (socket says 172.21.15.3:56130) 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 -- 172.21.15.3:0/4235542175 learned_addr learned my addr 172.21.15.3:0/4235542175 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f5231b9c700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 -- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 msgr2=0x559d1d24e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 -- 172.21.15.3:0/4235542175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236430 msgr2=0x559d1d24b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236430 0x559d1d24b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.426+0000 7f523139b700 1 -- 172.21.15.3:0/4235542175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52280b5040 con 0x559d1d235ad0 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f523139b700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5228020040 tx=0x7f522800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20df9cbb6dd7a412 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5214ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228005d10 con 0x559d1d235ad0 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f522809d070 con 0x559d1d235ad0 2023-08-11T14:04:31.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5228097380 con 0x559d1d235ad0 2023-08-11T14:04:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5228099240 con 0x559d1d235ad0 2023-08-11T14:04:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5246fb9b80 10 client.15675 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.427+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559d1cfda250 con 0x559d1d235ad0 2023-08-11T14:04:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522809d070 con 0x559d1d235ad0 2023-08-11T14:04:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f522809c070 con 0x559d1d235ad0 2023-08-11T14:04:31.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 10 client.15675 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 20 client.15675 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d2a8180 0x559d1d2aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 4 client.15675 mds_command: new command op to 24511 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.428+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d1d180820 con 0x559d1d2a8180 2023-08-11T14:04:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.430+0000 7f5230b9a700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d2a8180 0x559d1d2aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.430+0000 7f5230b9a700 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d2a8180 0x559d1d2aa570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94f96485ae8a0517 server_cookie=97d4918d29efe2b0 in_seq=0 out_seq=0 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.430+0000 7f5214ff9700 10 client.15675 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5214ff9700 1 -- 172.21.15.3:0/4235542175 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d1d180820 con 0x559d1d2a8180 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5214ff9700 10 client.15675 handle_command_reply: tid=0 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 client.15675 shutdown 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 msgr2=0x559d1d24b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5228020040 tx=0x7f522800ef90 comp rx=0 tx=0).stop 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 shutdown_connections 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x559d1d2a8180 0x559d1d2aa570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559d1d236df0 0x559d1d24e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5214ff9700 0 client.15675 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d1d235ad0 0x559d1d24b1a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 --2- 172.21.15.3:0/4235542175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d1d236430 0x559d1d24b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 >> 172.21.15.3:0/4235542175 conn(0x559d1d162950 msgr2=0x559d1d23d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.431+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 shutdown_connections 2023-08-11T14:04:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.432+0000 7f5246fb9b80 1 -- 172.21.15.3:0/4235542175 wait complete. 2023-08-11T14:04:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.432+0000 7f5246fb9b80 20 client.15675 trim_cache size 0 max 16384 2023-08-11T14:04:31.439 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:04:31.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 msgr2=0x7f52180a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 0x7f52180a2410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d70 tx=0x7f521c009260 comp rx=0 tx=0).stop 2023-08-11T14:04:31.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 msgr2=0x7f522c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f522800ec10 tx=0x7f522800ef20 comp rx=0 tx=0).stop 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 shutdown_connections 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.433+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f522c100500 0x7f522c19ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.434+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f521809ff50 0x7f52180a2410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.434+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f522c0ff1e0 0x7f522c198970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.434+0000 7f52335ff700 1 --2- 172.21.15.3:0/1393122188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f522c0ffb40 0x7f522c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.434+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 >> 172.21.15.3:0/1393122188 conn(0x7f522c0fad80 msgr2=0x7f522c0fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.438+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 shutdown_connections 2023-08-11T14:04:31.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.438+0000 7f52335ff700 1 -- 172.21.15.3:0/1393122188 wait complete. 2023-08-11T14:04:31.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-08-11T14:04:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:31 smithi103 ceph-mon[130434]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 MiB/s wr, 96 op/s 2023-08-11T14:04:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:31 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2580398633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:04:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 -- 172.21.15.3:0/501489529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80073380 msgr2=0x7f8c80073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 --2- 172.21.15.3:0/501489529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80073380 0x7f8c80073800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009990 tx=0x7f8c74009ca0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 -- 172.21.15.3:0/501489529 shutdown_connections 2023-08-11T14:04:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 --2- 172.21.15.3:0/501489529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c8010cf00 0x7f8c8010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 --2- 172.21.15.3:0/501489529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80074960 0x7f8c80072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 --2- 172.21.15.3:0/501489529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80073380 0x7f8c80073800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 -- 172.21.15.3:0/501489529 >> 172.21.15.3:0/501489529 conn(0x7f8c800fbea0 msgr2=0x7f8c800fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 -- 172.21.15.3:0/501489529 shutdown_connections 2023-08-11T14:04:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.751+0000 7f8c869ff700 1 -- 172.21.15.3:0/501489529 wait complete. 2023-08-11T14:04:31.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 Processor -- start 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 -- start start 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80073380 0x7f8c8019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 0x7f8c801a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a47e0 con 0x7f8c8010cf00 2023-08-11T14:04:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.752+0000 7f8c869ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a4950 con 0x7f8c80074960 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c869ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801a4b10 con 0x7f8c80073380 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53392/0 (socket says 172.21.15.3:53392) 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/1344904188 learned_addr learned my addr 172.21.15.3:0/1344904188 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 0x7f8c801a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80073380 0x7f8c8019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/1344904188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80073380 msgr2=0x7f8c8019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80073380 0x7f8c8019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 msgr2=0x7f8c801a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 0x7f8c801a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/1344904188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740095f0 con 0x7f8c80074960 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.753+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 0x7f8c801a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005720 tx=0x7f8c74009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c7d7fa700 1 -- 172.21.15.3:0/1344904188 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401d070 con 0x7f8c80074960 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c7d7fa700 1 -- 172.21.15.3:0/1344904188 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c74021410 con 0x7f8c80074960 2023-08-11T14:04:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c801a4d90 con 0x7f8c80074960 2023-08-11T14:04:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c7d7fa700 1 -- 172.21.15.3:0/1344904188 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74017650 con 0x7f8c80074960 2023-08-11T14:04:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.754+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801a51b0 con 0x7f8c80074960 2023-08-11T14:04:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.757+0000 7f8c7d7fa700 1 -- 172.21.15.3:0/1344904188 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f8c74021580 con 0x7f8c80074960 2023-08-11T14:04:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.758+0000 7f8c9a223b80 1 Processor -- start 2023-08-11T14:04:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 -- start start 2023-08-11T14:04:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c7d7fa700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 0x7f8c680a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa044c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c7d7fa700 1 -- 172.21.15.3:0/1344904188 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8c740cc730 con 0x7f8c80074960 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa044d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa03a4630 con 0x559aa044cdf0 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa03a1f50 con 0x559aa044c430 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa039d840 con 0x559aa044bad0 2023-08-11T14:04:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 0x7f8c680a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42194/0 (socket says 172.21.15.3:42194) 2023-08-11T14:04:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c7ffff700 1 -- 172.21.15.3:0/1700511070 learned_addr learned my addr 172.21.15.3:0/1700511070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.759+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa044d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa044c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 -- 172.21.15.3:0/1700511070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 msgr2=0x559aa044c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa044c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 -- 172.21.15.3:0/1700511070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 msgr2=0x559aa044d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa044d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 -- 172.21.15.3:0/1700511070 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aa021d630 con 0x559aa044bad0 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa044c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009d20 tx=0x7f8c70011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe0b9a1f5f660d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 0x7f8c680a2410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009fd0 tx=0x7f8c78003cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c667fc700 1 -- 172.21.15.3:0/1700511070 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70019630 con 0x559aa044bad0 2023-08-11T14:04:31.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c667fc700 1 -- 172.21.15.3:0/1700511070 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70019c50 con 0x559aa044bad0 2023-08-11T14:04:31.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.760+0000 7f8c667fc700 1 -- 172.21.15.3:0/1700511070 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000eae0 con 0x559aa044bad0 2023-08-11T14:04:31.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 -- 172.21.15.3:0/1700511070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 msgr2=0x559aa044bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009d20 tx=0x7f8c70011720 comp rx=0 tx=0).stop 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 -- 172.21.15.3:0/1700511070 shutdown_connections 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa044d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044bad0 0x559aa044bef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/1700511070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa044c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 -- 172.21.15.3:0/1700511070 >> 172.21.15.3:0/1700511070 conn(0x559aa0378950 msgr2=0x559aa0446420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 -- 172.21.15.3:0/1700511070 shutdown_connections 2023-08-11T14:04:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.761+0000 7f8c9a223b80 1 -- 172.21.15.3:0/1700511070 wait complete. 2023-08-11T14:04:31.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 -- start start 2023-08-11T14:04:31.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 Processor -- start 2023-08-11T14:04:31.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa0455620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044cdf0 0x559aa045c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.762+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa033bac0 con 0x559aa044bad0 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa033b7c0 con 0x559aa044cdf0 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa021f920 con 0x559aa044c430 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c9a223b80 10 client.0 _ll_register_callbacks cb 0x559aa033eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34316/0 (socket says 172.21.15.3:34316) 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2979671104 learned_addr learned my addr 172.21.15.3:0/2979671104 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044cdf0 0x559aa045c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa0455620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2979671104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 msgr2=0x559aa0455620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa0455620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044cdf0 msgr2=0x559aa045c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044cdf0 0x559aa045c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.763+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2979671104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c70009980 con 0x559aa044bad0 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005920 tx=0x7f8c700056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd32a7f71c9b034e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70019630 con 0x559aa044bad0 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8c70005ce0 con 0x559aa044bad0 2023-08-11T14:04:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aa043ea90 con 0x559aa044bad0 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c70019790 con 0x559aa044bad0 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c9a223b80 10 client.15687 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.764+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559aa01c9de0 con 0x559aa044bad0 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559aa043ea90 con 0x559aa044bad0 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c700048f0 con 0x559aa044bad0 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 10 client.15687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-08-11T14:04:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 20 client.15687 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa0460df0 0x559aa0463210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 4 client.15687 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559aa041a4e0 con 0x559aa0460df0 2023-08-11T14:04:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.765+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa0460df0 0x559aa0463210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.766+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa0460df0 0x559aa0463210 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a62bf547073af32a server_cookie=a1c53595ad6fb0d7 in_seq=0 out_seq=0 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.766+0000 7f8c5f7fe700 10 client.15687 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/2979671104 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559aa041a4e0 con 0x559aa0460df0 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c5f7fe700 10 client.15687 handle_command_reply: tid=0 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 client.15687 shutdown 2023-08-11T14:04:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 msgr2=0x559aa04550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c70005920 tx=0x7f8c700056c0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 shutdown_connections 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa0460df0 0x559aa0463210 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044cdf0 0x559aa045c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa04550e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2979671104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa0455620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 >> 172.21.15.3:0/2979671104 conn(0x559aa0378950 msgr2=0x559aa0440cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c5f7fe700 0 client.15687 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:04:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 shutdown_connections 2023-08-11T14:04:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.767+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2979671104 wait complete. 2023-08-11T14:04:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.768+0000 7f8c9a223b80 20 client.15687 trim_cache size 0 max 16384 2023-08-11T14:04:31.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.782+0000 7f8c9a223b80 1 Processor -- start 2023-08-11T14:04:31.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.782+0000 7f8c9a223b80 1 -- start start 2023-08-11T14:04:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.782+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.782+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa045c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.782+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa04501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa0456230 con 0x559aa044cdf0 2023-08-11T14:04:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa0450840 con 0x559aa044bad0 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa0460af0 con 0x559aa044c430 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa045c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa045c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42204/0 (socket says 172.21.15.3:42204) 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/2745224869 learned_addr learned my addr 172.21.15.3:0/2745224869 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53420/0 (socket says 172.21.15.3:53420) 2023-08-11T14:04:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa04501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2745224869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 msgr2=0x559aa045c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa045c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2745224869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 msgr2=0x559aa04501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa04501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c7ffff700 1 -- 172.21.15.3:0/2745224869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aa01b2c60 con 0x559aa044bad0 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.783+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa04501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.784+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009c70 tx=0x7f8c70011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df19df8abb03079 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.784+0000 7f8c66ffd700 1 -- 172.21.15.3:0/2745224869 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70005ce0 con 0x559aa044bad0 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.784+0000 7f8c66ffd700 1 -- 172.21.15.3:0/2745224869 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70018f00 con 0x559aa044bad0 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.784+0000 7f8c66ffd700 1 -- 172.21.15.3:0/2745224869 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70019690 con 0x559aa044bad0 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2745224869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 msgr2=0x559aa044fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009c70 tx=0x7f8c70011720 comp rx=0 tx=0).stop 2023-08-11T14:04:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2745224869 shutdown_connections 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044cdf0 0x559aa04501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044bad0 0x559aa044fcb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/2745224869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044c430 0x559aa045c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2745224869 >> 172.21.15.3:0/2745224869 conn(0x559aa0378950 msgr2=0x559aa04463d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2745224869 shutdown_connections 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.785+0000 7f8c9a223b80 1 -- 172.21.15.3:0/2745224869 wait complete. 2023-08-11T14:04:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 -- start start 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 Processor -- start 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa045c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 0x559aa04591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa045c9f0 con 0x559aa044bad0 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa04596f0 con 0x559aa044c430 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aa04598b0 con 0x559aa044cdf0 2023-08-11T14:04:31.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.786+0000 7f8c9a223b80 10 client.0 _ll_register_callbacks cb 0x7f8c78004200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa045c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53428/0 (socket says 172.21.15.3:53428) 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/653872386 learned_addr learned my addr 172.21.15.3:0/653872386 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 0x559aa04591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/653872386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 msgr2=0x559aa04591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 0x559aa04591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/653872386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 msgr2=0x559aa045c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa045c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c7f7fe700 1 -- 172.21.15.3:0/653872386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559aa01c9de0 con 0x559aa044c430 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.787+0000 7f8c84f9c700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 0x559aa04591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c7f7fe700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c740c9e50 tx=0x7f8c74021c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ea5615a1ca9a3d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74099fb0 con 0x559aa044c430 2023-08-11T14:04:31.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8c7409dde0 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c70019c50 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c740a1d30 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 10 client.25721 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559aa01f0250 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70019c50 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8c74080080 con 0x559aa044c430 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 10 client.25721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-08-11T14:04:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 20 client.25721 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.788+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa04b6040 0x559aa04b8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c9a223b80 4 client.25721 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:04:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559aa0396820 con 0x559aa04b6040 2023-08-11T14:04:31.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:31 smithi187 ceph-mon[130973]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 MiB/s wr, 96 op/s 2023-08-11T14:04:31.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:31 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2580398633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:04:31.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa04b6040 0x559aa04b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:31.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c7ffff700 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa04b6040 0x559aa04b8460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=848795f7b0f3e41 server_cookie=ebc2342886d897d in_seq=0 out_seq=0 2023-08-11T14:04:31.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c5f7fe700 10 client.25721 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:04:31.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.789+0000 7f8c5f7fe700 1 -- 172.21.15.3:0/653872386 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559aa0396820 con 0x559aa04b6040 2023-08-11T14:04:31.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c5f7fe700 10 client.25721 handle_command_reply: tid=0 2023-08-11T14:04:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:31 smithi003 ceph-mon[97195]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 MiB/s wr, 96 op/s 2023-08-11T14:04:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:31 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2580398633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:04:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 client.25721 shutdown 2023-08-11T14:04:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 msgr2=0x559aa0458c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c740c9e50 tx=0x7f8c74021c20 comp rx=0 tx=0).stop 2023-08-11T14:04:31.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 shutdown_connections 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x559aa04b6040 0x559aa04b8460 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559aa044cdf0 0x559aa04591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559aa044bad0 0x559aa045c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 --2- 172.21.15.3:0/653872386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559aa044c430 0x559aa0458c70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 >> 172.21.15.3:0/653872386 conn(0x559aa0378950 msgr2=0x559aa0446280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c5f7fe700 0 client.25721 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:04:31.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 shutdown_connections 2023-08-11T14:04:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 1 -- 172.21.15.3:0/653872386 wait complete. 2023-08-11T14:04:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.790+0000 7f8c9a223b80 20 client.25721 trim_cache size 0 max 16384 2023-08-11T14:04:31.801 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 msgr2=0x7f8c680a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 0x7f8c680a2410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009fd0 tx=0x7f8c78003cf0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 msgr2=0x7f8c8019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005720 tx=0x7f8c74009580 comp rx=0 tx=0).stop 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 shutdown_connections 2023-08-11T14:04:31.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c8010cf00 0x7f8c801a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c80073380 0x7f8c8019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8c6809ff50 0x7f8c680a2410 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 --2- 172.21.15.3:0/1344904188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80074960 0x7f8c8019d1b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 >> 172.21.15.3:0/1344904188 conn(0x7f8c800fbea0 msgr2=0x7f8c800fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 shutdown_connections 2023-08-11T14:04:31.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:31.791+0000 7f8c869ff700 1 -- 172.21.15.3:0/1344904188 wait complete. 2023-08-11T14:04:31.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-08-11T14:04:32.021 INFO:teuthology.orchestra.run.smithi003.stderr:6300000 of 50000000 tuples (12%) done (elapsed 17.52 s, remaining 121.54 s) 2023-08-11T14:04:32.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/2442812702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410a290 msgr2=0x7fbb8410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/2442812702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410a290 0x7fbb8410c720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009f90 tx=0x7fbb7c00b330 comp rx=0 tx=0).stop 2023-08-11T14:04:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/2442812702 shutdown_connections 2023-08-11T14:04:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/2442812702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb8410cc60 0x7fbb8410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/2442812702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/2442812702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410a290 0x7fbb8410c720 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009f90 tx=0x7fbb7c00b330 comp rx=0 tx=0).stop 2023-08-11T14:04:32.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.168+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/2442812702 >> 172.21.15.3:0/2442812702 conn(0x7fbb8406d6d0 msgr2=0x7fbb8406fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.171+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/2442812702 shutdown_connections 2023-08-11T14:04:32.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.172+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/2442812702 wait complete. 2023-08-11T14:04:32.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.173+0000 7fbb8a9ff700 1 Processor -- start 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 -- start start 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410cc60 0x7fbb84116580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 0x7fbb841b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84116be0 con 0x7fbb841b3660 2023-08-11T14:04:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841b5f90 con 0x7fbb8410cc60 2023-08-11T14:04:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.174+0000 7fbb8a9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841b60d0 con 0x7fbb84107920 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb837fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410cc60 0x7fbb84116580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42228/0 (socket says 172.21.15.3:42228) 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 -- 172.21.15.3:0/955997763 learned_addr learned my addr 172.21.15.3:0/955997763 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 0x7fbb841b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 -- 172.21.15.3:0/955997763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410cc60 msgr2=0x7fbb84116580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410cc60 0x7fbb84116580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 -- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 msgr2=0x7fbb841b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 0x7fbb841b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb83fff700 1 -- 172.21.15.3:0/955997763 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009c40 con 0x7fbb84107920 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.175+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 0x7fbb841b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb83fff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e9c0 tx=0x7fbb7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb817fa700 1 -- 172.21.15.3:0/955997763 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800cbd0 con 0x7fbb84107920 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb817fa700 1 -- 172.21.15.3:0/955997763 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7800eeb0 con 0x7fbb84107920 2023-08-11T14:04:32.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb817fa700 1 -- 172.21.15.3:0/955997763 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78005580 con 0x7fbb84107920 2023-08-11T14:04:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb841b6300 con 0x7fbb84107920 2023-08-11T14:04:32.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.176+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841b67c0 con 0x7fbb84107920 2023-08-11T14:04:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.184+0000 7fbb9e30eb80 1 Processor -- start 2023-08-11T14:04:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.184+0000 7fbb817fa700 1 -- 172.21.15.3:0/955997763 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbb78004750 con 0x7fbb84107920 2023-08-11T14:04:32.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 -- start start 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee4b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4bdf0 0x561fdee4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdeda3630 con 0x561fdee4b430 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdeda0f50 con 0x561fdee4aad0 2023-08-11T14:04:32.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.185+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fded9c840 con 0x561fdee4bdf0 2023-08-11T14:04:32.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.186+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.186+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53446/0 (socket says 172.21.15.3:53446) 2023-08-11T14:04:32.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.186+0000 7fbb88f9c700 1 -- 172.21.15.3:0/571462135 learned_addr learned my addr 172.21.15.3:0/571462135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.186+0000 7fbb837fe700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4bdf0 0x561fdee4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb83fff700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee4b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb88f9c700 1 -- 172.21.15.3:0/571462135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4bdf0 msgr2=0x561fdee4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4bdf0 0x561fdee4c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb88f9c700 1 -- 172.21.15.3:0/571462135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 msgr2=0x561fdee4b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee4b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb88f9c700 1 -- 172.21.15.3:0/571462135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fdec1c630 con 0x561fdee4aad0 2023-08-11T14:04:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb83fff700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee4b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:04:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.187+0000 7fbb817fa700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 0x7fbb6c0a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.188+0000 7fbb817fa700 1 -- 172.21.15.3:0/955997763 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbb78014070 con 0x7fbb84107920 2023-08-11T14:04:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.189+0000 7fbb83fff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 0x7fbb6c0a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.189+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008cc0 tx=0x7fbb7400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1afd155b508bbd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.190+0000 7fbb6a7fc700 1 -- 172.21.15.3:0/571462135 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400ed50 con 0x561fdee4aad0 2023-08-11T14:04:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.190+0000 7fbb6a7fc700 1 -- 172.21.15.3:0/571462135 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7400eeb0 con 0x561fdee4aad0 2023-08-11T14:04:32.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.190+0000 7fbb6a7fc700 1 -- 172.21.15.3:0/571462135 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74010430 con 0x561fdee4aad0 2023-08-11T14:04:32.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/571462135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 msgr2=0x561fdee4aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008cc0 tx=0x7fbb7400e350 comp rx=0 tx=0).stop 2023-08-11T14:04:32.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb83fff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 0x7fbb6c0a2510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb78098c80 tx=0x7fbb7800cd30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/571462135 shutdown_connections 2023-08-11T14:04:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4bdf0 0x561fdee4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee4aef0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/571462135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee4b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/571462135 >> 172.21.15.3:0/571462135 conn(0x561fded77950 msgr2=0x561fdee45420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/571462135 shutdown_connections 2023-08-11T14:04:32.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.191+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/571462135 wait complete. 2023-08-11T14:04:32.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.192+0000 7fbb9e30eb80 1 -- start start 2023-08-11T14:04:32.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.192+0000 7fbb9e30eb80 1 Processor -- start 2023-08-11T14:04:32.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.193+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee54680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4bdf0 0x561fdee5b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fded3aac0 con 0x561fdee4b430 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fded3a7c0 con 0x561fdee4bdf0 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdec1e920 con 0x561fdee4aad0 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb9e30eb80 10 client.0 _ll_register_callbacks cb 0x561fded3dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42246/0 (socket says 172.21.15.3:42246) 2023-08-11T14:04:32.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb88f9c700 1 -- 172.21.15.3:0/129841299 learned_addr learned my addr 172.21.15.3:0/129841299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.194+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4bdf0 0x561fdee5b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 -- 172.21.15.3:0/129841299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4bdf0 msgr2=0x561fdee5b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4bdf0 0x561fdee5b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 -- 172.21.15.3:0/129841299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 msgr2=0x561fdee54680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb837fe700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee54680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee54680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 -- 172.21.15.3:0/129841299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740088f0 con 0x561fdee4aad0 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb837fe700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee54680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008c90 tx=0x7fbb74004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90b602f2f84952c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.195+0000 7fbb637fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74029670 con 0x561fdee4aad0 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbb7400ecf0 con 0x561fdee4aad0 2023-08-11T14:04:32.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fdee3da90 con 0x561fdee4aad0 2023-08-11T14:04:32.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb740297d0 con 0x561fdee4aad0 2023-08-11T14:04:32.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb9e30eb80 10 client.25741 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:32.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.196+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fdebc8de0 con 0x561fdee4aad0 2023-08-11T14:04:32.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561fdee3da90 con 0x561fdee4aad0 2023-08-11T14:04:32.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbb740187f0 con 0x561fdee4aad0 2023-08-11T14:04:32.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 10 client.25741 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:32.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 10 client.25741 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 20 client.25741 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdee5fe30 0x561fdee62250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 4 client.25741 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fdee194e0 con 0x561fdee5fe30 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.197+0000 7fbb837fe700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdee5fe30 0x561fdee62250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.198+0000 7fbb837fe700 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdee5fe30 0x561fdee62250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3d8cc398d2fce80 server_cookie=4869b42385bd8da3 in_seq=0 out_seq=0 2023-08-11T14:04:32.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.198+0000 7fbb637fe700 10 client.25741 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb637fe700 1 -- 172.21.15.3:0/129841299 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561fdee194e0 con 0x561fdee5fe30 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb637fe700 10 client.25741 handle_command_reply: tid=0 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 client.25741 shutdown 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 msgr2=0x561fdee54140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbb74008c90 tx=0x7fbb74004f10 comp rx=0 tx=0).stop 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 shutdown_connections 2023-08-11T14:04:32.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdee5fe30 0x561fdee62250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4bdf0 0x561fdee5b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee54140 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/129841299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4b430 0x561fdee54680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb637fe700 0 client.25741 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.199+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 >> 172.21.15.3:0/129841299 conn(0x561fded77950 msgr2=0x561fdee3fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.200+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 shutdown_connections 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.200+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/129841299 wait complete. 2023-08-11T14:04:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.200+0000 7fbb9e30eb80 20 client.25741 trim_cache size 0 max 16384 2023-08-11T14:04:32.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 Processor -- start 2023-08-11T14:04:32.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 -- start start 2023-08-11T14:04:32.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4b430 0x561fdee5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee4ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee41e70 con 0x561fdee4bdf0 2023-08-11T14:04:32.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee552f0 con 0x561fdee4aad0 2023-08-11T14:04:32.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.220+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee46100 con 0x561fdee4b430 2023-08-11T14:04:32.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.221+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.221+0000 7fbb88f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53454/0 (socket says 172.21.15.3:53454) 2023-08-11T14:04:32.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.221+0000 7fbb88f9c700 1 -- 172.21.15.3:0/2215096555 learned_addr learned my addr 172.21.15.3:0/2215096555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.221+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee4ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb837fe700 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4b430 0x561fdee5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 -- 172.21.15.3:0/2215096555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4b430 msgr2=0x561fdee5b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4b430 0x561fdee5b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 -- 172.21.15.3:0/2215096555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 msgr2=0x561fdee4ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee4ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 -- 172.21.15.3:0/2215096555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fdebb1c60 con 0x561fdee4aad0 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e158fe0bae7217 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.222+0000 7fbb6affd700 1 -- 172.21.15.3:0/2215096555 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74018cf0 con 0x561fdee4aad0 2023-08-11T14:04:32.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.223+0000 7fbb6affd700 1 -- 172.21.15.3:0/2215096555 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7401d030 con 0x561fdee4aad0 2023-08-11T14:04:32.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.223+0000 7fbb6affd700 1 -- 172.21.15.3:0/2215096555 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740296d0 con 0x561fdee4aad0 2023-08-11T14:04:32.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.223+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/2215096555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 msgr2=0x561fdee521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.223+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74004f10 comp rx=0 tx=0).stop 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/2215096555 shutdown_connections 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee4ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4aad0 0x561fdee521a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/2215096555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4b430 0x561fdee5b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/2215096555 >> 172.21.15.3:0/2215096555 conn(0x561fded77950 msgr2=0x561fdee453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/2215096555 shutdown_connections 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/2215096555 wait complete. 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.224+0000 7fbb9e30eb80 1 -- start start 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 Processor -- start 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee5a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee5aa40 con 0x561fdee4bdf0 2023-08-11T14:04:32.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee5abb0 con 0x561fdee4b430 2023-08-11T14:04:32.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdee5ad70 con 0x561fdee4aad0 2023-08-11T14:04:32.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.225+0000 7fbb9e30eb80 10 client.0 _ll_register_callbacks cb 0x7fbb74004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:32.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53456/0 (socket says 172.21.15.3:53456) 2023-08-11T14:04:32.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 -- 172.21.15.3:0/1862012586 learned_addr learned my addr 172.21.15.3:0/1862012586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee5a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 -- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 msgr2=0x561fdee56fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee56fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 -- 172.21.15.3:0/1862012586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 msgr2=0x561fdee5a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee5a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.226+0000 7fbb837fe700 1 -- 172.21.15.3:0/1862012586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fdebc8de0 con 0x561fdee4b430 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.227+0000 7fbb837fe700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009040 tx=0x7fbb7c007490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7feeb0301f637f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.227+0000 7fbb637fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c022040 con 0x561fdee4b430 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbb7c0244b0 con 0x561fdee4b430 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb7400eeb0 con 0x561fdee4b430 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb7c00e040 con 0x561fdee4b430 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb9e30eb80 10 client.25739 fetch_fsmap learned FSMap version 27 2023-08-11T14:04:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.228+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fdebef250 con 0x561fdee4b430 2023-08-11T14:04:32.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7400eeb0 con 0x561fdee4b430 2023-08-11T14:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbb7c00b050 con 0x561fdee4b430 2023-08-11T14:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 27 2023-08-11T14:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 10 client.25739 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-08-11T14:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 20 client.25739 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdeeb4580 0x561fdeeb69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 4 client.25739 mds_command: new command op to 24532 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.229+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fded95820 con 0x561fdeeb4580 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.230+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdeeb4580 0x561fdeeb69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.231+0000 7fbb88f9c700 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdeeb4580 0x561fdeeb69a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1d4d535b0a3bbbf server_cookie=742895cad863ed13 in_seq=0 out_seq=0 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.231+0000 7fbb637fe700 10 client.25739 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.231+0000 7fbb637fe700 1 -- 172.21.15.3:0/1862012586 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fded95820 con 0x561fdeeb4580 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.231+0000 7fbb637fe700 10 client.25739 handle_command_reply: tid=0 2023-08-11T14:04:32.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.231+0000 7fbb9e30eb80 1 client.25739 shutdown 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 msgr2=0x561fdee574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009040 tx=0x7fbb7c007490 comp rx=0 tx=0).stop 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 shutdown_connections 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x561fdeeb4580 0x561fdeeb69a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561fdee4bdf0 0x561fdee5a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fdee4aad0 0x561fdee56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 --2- 172.21.15.3:0/1862012586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fdee4b430 0x561fdee574e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 >> 172.21.15.3:0/1862012586 conn(0x561fded77950 msgr2=0x561fdee4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb637fe700 0 client.25739 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 shutdown_connections 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 1 -- 172.21.15.3:0/1862012586 wait complete. 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.232+0000 7fbb9e30eb80 20 client.25739 trim_cache size 0 max 16384 2023-08-11T14:04:32.237 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 msgr2=0x7fbb6c0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 0x7fbb6c0a2510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbb78098c80 tx=0x7fbb7800cd30 comp rx=0 tx=0).stop 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 msgr2=0x7fbb84116040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e9c0 tx=0x7fbb7800ecd0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 shutdown_connections 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb841b3660 0x7fbb841b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb84107920 0x7fbb84116040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbb6c0a0050 0x7fbb6c0a2510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 --2- 172.21.15.3:0/955997763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb8410cc60 0x7fbb84116580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 >> 172.21.15.3:0/955997763 conn(0x7fbb8406d6d0 msgr2=0x7fbb8406fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 shutdown_connections 2023-08-11T14:04:32.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.233+0000 7fbb8a9ff700 1 -- 172.21.15.3:0/955997763 wait complete. 2023-08-11T14:04:32.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-08-11T14:04:32.489 INFO:teuthology.orchestra.run.smithi003.stderr:6400000 of 50000000 tuples (12%) done (elapsed 17.99 s, remaining 122.55 s) 2023-08-11T14:04:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 -- 172.21.15.3:0/1688675876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8074960 msgr2=0x7f51f8072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 --2- 172.21.15.3:0/1688675876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8074960 0x7f51f8072db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009990 tx=0x7f51f4009ca0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 -- 172.21.15.3:0/1688675876 shutdown_connections 2023-08-11T14:04:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 --2- 172.21.15.3:0/1688675876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 0x7f51f810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 --2- 172.21.15.3:0/1688675876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8074960 0x7f51f8072db0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 --2- 172.21.15.3:0/1688675876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8073380 0x7f51f8073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 -- 172.21.15.3:0/1688675876 >> 172.21.15.3:0/1688675876 conn(0x7f51f80fbea0 msgr2=0x7f51f80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 -- 172.21.15.3:0/1688675876 shutdown_connections 2023-08-11T14:04:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.499+0000 7f52007ff700 1 -- 172.21.15.3:0/1688675876 wait complete. 2023-08-11T14:04:32.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.501+0000 7f52007ff700 1 Processor -- start 2023-08-11T14:04:32.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.501+0000 7f52007ff700 1 -- start start 2023-08-11T14:04:32.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.501+0000 7f52007ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.501+0000 7f52007ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 0x7f51f819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.501+0000 7f52007ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 0x7f51f81a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f52007ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a47a0 con 0x7f51f8074960 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f52007ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a4910 con 0x7f51f810cf00 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f52007ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81a4ad0 con 0x7f51f8073380 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42266/0 (socket says 172.21.15.3:42266) 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 -- 172.21.15.3:0/3842890792 learned_addr learned my addr 172.21.15.3:0/3842890792 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fed9c700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 0x7f51f81a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 0x7f51f819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 -- 172.21.15.3:0/3842890792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 msgr2=0x7f51f81a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 0x7f51f81a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 -- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 msgr2=0x7f51f819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 0x7f51f819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 -- 172.21.15.3:0/3842890792 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40095f0 con 0x7f51f8073380 2023-08-11T14:04:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 0x7f51f819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.502+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.503+0000 7f51ef7fe700 1 -- 172.21.15.3:0/3842890792 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f401d070 con 0x7f51f8073380 2023-08-11T14:04:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.503+0000 7f51ef7fe700 1 -- 172.21.15.3:0/3842890792 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f4022470 con 0x7f51f8073380 2023-08-11T14:04:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.503+0000 7f51ef7fe700 1 -- 172.21.15.3:0/3842890792 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017650 con 0x7f51f8073380 2023-08-11T14:04:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.503+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f81a4d50 con 0x7f51f8073380 2023-08-11T14:04:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.503+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81a5110 con 0x7f51f8073380 2023-08-11T14:04:32.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.506+0000 7f51ef7fe700 1 -- 172.21.15.3:0/3842890792 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f51f400fad0 con 0x7f51f8073380 2023-08-11T14:04:32.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 Processor -- start 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f51ef7fe700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 0x7f51e40a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 -- start start 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f51ef7fe700 1 -- 172.21.15.3:0/3842890792 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f51f40cc9d0 con 0x7f51f8073380 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f51fed9c700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 0x7f51e40a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ed2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ed4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872e2b630 con 0x55a872ed2ad0 2023-08-11T14:04:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872e28f50 con 0x55a872ed3df0 2023-08-11T14:04:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.508+0000 7f52141d7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872e24840 con 0x55a872ed3430 2023-08-11T14:04:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42278/0 (socket says 172.21.15.3:42278) 2023-08-11T14:04:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 -- 172.21.15.3:0/4182040210 learned_addr learned my addr 172.21.15.3:0/4182040210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fe59b700 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ed2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fed9c700 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ed4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 -- 172.21.15.3:0/4182040210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 msgr2=0x55a872ed4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ed4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 -- 172.21.15.3:0/4182040210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 msgr2=0x55a872ed2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ed2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fed9c700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 0x7f51e40a2560 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51f000f590 tx=0x7f51f000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 -- 172.21.15.3:0/4182040210 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a872ca4630 con 0x55a872ed3430 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.509+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009d60 tx=0x7f51e80116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2569948bd7c22c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f51ecff9700 1 -- 172.21.15.3:0/4182040210 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8011e20 con 0x55a872ed3430 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f51ecff9700 1 -- 172.21.15.3:0/4182040210 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e8012910 con 0x55a872ed3430 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f51ecff9700 1 -- 172.21.15.3:0/4182040210 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800f840 con 0x55a872ed3430 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f52141d7b80 1 -- 172.21.15.3:0/4182040210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 msgr2=0x55a872ed38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f52141d7b80 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009d60 tx=0x7f51e80116c0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f52141d7b80 1 -- 172.21.15.3:0/4182040210 shutdown_connections 2023-08-11T14:04:32.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.510+0000 7f52141d7b80 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ed4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ed2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 --2- 172.21.15.3:0/4182040210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872ed38b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 -- 172.21.15.3:0/4182040210 >> 172.21.15.3:0/4182040210 conn(0x55a872dff950 msgr2=0x55a872ecd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 -- 172.21.15.3:0/4182040210 shutdown_connections 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 -- 172.21.15.3:0/4182040210 wait complete. 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.511+0000 7f52141d7b80 1 -- start start 2023-08-11T14:04:32.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 Processor -- start 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3430 0x55a872edc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3df0 0x55a872ee3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872dc2ac0 con 0x55a872ed3df0 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872dc27c0 con 0x55a872ed3430 2023-08-11T14:04:32.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.512+0000 7f52141d7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872ca6920 con 0x55a872ed2ad0 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42282/0 (socket says 172.21.15.3:42282) 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 -- 172.21.15.3:0/3813936298 learned_addr learned my addr 172.21.15.3:0/3813936298 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f52141d7b80 10 client.0 _ll_register_callbacks cb 0x55a872dc5ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fed9c700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3df0 0x55a872ee3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3430 0x55a872edc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 -- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3430 msgr2=0x55a872edc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3430 0x55a872edc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 -- 172.21.15.3:0/3813936298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3df0 msgr2=0x55a872ee3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3df0 0x55a872ee3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 -- 172.21.15.3:0/3813936298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e8009990 con 0x55a872ed2ad0 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51fe59b700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f51f409a0c0 tx=0x7f51f4099bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340f6f8d728c601a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.513+0000 7f51e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b1070 con 0x55a872ed2ad0 2023-08-11T14:04:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f51f409a130 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a872ec5a90 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f40b2260 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 10 client.25759 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a872c50de0 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a872ec5a90 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f51f40a1a20 con 0x55a872ed2ad0 2023-08-11T14:04:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 20 client.25759 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872ee7a60 0x55a872ee9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 4 client.25759 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.514+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a872ea14e0 con 0x55a872ee7a60 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.515+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872ee7a60 0x55a872ee9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.515+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872ee7a60 0x55a872ee9e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c30178aa1f40248 server_cookie=e7cf54eb9a66ce7d in_seq=0 out_seq=0 2023-08-11T14:04:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.515+0000 7f51e1ffb700 10 client.25759 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f51e1ffb700 1 -- 172.21.15.3:0/3813936298 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a872ea14e0 con 0x55a872ee7a60 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f51e1ffb700 10 client.25759 handle_command_reply: tid=0 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 client.25759 shutdown 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 msgr2=0x55a872edc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f51f409a0c0 tx=0x7f51f4099bc0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 shutdown_connections 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3df0 0x55a872ee3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872edc0f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872ee7a60 0x55a872ee9e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 --2- 172.21.15.3:0/3813936298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3430 0x55a872edc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f51e1ffb700 0 client.25759 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 >> 172.21.15.3:0/3813936298 conn(0x55a872dff950 msgr2=0x55a872ec7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 shutdown_connections 2023-08-11T14:04:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 1 -- 172.21.15.3:0/3813936298 wait complete. 2023-08-11T14:04:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.516+0000 7f52141d7b80 20 client.25759 trim_cache size 0 max 16384 2023-08-11T14:04:32.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 Processor -- start 2023-08-11T14:04:32.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 -- start start 2023-08-11T14:04:32.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3430 0x55a872ee3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ec7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872eceae0 con 0x55a872ed3430 2023-08-11T14:04:32.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872edabd0 con 0x55a872ed3df0 2023-08-11T14:04:32.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.530+0000 7f52141d7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872edd270 con 0x55a872ed2ad0 2023-08-11T14:04:32.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42292/0 (socket says 172.21.15.3:42292) 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 -- 172.21.15.3:0/2704113506 learned_addr learned my addr 172.21.15.3:0/2704113506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fed9c700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ec7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3430 0x55a872ee3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 -- 172.21.15.3:0/2704113506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 msgr2=0x55a872ec7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ec7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 -- 172.21.15.3:0/2704113506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3430 msgr2=0x55a872ee3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3430 0x55a872ee3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 -- 172.21.15.3:0/2704113506 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a872edb110 con 0x55a872ed2ad0 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fed9c700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ec7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.531+0000 7f51fe59b700 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f51f4096ba0 tx=0x7f51f4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbaa6f4916a496 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f51ed7fa700 1 -- 172.21.15.3:0/2704113506 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b1070 con 0x55a872ed2ad0 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f51ed7fa700 1 -- 172.21.15.3:0/2704113506 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f409c070 con 0x55a872ed2ad0 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f51ed7fa700 1 -- 172.21.15.3:0/2704113506 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40a1f40 con 0x55a872ed2ad0 2023-08-11T14:04:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 -- 172.21.15.3:0/2704113506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 msgr2=0x55a872ed1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f51f4096ba0 tx=0x7f51f4005c00 comp rx=0 tx=0).stop 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 -- 172.21.15.3:0/2704113506 shutdown_connections 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ec7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed2ad0 0x55a872ed1fe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 --2- 172.21.15.3:0/2704113506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed3430 0x55a872ee3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.532+0000 7f52141d7b80 1 -- 172.21.15.3:0/2704113506 >> 172.21.15.3:0/2704113506 conn(0x55a872dff950 msgr2=0x55a872ed18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.533+0000 7f52141d7b80 1 -- 172.21.15.3:0/2704113506 shutdown_connections 2023-08-11T14:04:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.533+0000 7f52141d7b80 1 -- 172.21.15.3:0/2704113506 wait complete. 2023-08-11T14:04:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.533+0000 7f52141d7b80 1 -- start start 2023-08-11T14:04:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.533+0000 7f52141d7b80 1 Processor -- start 2023-08-11T14:04:32.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872edfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ee0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872ee0850 con 0x55a872ed2ad0 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872ee3810 con 0x55a872ed3df0 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a872ee39a0 con 0x55a872ed3430 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f52141d7b80 10 client.0 _ll_register_callbacks cb 0x7f51f4099fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fe59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34438/0 (socket says 172.21.15.3:34438) 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fe59b700 1 -- 172.21.15.3:0/1377422590 learned_addr learned my addr 172.21.15.3:0/1377422590 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872edfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fed9c700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ee0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fdd9a700 1 -- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 msgr2=0x55a872ee0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ee0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fdd9a700 1 -- 172.21.15.3:0/1377422590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 msgr2=0x55a872ee3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.534+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51fdd9a700 1 -- 172.21.15.3:0/1377422590 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a872c50de0 con 0x55a872ed3430 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51fe59b700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51fdd9a700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872edfc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f51e801c040 tx=0x7f51e8009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7599a4619d89298e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8012450 con 0x55a872ed3430 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e8012910 con 0x55a872ed3430 2023-08-11T14:04:32.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f51e80125b0 con 0x55a872ed3430 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51e8012770 con 0x55a872ed3430 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f52141d7b80 10 client.25765 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.535+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a872c39c60 con 0x55a872ed3430 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51e8012910 con 0x55a872ed3430 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f51e8005140 con 0x55a872ed3430 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 20 client.25765 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872f3cff0 0x55a872f3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 4 client.25765 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a872e1d820 con 0x55a872f3cff0 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.536+0000 7f51fe59b700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872f3cff0 0x55a872f3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f51fe59b700 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872f3cff0 0x55a872f3f410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35fa253c9b5d73a7 server_cookie=93400f6d586be0bd in_seq=0 out_seq=0 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f51e1ffb700 10 client.25765 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f51e1ffb700 1 -- 172.21.15.3:0/1377422590 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a872e1d820 con 0x55a872f3cff0 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f51e1ffb700 10 client.25765 handle_command_reply: tid=0 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 client.25765 shutdown 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 msgr2=0x55a872edfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872edfc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f51e801c040 tx=0x7f51e8009f40 comp rx=0 tx=0).stop 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 shutdown_connections 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a872f3cff0 0x55a872f3f410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a872ed3df0 0x55a872ee0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a872ed2ad0 0x55a872ee3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 --2- 172.21.15.3:0/1377422590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a872ed3430 0x55a872edfc30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f51e1ffb700 0 client.25765 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 >> 172.21.15.3:0/1377422590 conn(0x55a872dff950 msgr2=0x55a872ed2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 shutdown_connections 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.537+0000 7f52141d7b80 1 -- 172.21.15.3:0/1377422590 wait complete. 2023-08-11T14:04:32.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52141d7b80 20 client.25765 trim_cache size 0 max 16384 2023-08-11T14:04:32.548 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:04:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 msgr2=0x7f51e40a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 0x7f51e40a2560 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51f000f590 tx=0x7f51f000b040 comp rx=0 tx=0).stop 2023-08-11T14:04:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 msgr2=0x7f51f819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f4005740 comp rx=0 tx=0).stop 2023-08-11T14:04:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 shutdown_connections 2023-08-11T14:04:32.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51e40a00a0 0x7f51e40a2560 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51f810cf00 0x7f51f81a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f8073380 0x7f51f819cc60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 --2- 172.21.15.3:0/3842890792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51f8074960 0x7f51f819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 >> 172.21.15.3:0/3842890792 conn(0x7f51f80fbea0 msgr2=0x7f51f80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.538+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 shutdown_connections 2023-08-11T14:04:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:32.539+0000 7f52007ff700 1 -- 172.21.15.3:0/3842890792 wait complete. 2023-08-11T14:04:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:6500000 of 50000000 tuples (13%) done (elapsed 18.17 s, remaining 121.60 s) 2023-08-11T14:04:32.936 INFO:teuthology.orchestra.run.smithi003.stderr:6600000 of 50000000 tuples (13%) done (elapsed 18.44 s, remaining 121.23 s) 2023-08-11T14:04:33.237 INFO:teuthology.orchestra.run.smithi003.stderr:6700000 of 50000000 tuples (13%) done (elapsed 18.74 s, remaining 121.09 s) 2023-08-11T14:04:33.443 INFO:teuthology.orchestra.run.smithi003.stderr:6800000 of 50000000 tuples (13%) done (elapsed 18.94 s, remaining 120.35 s) 2023-08-11T14:04:33.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-08-11T14:04:33.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-08-11T14:04:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:33 smithi103 ceph-mon[130434]: pgmap v366: 129 pgs: 129 active+clean; 935 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 108 MiB/s wr, 122 op/s 2023-08-11T14:04:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:33 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:33.711 INFO:teuthology.orchestra.run.smithi003.stderr:6900000 of 50000000 tuples (13%) done (elapsed 19.21 s, remaining 120.00 s) 2023-08-11T14:04:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 -- 172.21.15.3:0/290869747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474073380 msgr2=0x7f6474073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 --2- 172.21.15.3:0/290869747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474073380 0x7f6474073800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6460009990 tx=0x7f6460009ca0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 -- 172.21.15.3:0/290869747 shutdown_connections 2023-08-11T14:04:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 --2- 172.21.15.3:0/290869747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64741087d0 0x7f647410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 --2- 172.21.15.3:0/290869747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6474074960 0x7f6474072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 --2- 172.21.15.3:0/290869747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474073380 0x7f6474073800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 -- 172.21.15.3:0/290869747 >> 172.21.15.3:0/290869747 conn(0x7f64740fbe60 msgr2=0x7f64740fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 -- 172.21.15.3:0/290869747 shutdown_connections 2023-08-11T14:04:33.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.787+0000 7f647abff700 1 -- 172.21.15.3:0/290869747 wait complete. 2023-08-11T14:04:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.788+0000 7f647abff700 1 Processor -- start 2023-08-11T14:04:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.788+0000 7f647abff700 1 -- start start 2023-08-11T14:04:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 0x7f6474198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 0x7f6474198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 0x7f647419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741a03d0 con 0x7f6474073380 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741a0510 con 0x7f64741087d0 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647abff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741a06a0 con 0x7f6474074960 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 0x7f6474198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f6473fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 0x7f6474198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647919c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 0x7f647419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 0x7f6474198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34470/0 (socket says 172.21.15.3:34470) 2023-08-11T14:04:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647919c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 0x7f647419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53506/0 (socket says 172.21.15.3:53506) 2023-08-11T14:04:33.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.789+0000 7f647899b700 1 -- 172.21.15.3:0/208789696 learned_addr learned my addr 172.21.15.3:0/208789696 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:33.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:33 smithi003 ceph-mon[97195]: pgmap v366: 129 pgs: 129 active+clean; 935 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 108 MiB/s wr, 122 op/s 2023-08-11T14:04:33.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:33 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 -- 172.21.15.3:0/208789696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 msgr2=0x7f647419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 0x7f647419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 -- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 msgr2=0x7f6474198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 0x7f6474198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 -- 172.21.15.3:0/208789696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600095f0 con 0x7f6474074960 2023-08-11T14:04:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6473fff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 0x7f6474198d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64600098e0 tx=0x7f6460009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6471ffb700 1 -- 172.21.15.3:0/208789696 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646001d070 con 0x7f6474074960 2023-08-11T14:04:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64741a0920 con 0x7f6474074960 2023-08-11T14:04:33.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6471ffb700 1 -- 172.21.15.3:0/208789696 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6460021410 con 0x7f6474074960 2023-08-11T14:04:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f6471ffb700 1 -- 172.21.15.3:0/208789696 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460005720 con 0x7f6474074960 2023-08-11T14:04:33.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.790+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64741a0da0 con 0x7f6474074960 2023-08-11T14:04:33.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-mon[130973]: pgmap v366: 129 pgs: 129 active+clean; 935 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 108 MiB/s wr, 122 op/s 2023-08-11T14:04:33.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:33.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.794+0000 7f6471ffb700 1 -- 172.21.15.3:0/208789696 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6460021580 con 0x7f6474074960 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 Processor -- start 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 -- start start 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e912e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e912f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e90866b0 con 0x55f3e912e4b0 2023-08-11T14:04:33.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e9083fd0 con 0x55f3e912ee70 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.795+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e907f8c0 con 0x55f3e912db50 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f6471ffb700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 0x7f64640a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f6471ffb700 1 -- 172.21.15.3:0/208789696 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f64600cc770 con 0x7f6474074960 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e912e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42314/0 (socket says 172.21.15.3:42314) 2023-08-11T14:04:33.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e912e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34478/0 (socket says 172.21.15.3:34478) 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 -- 172.21.15.3:0/2419823195 learned_addr learned my addr 172.21.15.3:0/2419823195 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 0x7f64640a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f6473fff700 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e912f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 -- 172.21.15.3:0/2419823195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 msgr2=0x55f3e912f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e912f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 -- 172.21.15.3:0/2419823195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 msgr2=0x55f3e912e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e912e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.796+0000 7f647919c700 1 -- 172.21.15.3:0/2419823195 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3e8eff6b0 con 0x55f3e912db50 2023-08-11T14:04:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.797+0000 7f647919c700 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f646c008ad0 tx=0x7f646c008de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453ca3260b04e89a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.797+0000 7f647919c700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 0x7f64640a2490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f646c011000 tx=0x7f646c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.797+0000 7f645effd700 1 -- 172.21.15.3:0/2419823195 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c00e070 con 0x55f3e912db50 2023-08-11T14:04:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.797+0000 7f645effd700 1 -- 172.21.15.3:0/2419823195 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f646c00f040 con 0x55f3e912db50 2023-08-11T14:04:33.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.797+0000 7f645effd700 1 -- 172.21.15.3:0/2419823195 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c0229a0 con 0x55f3e912db50 2023-08-11T14:04:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2419823195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 msgr2=0x55f3e912df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f646c008ad0 tx=0x7f646c008de0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2419823195 shutdown_connections 2023-08-11T14:04:33.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e912f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e912df70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2419823195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e912e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2419823195 >> 172.21.15.3:0/2419823195 conn(0x55f3e905a9d0 msgr2=0x55f3e91284a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2419823195 shutdown_connections 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.798+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2419823195 wait complete. 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.799+0000 7f648e5e4b80 1 -- start start 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.799+0000 7f648e5e4b80 1 Processor -- start 2023-08-11T14:04:33.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e91376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e913e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e901db40 con 0x55f3e912db50 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e901d840 con 0x55f3e912ee70 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e8f019a0 con 0x55f3e912e4b0 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f648e5e4b80 10 client.0 _ll_register_callbacks cb 0x55f3e9020b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:33.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f647899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34494/0 (socket says 172.21.15.3:34494) 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f647899b700 1 -- 172.21.15.3:0/1243199520 learned_addr learned my addr 172.21.15.3:0/1243199520 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f6473fff700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e91376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f647919c700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e913e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f6473fff700 1 -- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 msgr2=0x55f3e913e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f6473fff700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e913e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f6473fff700 1 -- 172.21.15.3:0/1243199520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 msgr2=0x55f3e91371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.800+0000 7f6473fff700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6473fff700 1 -- 172.21.15.3:0/1243199520 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c0086b0 con 0x55f3e912e4b0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f647899b700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6473fff700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e91376e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6460099bd0 tx=0x7f6460021b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3ff0982c9d0905 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6453fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646009a330 con 0x55f3e912e4b0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f646009dd00 con 0x55f3e912e4b0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3e9120b10 con 0x55f3e912e4b0 2023-08-11T14:04:33.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64600a1c50 con 0x55f3e912e4b0 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f648e5e4b80 10 client.25777 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f3e8eabe60 con 0x55f3e912e4b0 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.801+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3e9120b10 con 0x55f3e912e4b0 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6460094ae0 con 0x55f3e912e4b0 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 10 client.25777 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 10 client.25777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:33.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 20 client.25777 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9142c40 0x55f3e9145060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 4 client.25777 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3e90fc560 con 0x55f3e9142c40 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.802+0000 7f647899b700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9142c40 0x55f3e9145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.803+0000 7f647899b700 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9142c40 0x55f3e9145060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1900f58eb816bfe server_cookie=f4b75ac51076d919 in_seq=0 out_seq=0 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.803+0000 7f6453fff700 10 client.25777 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.803+0000 7f6453fff700 1 -- 172.21.15.3:0/1243199520 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f3e90fc560 con 0x55f3e9142c40 2023-08-11T14:04:33.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.803+0000 7f6453fff700 10 client.25777 handle_command_reply: tid=0 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.803+0000 7f648e5e4b80 1 client.25777 shutdown 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 msgr2=0x55f3e91376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e91376e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6460099bd0 tx=0x7f6460021b80 comp rx=0 tx=0).stop 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 shutdown_connections 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9142c40 0x55f3e9145060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e913e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912db50 0x55f3e91371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f6453fff700 0 client.25777 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/1243199520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e91376e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 >> 172.21.15.3:0/1243199520 conn(0x55f3e905a9d0 msgr2=0x55f3e9122d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 shutdown_connections 2023-08-11T14:04:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 1 -- 172.21.15.3:0/1243199520 wait complete. 2023-08-11T14:04:33.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.804+0000 7f648e5e4b80 20 client.25777 trim_cache size 0 max 16384 2023-08-11T14:04:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 Processor -- start 2023-08-11T14:04:33.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 -- start start 2023-08-11T14:04:33.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e913e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e91372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e9120af0 con 0x55f3e912e4b0 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e9129da0 con 0x55f3e912ee70 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.823+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e9142a20 con 0x55f3e912db50 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42324/0 (socket says 172.21.15.3:42324) 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f6473fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e913e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 -- 172.21.15.3:0/3330000060 learned_addr learned my addr 172.21.15.3:0/3330000060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647919c700 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e91372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 -- 172.21.15.3:0/3330000060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 msgr2=0x55f3e91372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e91372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 -- 172.21.15.3:0/3330000060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 msgr2=0x55f3e913e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e913e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 -- 172.21.15.3:0/3330000060 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3e8e94ce0 con 0x55f3e912db50 2023-08-11T14:04:33.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647919c700 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e91372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:33.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.824+0000 7f647899b700 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64680096b0 tx=0x7f6468004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832bef82e6455550 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.825+0000 7f645f7fe700 1 -- 172.21.15.3:0/3330000060 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468016050 con 0x55f3e912db50 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.825+0000 7f645f7fe700 1 -- 172.21.15.3:0/3330000060 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6468017040 con 0x55f3e912db50 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.825+0000 7f645f7fe700 1 -- 172.21.15.3:0/3330000060 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646801ca60 con 0x55f3e912db50 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- 172.21.15.3:0/3330000060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 msgr2=0x55f3e9136db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64680096b0 tx=0x7f6468004750 comp rx=0 tx=0).stop 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- 172.21.15.3:0/3330000060 shutdown_connections 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912ee70 0x55f3e91372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912db50 0x55f3e9136db0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/3330000060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912e4b0 0x55f3e913e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- 172.21.15.3:0/3330000060 >> 172.21.15.3:0/3330000060 conn(0x55f3e905a9d0 msgr2=0x55f3e9131e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- 172.21.15.3:0/3330000060 shutdown_connections 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- 172.21.15.3:0/3330000060 wait complete. 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.826+0000 7f648e5e4b80 1 -- start start 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 Processor -- start 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e913bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912ee70 0x55f3e913ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e913d340 con 0x55f3e912ee70 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e913d4b0 con 0x55f3e912db50 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3e913d670 con 0x55f3e912e4b0 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f648e5e4b80 10 client.0 _ll_register_callbacks cb 0x7f6460099a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f647899b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f647899b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53540/0 (socket says 172.21.15.3:53540) 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f647899b700 1 -- 172.21.15.3:0/2006010507 learned_addr learned my addr 172.21.15.3:0/2006010507 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.827+0000 7f6473fff700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e913bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647919c700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912ee70 0x55f3e913ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 -- 172.21.15.3:0/2006010507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 msgr2=0x55f3e913bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e913bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 -- 172.21.15.3:0/2006010507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912ee70 msgr2=0x55f3e913ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912ee70 0x55f3e913ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 -- 172.21.15.3:0/2006010507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3e8eff6b0 con 0x55f3e912db50 2023-08-11T14:04:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f6473fff700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e913bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f647899b700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64680092a0 tx=0x7f646801df30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ed98b5f9d5a6d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:33.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.828+0000 7f6453fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:33.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468004230 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f646c00f040 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6468004390 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6468016020 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 10 client.25775 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f646801d8c0 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6468038020 con 0x55f3e912db50 2023-08-11T14:04:33.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6468026580 con 0x55f3e912db50 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 20 client.25775 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9198da0 0x55f3e919b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 4 client.25775 mds_command: new command op to 24569 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.829+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f3e90788a0 con 0x55f3e9198da0 2023-08-11T14:04:33.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.830+0000 7f6473fff700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9198da0 0x55f3e919b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.830+0000 7f6453fff700 10 client.25775 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.830+0000 7f6473fff700 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9198da0 0x55f3e919b1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4b277458ec701 server_cookie=701630554127746f in_seq=0 out_seq=0 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f6453fff700 1 -- 172.21.15.3:0/2006010507 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f3e90788a0 con 0x55f3e9198da0 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f6453fff700 10 client.25775 handle_command_reply: tid=0 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 client.25775 shutdown 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 msgr2=0x55f3e913b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f64680092a0 tx=0x7f646801df30 comp rx=0 tx=0).stop 2023-08-11T14:04:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 shutdown_connections 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f3e912ee70 0x55f3e913ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f3e9198da0 0x55f3e919b1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f6453fff700 0 client.25775 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3e912db50 0x55f3e913b8b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 --2- 172.21.15.3:0/2006010507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3e912e4b0 0x55f3e913bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 >> 172.21.15.3:0/2006010507 conn(0x55f3e905a9d0 msgr2=0x55f3e9136d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 shutdown_connections 2023-08-11T14:04:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.831+0000 7f648e5e4b80 1 -- 172.21.15.3:0/2006010507 wait complete. 2023-08-11T14:04:33.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f648e5e4b80 20 client.25775 trim_cache size 0 max 16384 2023-08-11T14:04:33.840 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:04:33.840 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2023-08-11T14:04:33.840 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "acd37ee9-5ed9-4323-8a9d-092246009150", 2023-08-11T14:04:33.840 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2023-08-11T14:04:33.841 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:04:33.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 msgr2=0x7f64640a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 0x7f64640a2490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f646c011000 tx=0x7f646c01e000 comp rx=0 tx=0).stop 2023-08-11T14:04:33.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 msgr2=0x7f6474198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 0x7f6474198d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64600098e0 tx=0x7f6460009580 comp rx=0 tx=0).stop 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 shutdown_connections 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f646409ffd0 0x7f64640a2490 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f64741087d0 0x7f647419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6474073380 0x7f6474198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 --2- 172.21.15.3:0/208789696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6474074960 0x7f6474198d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.832+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 >> 172.21.15.3:0/208789696 conn(0x7f64740fbe60 msgr2=0x7f64740fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.833+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 shutdown_connections 2023-08-11T14:04:33.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:33.833+0000 7f647abff700 1 -- 172.21.15.3:0/208789696 wait complete. 2023-08-11T14:04:33.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-11T14:04:34.088 INFO:teuthology.orchestra.run.smithi003.stderr:7000000 of 50000000 tuples (14%) done (elapsed 19.59 s, remaining 120.33 s) 2023-08-11T14:04:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 -- 172.21.15.3:0/3445290182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8074960 msgr2=0x7feaa8072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 --2- 172.21.15.3:0/3445290182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8074960 0x7feaa8072db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea98009990 tx=0x7fea98009ca0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 -- 172.21.15.3:0/3445290182 shutdown_connections 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 --2- 172.21.15.3:0/3445290182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa810cf00 0x7feaa810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 --2- 172.21.15.3:0/3445290182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8074960 0x7feaa8072db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 --2- 172.21.15.3:0/3445290182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa8073380 0x7feaa8073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 -- 172.21.15.3:0/3445290182 >> 172.21.15.3:0/3445290182 conn(0x7feaa80fbea0 msgr2=0x7feaa80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 -- 172.21.15.3:0/3445290182 shutdown_connections 2023-08-11T14:04:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.095+0000 7feaae1ff700 1 -- 172.21.15.3:0/3445290182 wait complete. 2023-08-11T14:04:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.097+0000 7feaae1ff700 1 Processor -- start 2023-08-11T14:04:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.097+0000 7feaae1ff700 1 -- start start 2023-08-11T14:04:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa8074960 0x7feaa819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa810cf00 0x7feaa81a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81a47c0 con 0x7feaa8073380 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81a4930 con 0x7feaa8074960 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaae1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81a4af0 con 0x7feaa810cf00 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34536/0 (socket says 172.21.15.3:34536) 2023-08-11T14:04:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 -- 172.21.15.3:0/2057784989 learned_addr learned my addr 172.21.15.3:0/2057784989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa8074960 0x7feaa819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa7fff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa810cf00 0x7feaa81a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 -- 172.21.15.3:0/2057784989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa810cf00 msgr2=0x7feaa81a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa810cf00 0x7feaa81a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 -- 172.21.15.3:0/2057784989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa8074960 msgr2=0x7feaa819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa8074960 0x7feaa819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.098+0000 7feaa77fe700 1 -- 172.21.15.3:0/2057784989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980095f0 con 0x7feaa8073380 2023-08-11T14:04:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b680 tx=0x7fea98005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaa4ff9700 1 -- 172.21.15.3:0/2057784989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9801d070 con 0x7feaa8073380 2023-08-11T14:04:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81a4d70 con 0x7feaa8073380 2023-08-11T14:04:34.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaa4ff9700 1 -- 172.21.15.3:0/2057784989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea98022470 con 0x7feaa8073380 2023-08-11T14:04:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaa4ff9700 1 -- 172.21.15.3:0/2057784989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98017650 con 0x7feaa8073380 2023-08-11T14:04:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.099+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81a51a0 con 0x7feaa8073380 2023-08-11T14:04:34.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.102+0000 7feaa4ff9700 1 -- 172.21.15.3:0/2057784989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fea980225e0 con 0x7feaa8073380 2023-08-11T14:04:34.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.104+0000 7feac1a8ab80 1 Processor -- start 2023-08-11T14:04:34.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 -- start start 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead186ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa4ff9700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 0x7fea900a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead188270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead0df630 con 0x55eead187df0 2023-08-11T14:04:34.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead0dcf50 con 0x55eead187430 2023-08-11T14:04:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead0d8840 con 0x55eead186ad0 2023-08-11T14:04:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa4ff9700 1 -- 172.21.15.3:0/2057784989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fea980ccae0 con 0x7feaa8073380 2023-08-11T14:04:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53564/0 (socket says 172.21.15.3:53564) 2023-08-11T14:04:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa6ffd700 1 -- 172.21.15.3:0/4104701072 learned_addr learned my addr 172.21.15.3:0/4104701072 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa7fff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 0x7fea900a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.105+0000 7feaa77fe700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead186ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa7fff700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead188270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 -- 172.21.15.3:0/4104701072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 msgr2=0x55eead1878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 -- 172.21.15.3:0/4104701072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 msgr2=0x55eead188270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead188270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 -- 172.21.15.3:0/4104701072 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeacf58630 con 0x55eead186ad0 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa7fff700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead188270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa77fe700 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead186ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea980ca260 tx=0x7fea98005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859e0df580d6fe11 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7feaa7fff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 0x7fea900a2410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55eead182400 tx=0x7feaa0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.106+0000 7fea8dffb700 1 -- 172.21.15.3:0/4104701072 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98099c60 con 0x55eead186ad0 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7fea8dffb700 1 -- 172.21.15.3:0/4104701072 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea980177b0 con 0x55eead186ad0 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7fea8dffb700 1 -- 172.21.15.3:0/4104701072 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980228e0 con 0x55eead186ad0 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 -- 172.21.15.3:0/4104701072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 msgr2=0x55eead186ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead186ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea980ca260 tx=0x7fea98005fd0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 -- 172.21.15.3:0/4104701072 shutdown_connections 2023-08-11T14:04:34.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead188270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead186ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/4104701072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead1878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 -- 172.21.15.3:0/4104701072 >> 172.21.15.3:0/4104701072 conn(0x55eead0b3950 msgr2=0x55eead181420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 -- 172.21.15.3:0/4104701072 shutdown_connections 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.107+0000 7feac1a8ab80 1 -- 172.21.15.3:0/4104701072 wait complete. 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.108+0000 7feac1a8ab80 1 -- start start 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.108+0000 7feac1a8ab80 1 Processor -- start 2023-08-11T14:04:34.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187430 0x55eead1902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187df0 0x55eead197330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead076ac0 con 0x55eead187430 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead0767c0 con 0x55eead187df0 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeacf5a920 con 0x55eead186ad0 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feac1a8ab80 10 client.0 _ll_register_callbacks cb 0x55eead079ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42370/0 (socket says 172.21.15.3:42370) 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 -- 172.21.15.3:0/1575439157 learned_addr learned my addr 172.21.15.3:0/1575439157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187430 0x55eead1902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa7fff700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187df0 0x55eead197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 -- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187df0 msgr2=0x55eead197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187df0 0x55eead197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 -- 172.21.15.3:0/1575439157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187430 msgr2=0x55eead1902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187430 0x55eead1902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.109+0000 7feaa77fe700 1 -- 172.21.15.3:0/1575439157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c009960 con 0x55eead186ad0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7feaa77fe700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fea98096d70 tx=0x7fea98099c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e878af59f4a17e9c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7fea86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9809e130 con 0x55eead186ad0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fea9809a770 con 0x55eead186ad0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eead179a90 con 0x55eead186ad0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea980997f0 con 0x55eead186ad0 2023-08-11T14:04:34.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7feac1a8ab80 10 client.25795 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:34.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7feaa81a51a0 con 0x55eead186ad0 2023-08-11T14:04:34.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.110+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea980c1020 con 0x55eead186ad0 2023-08-11T14:04:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fea98099260 con 0x55eead186ad0 2023-08-11T14:04:34.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 10 client.25795 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 20 client.25795 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead19b6b0 0x55eead19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 4 client.25795 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eead1554e0 con 0x55eead19b6b0 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead19b6b0 0x55eead19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.111+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead19b6b0 0x55eead19daa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6be15d418219a83 server_cookie=ee395ef4382f502 in_seq=0 out_seq=0 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7fea86ffd700 10 client.25795 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7fea86ffd700 1 -- 172.21.15.3:0/1575439157 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55eead1554e0 con 0x55eead19b6b0 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7fea86ffd700 10 client.25795 handle_command_reply: tid=0 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7feac1a8ab80 1 client.25795 shutdown 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 msgr2=0x55eead18fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.112+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fea98096d70 tx=0x7fea98099c60 comp rx=0 tx=0).stop 2023-08-11T14:04:34.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 shutdown_connections 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead19b6b0 0x55eead19daa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187df0 0x55eead197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7fea86ffd700 0 client.25795 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead186ad0 0x55eead18fdb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/1575439157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187430 0x55eead1902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 >> 172.21.15.3:0/1575439157 conn(0x55eead0b3950 msgr2=0x55eead17bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 shutdown_connections 2023-08-11T14:04:34.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 1 -- 172.21.15.3:0/1575439157 wait complete. 2023-08-11T14:04:34.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.113+0000 7feac1a8ab80 20 client.25795 trim_cache size 0 max 16384 2023-08-11T14:04:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.130+0000 7feac1a8ab80 1 Processor -- start 2023-08-11T14:04:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 -- start start 2023-08-11T14:04:34.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead18ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 0x55eead1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 0x55eead18b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead1554e0 con 0x55eead1a2650 2023-08-11T14:04:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead190fa0 con 0x55eead187430 2023-08-11T14:04:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead18e9b0 con 0x55eead187df0 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 0x55eead18b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feaa77fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead18ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 0x55eead18b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34554/0 (socket says 172.21.15.3:34554) 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feaa7fff700 1 -- 172.21.15.3:0/2818456103 learned_addr learned my addr 172.21.15.3:0/2818456103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.131+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 0x55eead1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 -- 172.21.15.3:0/2818456103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 msgr2=0x55eead1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 0x55eead1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 -- 172.21.15.3:0/2818456103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 msgr2=0x55eead18b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 0x55eead18b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 -- 172.21.15.3:0/2818456103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeacf04de0 con 0x55eead187430 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 0x55eead1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7feaa77fe700 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead18ac50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea98096800 tx=0x7fea98005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3926ec247017adf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7fea8e7fc700 1 -- 172.21.15.3:0/2818456103 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9809e130 con 0x55eead187430 2023-08-11T14:04:34.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7fea8e7fc700 1 -- 172.21.15.3:0/2818456103 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea980c1070 con 0x55eead187430 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.132+0000 7fea8e7fc700 1 -- 172.21.15.3:0/2818456103 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980a1bf0 con 0x55eead187430 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- 172.21.15.3:0/2818456103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 msgr2=0x55eead18ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead18ac50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea98096800 tx=0x7fea98005fd0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- 172.21.15.3:0/2818456103 shutdown_connections 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead1a2650 0x55eead18b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead187430 0x55eead18ac50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/2818456103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187df0 0x55eead1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- 172.21.15.3:0/2818456103 >> 172.21.15.3:0/2818456103 conn(0x55eead0b3950 msgr2=0x55eead1812f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- 172.21.15.3:0/2818456103 shutdown_connections 2023-08-11T14:04:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- 172.21.15.3:0/2818456103 wait complete. 2023-08-11T14:04:34.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.133+0000 7feac1a8ab80 1 -- start start 2023-08-11T14:04:34.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 Processor -- start 2023-08-11T14:04:34.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 0x55eead193540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead193a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 0x55eead196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead196fe0 con 0x55eead187df0 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead197150 con 0x55eead1a2650 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eead197310 con 0x55eead187430 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feac1a8ab80 10 client.0 _ll_register_callbacks cb 0x7fea9c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:34.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 0x55eead196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 0x55eead196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53580/0 (socket says 172.21.15.3:53580) 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feaa6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead193a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feaa7fff700 1 -- 172.21.15.3:0/3195915690 learned_addr learned my addr 172.21.15.3:0/3195915690 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.134+0000 7feaa77fe700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 0x55eead193540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 -- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 msgr2=0x55eead196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 0x55eead196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 -- 172.21.15.3:0/3195915690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 msgr2=0x55eead193a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:34.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead193a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 -- 172.21.15.3:0/3195915690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81a51a0 con 0x55eead187430 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead193a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feaa77fe700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 0x55eead193540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea98099540 tx=0x7fea980ca1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bae4ff7134ebb835 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7fea86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980a2080 con 0x55eead187430 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea980177b0 con 0x55eead187430 2023-08-11T14:04:34.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.135+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fea980c2080 con 0x55eead187430 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea9809a300 con 0x55eead187430 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 10 client.25801 fetch_fsmap learned FSMap version 28 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55eeaceedc60 con 0x55eead187430 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea980177b0 con 0x55eead187430 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fea980b9330 con 0x55eead187430 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 10 client.25801 fetch_fsmap finished waiting for FSMap version 28 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:34.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 20 client.25801 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead1f00e0 0x55eead1f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 4 client.25801 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.136+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eead0d1820 con 0x55eead1f00e0 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead1f00e0 0x55eead1f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7feaa6ffd700 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead1f00e0 0x55eead1f24d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63c14554bc81746 server_cookie=911b8fee0ca0c52f in_seq=0 out_seq=0 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7fea86ffd700 10 client.25801 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:34.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7fea86ffd700 1 -- 172.21.15.3:0/3195915690 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eead0d1820 con 0x55eead1f00e0 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7fea86ffd700 10 client.25801 handle_command_reply: tid=0 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.137+0000 7feac1a8ab80 1 client.25801 shutdown 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 msgr2=0x55eead193540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 0x55eead193540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea98099540 tx=0x7fea980ca1e0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 shutdown_connections 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55eead1f00e0 0x55eead1f24d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eead1a2650 0x55eead196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7fea86ffd700 0 client.25801 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eead187430 0x55eead193540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 --2- 172.21.15.3:0/3195915690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eead187df0 0x55eead193a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 >> 172.21.15.3:0/3195915690 conn(0x55eead0b3950 msgr2=0x55eead18ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 shutdown_connections 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 1 -- 172.21.15.3:0/3195915690 wait complete. 2023-08-11T14:04:34.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.138+0000 7feac1a8ab80 20 client.25801 trim_cache size 0 max 16384 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "acd37ee9-5ed9-4323-8a9d-092246009150": { 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "acd37ee9-5ed9-4323-8a9d-092246009150", 2023-08-11T14:04:34.148 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2023-08-11T14:04:34.149 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:04:34.149 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:04:34.149 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:04:34.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 msgr2=0x7fea900a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 0x7fea900a2410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55eead182400 tx=0x7feaa0009920 comp rx=0 tx=0).stop 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 msgr2=0x7feaa819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b680 tx=0x7fea98005740 comp rx=0 tx=0).stop 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 shutdown_connections 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaa810cf00 0x7feaa81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaa8073380 0x7feaa819cc80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fea9009ff50 0x7fea900a2410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 --2- 172.21.15.3:0/2057784989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa8074960 0x7feaa819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:34.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 >> 172.21.15.3:0/2057784989 conn(0x7feaa80fbea0 msgr2=0x7feaa80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:34.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 shutdown_connections 2023-08-11T14:04:34.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:04:34.139+0000 7feaae1ff700 1 -- 172.21.15.3:0/2057784989 wait complete. 2023-08-11T14:04:34.158 INFO:tasks.cephfs.filesystem:scrub status for tag:acd37ee9-5ed9-4323-8a9d-092246009150 - {'path': '/', 'tag': 'acd37ee9-5ed9-4323-8a9d-092246009150', 'options': 'recursive,force'} 2023-08-11T14:04:34.221 INFO:teuthology.orchestra.run.smithi003.stderr:7100000 of 50000000 tuples (14%) done (elapsed 19.72 s, remaining 119.16 s) 2023-08-11T14:04:34.302 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.834+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388/ auth v4 ap=1 f() n(v0 rc2023-08-11T14:02:00.330718+0000 1=0+1) (inest lock) (iversion lock) | dirfrag=0 replicated=0 authpin=1 scrubqueue=0 0x55efd0fc8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000003:[<0x10000000002/a00788a1-1990-44f3-ba7e-f12ca9c8c388 v4>,<0x10000000001/sv_0 v34>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000003:[<0x10000000002/a00788a1-1990-44f3-ba7e-f12ca9c8c388 v4>,<0x10000000001/sv_0 v34>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 rc2023-08-11T14:02:00.330718+0000 1=0+1)","error_str":""},"return_code":0} 2023-08-11T14:04:34.302 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.834+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [2,head] /volumes/_nogroup/sv_0/.meta auth v14 ap=1 s=123 n(v0 rc2023-08-11T14:02:00.339691+0000 b123 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55efd10ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000004:[<0x10000000002/.meta v14>,<0x10000000001/sv_0 v34>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000004:[<0x10000000002/.meta v14>,<0x10000000001/sv_0 v34>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.302 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.838+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001c [...2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={15426=pAsLsXsFsx/pAsLsXsFsx@2},l=15426 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efcffc7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001c:[<0x1000000001b/snapshots v2>,<0x10000000006/pg_logical v356>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x1000000001c:[<0x1000000001b/snapshots v2>,<0x10000000006/pg_logical v1433>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2023-08-11T14:04:34.302 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.838+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e9 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat auth v4 ap=1 s=0 n(v0 rc2023-08-11T14:04:14.032501+0000 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=15426 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e9:[<0x10000000019/global.stat v4>,<0x10000000006/pg_stat_tmp v504>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005e9:[<0x10000000019/global.stat v4>,<0x10000000006/pg_stat_tmp v504>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.838+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001d [...2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings/ auth v4 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={15426=pAsLsXsFsx/pAsLsXsFsx@2},l=15426 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001d:[<0x1000000001b/mappings v4>,<0x10000000006/pg_logical v356>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x1000000001d:[<0x1000000001b/mappings v4>,<0x10000000006/pg_logical v1433>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.839+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e5 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000 auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e5:[<0x1000000000c/0000 v20>,<0x10000000006/pg_notify v484>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005e5:[<0x1000000000c/0000 v20>,<0x10000000006/pg_notify v484>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.842+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000012 [...2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/ auth{2=1,3=1,4=1} v12 ap=1 f(v0 m2023-08-11T14:04:03.637631+0000 1=1+0) n(v0 rc2023-08-11T14:04:03.638631+0000 b8192 2=1+1) (inest lock) (ifile excl) (iversion lock) caps={15426=pAsLsXsFsx/pAsLsXsFsx@1},l=15426 | importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 authpin=1 discoverbase=0 scrubqueue=0 0x55efd10c6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000012:[<0x10000000011/members v12>,<0x10000000006/pg_multixact v362>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000012:[<0x10000000011/members v12>,<0x10000000006/pg_multixact v1062>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-08-11T14:04:03.637631+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-08-11T14:04:03.638631+0000 b8192 2=1+1)","memory_value.dirstat":"f(v0 m2023-08-11T14:04:03.637631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2023-08-11T14:04:03.638631+0000 b8192 2=1+1)","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.842+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000013 [...2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/ auth{2=1,3=1,4=1} v17 ap=1 f(v0 m2023-08-11T14:04:03.636631+0000 1=1+0) n(v0 rc2023-08-11T14:04:24.056375+0000 b8192 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={15426=pAsLsXsFsx/pAsLsXsFsx@1},l=15426 | importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 authpin=1 discoverbase=0 scrubqueue=0 0x55efd10c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000013:[<0x10000000011/offsets v10>,<0x10000000006/pg_multixact v362>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000013:[<0x10000000011/offsets v17>,<0x10000000006/pg_multixact v1062>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-08-11T14:04:03.636631+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-08-11T14:04:24.056375+0000 b8192 2=1+1)","memory_value.dirstat":"f(v0 m2023-08-11T14:04:03.636631+0000 1=1+0)","memory_value.rstat":"n(v0 rc2023-08-11T14:04:24.056375+0000 b8192 2=1+1)","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.842+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000 auth v8 ap=1 s=8192 n(v0 rc2023-08-11T14:04:24.056375+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0d1a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000226:[<0x10000000013/0000 v4>,<0x10000000011/offsets v10>,<0x10000000006/pg_multixact v362>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000226:[<0x10000000013/0000 v8>,<0x10000000011/offsets v17>,<0x10000000006/pg_multixact v1062>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.303 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.844+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000 auth v4 ap=1 s=8192 n(v0 rc2023-08-11T14:04:03.638631+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd111b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000227:[<0x10000000012/0000 v4>,<0x10000000011/members v12>,<0x10000000006/pg_multixact v362>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000227:[<0x10000000012/0000 v4>,<0x10000000011/members v12>,<0x10000000006/pg_multixact v1062>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.855+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000 auth v8 ap=1 s=8192 n(v0 rc2023-08-11T14:04:24.054375+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000224:[<0x1000000001a/0000 v4>,<0x10000000006/pg_xact v296>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000224:[<0x1000000001a/0000 v8>,<0x10000000006/pg_xact v1060>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.858+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e7 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e7:[<0x100000005e3/postgresql-Fri.log v2>,<0x10000000006/log v488>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005e7:[<0x100000005e3/postgresql-Fri.log v2>,<0x10000000006/log v488>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.860+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000 auth v6 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.055613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd112e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000225:[<0x1000000000f/0000 v4>,<0x10000000006/pg_subtrans v280>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000225:[<0x1000000000f/0000 v6>,<0x10000000006/pg_subtrans v400>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.861+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000027d [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100 auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000027d:[<0x10000000008/6100 v32>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000027d:[<0x10000000008/6100 v32>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.861+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000034a [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm auth v106 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000034a:[<0x10000000008/2966_vm v106>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000034a:[<0x10000000008/2966_vm v106>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.861+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000325 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm auth v186 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.058613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000325:[<0x10000000008/1213_vm v90>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000325:[<0x10000000008/1213_vm v186>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.304 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.861+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002fb [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593 auth v122 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.031626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002fb:[<0x10000000008/3593 v122>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002fb:[<0x10000000008/3593 v122>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.861+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control auth v262 ap=1 s=8192 n(v0 rc2023-08-11T14:04:32.448269+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15426 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd10c8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000223:[<0x10000000008/pg_control v120>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000223:[<0x10000000008/pg_control v262>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.862+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000297 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061 auth v172 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.134625+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000297:[<0x10000000008/4061 v172>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000297:[<0x10000000008/4061 v172>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.862+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002bb [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397 auth v190 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.601519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002bb:[<0x10000000008/2397 v158>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002bb:[<0x10000000008/2397 v190>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.862+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000295 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967 auth v136 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.135625+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000295:[<0x10000000008/2967 v136>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000295:[<0x10000000008/2967 v136>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.862+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025e [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396 auth v192 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.601519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10c9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025e:[<0x10000000008/2396 v166>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000025e:[<0x10000000008/2396 v192>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.863+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000385 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm auth v194 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.063613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10cf080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000385:[<0x10000000008/2396_fsm v184>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000385:[<0x10000000008/2396_fsm v194>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.305 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.863+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002da [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233 auth v196 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.059613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10cf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002da:[<0x10000000008/1233 v156>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002da:[<0x10000000008/1233 v196>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.863+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d3 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137 auth v198 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.057613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10cfb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d3:[<0x10000000008/1137 v134>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002d3:[<0x10000000008/1137 v198>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.863+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a7 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695 auth v200 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.065613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10d0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002a7:[<0x10000000008/2695 v170>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002a7:[<0x10000000008/2695 v200>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.863+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000342 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm auth v202 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.060613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10d0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000342:[<0x10000000008/1260_vm v102>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000342:[<0x10000000008/1260_vm v202>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.864+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000327 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm auth v204 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.059613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000327:[<0x10000000008/1214_vm v92>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000327:[<0x10000000008/1214_vm v204>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.864+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002f9 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965 auth v140 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.034626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10d1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002f9:[<0x10000000008/2965 v140>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002f9:[<0x10000000008/2965 v140>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.864+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b7 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672 auth v206 ap=1 s=16384 n(v0 rc2023-08-11T14:04:07.326585+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd10d1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002b7:[<0x10000000008/2672 v142>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002b7:[<0x10000000008/2672 v206>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.306 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.864+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000310 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm auth v84 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000310:[<0x10000000008/6000_vm v84>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000310:[<0x10000000008/6000_vm v84>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.865+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000030c [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm auth v82 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000030c:[<0x10000000008/6100_vm v82>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000030c:[<0x10000000008/6100_vm v82>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.865+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000256 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213 auth v208 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.057613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000256:[<0x10000000008/1213 v148>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000256:[<0x10000000008/1213 v208>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.865+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000273 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592 auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000273:[<0x10000000008/3592 v30>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000273:[<0x10000000008/3592 v30>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.865+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002b6 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671 auth v210 ap=1 s=16384 n(v0 rc2023-08-11T14:04:07.326585+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002b6:[<0x10000000008/2671 v146>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002b6:[<0x10000000008/2671 v210>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.865+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000258 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136 auth v212 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.056613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000258:[<0x10000000008/1136 v162>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000258:[<0x10000000008/1136 v212>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000293 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847 auth v144 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.137625+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000293:[<0x10000000008/2847 v144>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000293:[<0x10000000008/2847 v144>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.307 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000301 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001 auth v176 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.025626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000301:[<0x10000000008/6001 v176>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000301:[<0x10000000008/6001 v176>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000259 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm auth v214 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.057613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000259:[<0x10000000008/1136_fsm v16>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000259:[<0x10000000008/1136_fsm v214>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000254 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm auth v216 ap=1 s=24576 n(v0 rc2023-08-11T14:04:07.326585+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000254:[<0x10000000008/1262_fsm v6>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000254:[<0x10000000008/1262_fsm v216>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000296 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060 auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd0fb3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000296:[<0x10000000008/4060 v42>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000296:[<0x10000000008/4060 v42>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000034b [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm auth v108 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1102000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000034b:[<0x10000000008/2846_vm v108>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000034b:[<0x10000000008/2846_vm v108>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.866+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000292 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846 auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1102580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000292:[<0x10000000008/2846 v34>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000292:[<0x10000000008/2846 v34>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.308 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000253 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262 auth v218 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.601519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1102b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000253:[<0x10000000008/1262 v152>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000253:[<0x10000000008/1262 v218>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000383 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm auth v220 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.061613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1103080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000383:[<0x10000000008/1261_fsm v180>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000383:[<0x10000000008/1261_fsm v220>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000338 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm auth v222 ap=1 s=8192 n(v0 rc2023-08-11T14:04:07.326585+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1103600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000338:[<0x10000000008/2396_vm v96>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000338:[<0x10000000008/2396_vm v222>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dd [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698 auth v224 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.066613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1103b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dd:[<0x10000000008/2698 v128>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002dd:[<0x10000000008/2698 v224>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000309 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115 auth v126 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.016626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1104100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000309:[<0x10000000008/6115 v126>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000309:[<0x10000000008/6115 v126>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002dc [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697 auth v226 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.066613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1104680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002dc:[<0x10000000008/2697 v130>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002dc:[<0x10000000008/2697 v226>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.867+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000255 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964 auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1104c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000255:[<0x10000000008/2964 v8>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000255:[<0x10000000008/2964 v8>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.309 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000341 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm auth v228 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.061613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1105180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000341:[<0x10000000008/1261_vm v100>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000341:[<0x10000000008/1261_vm v228>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000314 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm auth v86 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1105700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000314:[<0x10000000008/3592_vm v86>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000314:[<0x10000000008/3592_vm v86>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000308 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114 auth v150 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.018626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000308:[<0x10000000008/6114 v150>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000308:[<0x10000000008/6114 v150>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000032b [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm auth v230 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.057613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000032b:[<0x10000000008/1136_vm v94>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000032b:[<0x10000000008/1136_vm v230>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000316 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm auth v88 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000316:[<0x10000000008/2964_vm v88>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000316:[<0x10000000008/2964_vm v88>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000294 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966 auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000294:[<0x10000000008/2966 v38>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000294:[<0x10000000008/2966 v38>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.868+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025b [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm auth v232 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.060613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025b:[<0x10000000008/1260_fsm v20>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000025b:[<0x10000000008/1260_fsm v232>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.310 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.869+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000349 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm auth v104 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000349:[<0x10000000008/4060_vm v104>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000349:[<0x10000000008/4060_vm v104>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.869+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000033b [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm auth v234 ap=1 s=8192 n(v0 rc2023-08-11T14:04:07.326585+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000033b:[<0x10000000008/1262_vm v98>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000033b:[<0x10000000008/1262_vm v234>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ea [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init auth v258 ap=1 s=0 n(v0 rc2023-08-11T14:04:14.148499+0000 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=15426 | importingcaps=0 caps=1 dirtyrstat=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ea:[<0x10000000008/pg_internal.init v258>,<0x10000000006/global v512>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ea:[<0x10000000008/pg_internal.init v258>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025c [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261 auth v236 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.061613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025c:[<0x10000000008/1261 v138>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000025c:[<0x10000000008/1261 v236>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000384 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm auth v238 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.059613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd110f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000384:[<0x10000000008/1214_fsm v182>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000384:[<0x10000000008/1214_fsm v238>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a5 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677 auth v242 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.064613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1118580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002a5:[<0x10000000008/2677 v154>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002a5:[<0x10000000008/2677 v242>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025d [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214 auth v240 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.058613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1118000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025d:[<0x10000000008/1214 v174>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000025d:[<0x10000000008/1214 v240>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.311 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.870+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000355 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map auth v160 ap=1 s=512 n(v0 rc2023-08-11T14:04:04.153625+0000 b512 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1118b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000355:[<0x10000000008/pg_filenode.map v160>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000355:[<0x10000000008/pg_filenode.map v160>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a6 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694 auth v244 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.065613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1119080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002a6:[<0x10000000008/2694 v168>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002a6:[<0x10000000008/2694 v244>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002d9 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232 auth v246 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.059613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1119600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002d9:[<0x10000000008/1232 v132>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002d9:[<0x10000000008/1232 v246>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a4 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676 auth v248 ap=1 s=16384 n(v0 rc2023-08-11T14:04:05.064613+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1119b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000002a4:[<0x10000000008/2676 v178>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x100000002a4:[<0x10000000008/2676 v248>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000302 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002 auth v164 ap=1 s=8192 n(v0 rc2023-08-11T14:04:04.023626+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd111a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000302:[<0x10000000008/6002 v164>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000302:[<0x10000000008/6002 v164>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000257 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm auth v250 ap=1 s=24576 n(v0 rc2023-08-11T14:04:05.058613+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd111a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000257:[<0x10000000008/1213_fsm v12>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x10000000257:[<0x10000000008/1213_fsm v250>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000025a [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260 auth v252 ap=1 s=8192 n(v0 rc2023-08-11T14:04:05.060613+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd111ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000025a:[<0x10000000008/1260 v124>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000025a:[<0x10000000008/1260 v252>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.312 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000026f [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000 auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd111b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000026f:[<0x10000000008/6000 v28>,<0x10000000006/global v326>,<0x10000000005/data v79>,<0x100000001fb/postgres v15>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v31>,<0x10000000001/sv_1 v51>,<0x10000000000/_nogroup v34>,<0x1/volumes v32>]//[3]","memoryvalue":"(4)0x1000000026f:[<0x10000000008/6000 v28>,<0x10000000006/global v512>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.871+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000592 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm auth v392 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.515520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1170680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000592:[<0x100000004bb/3603_vm v392>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000592:[<0x100000004bb/3603_vm v392>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000593 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575 auth v394 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.516520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1170c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000593:[<0x100000004bb/3575 v394>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000593:[<0x100000004bb/3575 v394>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f1 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm auth v478 ap=1 s=24576 n(v0 rc2023-08-11T14:04:14.220498+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd1171180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f1:[<0x100000004bb/16387_fsm v476>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005f1:[<0x100000004bb/16387_fsm v478>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000594 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467 auth v396 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.517520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd1171700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000594:[<0x100000004bb/3467 v396>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000594:[<0x100000004bb/3467 v396>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ef [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393 auth v490 ap=1 s=24576 n(v0 rc2023-08-11T14:04:24.361371+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd119a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ef:[<0x100000004bb/16393 v474>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ef:[<0x100000004bb/16393 v490>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000595 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm auth v398 ap=1 s=24576 n(v0 rc2023-08-11T14:04:12.518520+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000595:[<0x100000004bb/2617_fsm v398>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000595:[<0x100000004bb/2617_fsm v398>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.313 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.872+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000596 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119 auth v400 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.519520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000596:[<0x100000004bb/3119 v400>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000596:[<0x100000004bb/3119 v400>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.873+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000597 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674 auth v480 ap=1 s=385024 n(v0 rc2023-08-11T14:04:24.260372+0000 b385024 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd119b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000597:[<0x100000004bb/2674 v402>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000597:[<0x100000004bb/2674 v480>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.873+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000599 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm auth v406 ap=1 s=24576 n(v0 rc2023-08-11T14:04:12.523520+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000599:[<0x100000004bb/2603_fsm v406>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000599:[<0x100000004bb/2603_fsm v406>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.873+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000598 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668 auth v404 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.522520+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000598:[<0x100000004bb/2668 v404>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x10000000598:[<0x100000004bb/2668 v404>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.873+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059b [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702 auth v410 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.525520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059b:[<0x100000004bb/2702 v410>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059b:[<0x100000004bb/2702 v410>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.874+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059e [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712 auth v416 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.528520+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059e:[<0x100000004bb/3712 v416>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059e:[<0x100000004bb/3712 v416>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.874+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059c [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655 auth v412 ap=1 s=32768 n(v0 rc2023-08-11T14:04:12.526520+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059c:[<0x100000004bb/2655 v412>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059c:[<0x100000004bb/2655 v412>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.314 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.874+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059f [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608 auth v482 ap=1 s=442368 n(v0 rc2023-08-11T14:04:24.260372+0000 b442368 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd119e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059f:[<0x100000004bb/2608 v418>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059f:[<0x100000004bb/2608 v482>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.874+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a0 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831 auth v420 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.531520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a0:[<0x100000004bb/2831 v420>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a0:[<0x100000004bb/2831 v420>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.875+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a1 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm auth v422 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.532520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a1:[<0x100000004bb/12187_vm v422>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a1:[<0x100000004bb/12187_vm v422>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.875+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a2 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164 auth v424 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.533520+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a2:[<0x100000004bb/3164 v424>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a2:[<0x100000004bb/3164 v424>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.876+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a4 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111 auth v426 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.535519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a4:[<0x100000004bb/6111 v426>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a4:[<0x100000004bb/6111 v426>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.877+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a5 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682 auth v428 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.536519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a5:[<0x100000004bb/2682 v428>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a5:[<0x100000004bb/2682 v428>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.877+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059a [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110 auth v408 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.524520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059a:[<0x100000004bb/6110 v408>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059a:[<0x100000004bb/6110 v408>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.315 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.877+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059d [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597 auth v414 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.527520+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059d:[<0x100000004bb/3597 v414>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x1000000059d:[<0x100000004bb/3597 v414>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.877+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a6 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113 auth v430 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.537519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a6:[<0x100000004bb/113 v430>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a6:[<0x100000004bb/113 v430>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.878+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a7 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604 auth v432 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.538519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a7:[<0x100000004bb/3604 v432>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a7:[<0x100000004bb/3604 v432>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.878+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a3 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm auth v338 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd119f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a3:[<0x100000004bb/3501_vm v338>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a3:[<0x100000004bb/3501_vm v338>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.878+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a9 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm auth v436 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.541519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a9:[<0x100000004bb/2605_vm v436>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a9:[<0x100000004bb/2605_vm v436>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.879+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ab [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685 auth v440 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.543519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ab:[<0x100000004bb/2685 v440>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ab:[<0x100000004bb/2685 v440>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.880+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005eb [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init auth v472 ap=1 s=0 n(v0 rc2023-08-11T14:04:14.152499+0000 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005eb:[<0x100000004bb/pg_internal.init v472>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005eb:[<0x100000004bb/pg_internal.init v472>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.316 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.880+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ad [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838 auth v444 ap=1 s=417792 n(v0 rc2023-08-11T14:04:12.546519+0000 b417792 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ad:[<0x100000004bb/2838 v444>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ad:[<0x100000004bb/2838 v444>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.881+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ac [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663 auth v488 ap=1 s=49152 n(v0 rc2023-08-11T14:04:24.260372+0000 b49152 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd11a8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ac:[<0x100000004bb/2663 v442>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ac:[<0x100000004bb/2663 v488>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.882+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005af [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548 auth v446 ap=1 s=8192 n(v0 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11aa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005af:[<0x100000004bb/548 v446>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005af:[<0x100000004bb/548 v446>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.882+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b1 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603 auth v450 ap=1 s=32768 n(v0 rc2023-08-11T14:04:12.550519+0000 b32768 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11aac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b1:[<0x100000004bb/2603 v450>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b1:[<0x100000004bb/2603 v450>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.883+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a8 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691 auth v434 ap=1 s=270336 n(v0 rc2023-08-11T14:04:12.540519+0000 b270336 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a8:[<0x100000004bb/2691 v434>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005a8:[<0x100000004bb/2691 v434>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.883+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005aa [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440 auth v438 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.542519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005aa:[<0x100000004bb/3440 v438>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005aa:[<0x100000004bb/3440 v438>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.883+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b2 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764 auth v452 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.551519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11ab180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b2:[<0x100000004bb/3764 v452>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b2:[<0x100000004bb/3764 v452>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.317 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.884+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b3 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080 auth v454 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.552519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11ab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b3:[<0x100000004bb/3080 v454>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b3:[<0x100000004bb/3080 v454>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.884+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b0 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247 auth v486 ap=1 s=73728 n(v0 rc2023-08-11T14:04:24.159374+0000 b73728 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd11aa680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b0:[<0x100000004bb/1247 v448>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b0:[<0x100000004bb/1247 v486>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.885+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ae [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836 auth v360 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11a9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ae:[<0x100000004bb/2836 v360>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ae:[<0x100000004bb/2836 v360>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.885+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b4 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm auth v456 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.553519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b4:[<0x100000004bb/3394_vm v456>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b4:[<0x100000004bb/3394_vm v456>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.885+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b5 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754 auth v458 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.554519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b5:[<0x100000004bb/2754 v458>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b5:[<0x100000004bb/2754 v458>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.885+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b6 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199 auth v376 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b6:[<0x100000004bb/12199 v376>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b6:[<0x100000004bb/12199 v376>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.885+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b7 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455 auth v484 ap=1 s=40960 n(v0 rc2023-08-11T14:04:24.260372+0000 b40960 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd11b3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b7:[<0x100000004bb/3455 v460>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b7:[<0x100000004bb/3455 v484>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.318 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b8 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm auth v462 ap=1 s=24576 n(v0 rc2023-08-11T14:04:12.557519+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b8:[<0x100000004bb/12182_fsm v462>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b8:[<0x100000004bb/12182_fsm v462>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.319 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b9 [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259 auth v492 ap=1 s=90112 n(v0 rc2023-08-11T14:04:24.259372+0000 b90112 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55efd11b3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b9:[<0x100000004bb/1259 v464>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005b9:[<0x100000004bb/1259 v492>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.319 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ba [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207 auth v466 ap=1 s=8192 n(v0 rc2023-08-11T14:04:12.559519+0000 b8192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ba:[<0x100000004bb/12207 v466>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005ba:[<0x100000004bb/12207 v466>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.319 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005bb [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610 auth v468 ap=1 s=24576 n(v0 rc2023-08-11T14:04:12.560519+0000 b24576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005bb:[<0x100000004bb/2610 v468>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005bb:[<0x100000004bb/2610 v468>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.320 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005bc [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606 auth v470 ap=1 s=16384 n(v0 rc2023-08-11T14:04:12.561519+0000 b16384 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005bc:[<0x100000004bb/3606 v470>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005bc:[<0x100000004bb/3606 v470>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.320 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:04:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:04:33.886+0000 7f4f5b8fd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005bd [2,head] /volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613 auth v390 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15426=0-4194304@1} caps={15426=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15426 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55efd11b5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005bd:[<0x100000004bb/2613 v390>,<0x10000000014/12347 v3452>,<0x10000000006/base v990>,<0x10000000005/data v134>,<0x100000001fb/postgres v45>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v59>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v58>,<0x1/volumes v57>]//[3]","memoryvalue":"(4)0x100000005bd:[<0x100000004bb/2613 v390>,<0x10000000014/12347 v3502>,<0x10000000006/base v942>,<0x10000000005/data v150>,<0x100000001fb/postgres v61>,<0x100000001fa/62cd9d2c-a60b-4f8a-ba60-005c555889b1 v75>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v60>,<0x1/volumes v20>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-08-11T14:04:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:34 smithi003 ceph-mon[97195]: scrub queued for path: / 2023-08-11T14:04:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:34 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:34 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:34 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:7200000 of 50000000 tuples (14%) done (elapsed 20.07 s, remaining 119.28 s) 2023-08-11T14:04:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:34 smithi103 ceph-mon[130434]: scrub queued for path: / 2023-08-11T14:04:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:34 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:34 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:34 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:34 smithi187 ceph-mon[130973]: scrub queued for path: / 2023-08-11T14:04:34.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:34 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:34 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:34.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:34 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:7300000 of 50000000 tuples (14%) done (elapsed 20.39 s, remaining 119.27 s) 2023-08-11T14:04:35.007 INFO:teuthology.orchestra.run.smithi003.stderr:7400000 of 50000000 tuples (14%) done (elapsed 20.51 s, remaining 118.06 s) 2023-08-11T14:04:35.341 INFO:teuthology.orchestra.run.smithi003.stderr:7500000 of 50000000 tuples (15%) done (elapsed 20.84 s, remaining 118.10 s) 2023-08-11T14:04:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:7600000 of 50000000 tuples (15%) done (elapsed 20.97 s, remaining 116.97 s) 2023-08-11T14:04:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000003 (/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000004 (/volumes/_nogroup/sv_0/.meta) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.682 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.683 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.683 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.685 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.686 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.687 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000027d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.688 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000034a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.688 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000325 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.688 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002fb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.688 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.688 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000297 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000295 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000025e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002da (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002d3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.689 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000342 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000327 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002f9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000310 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000030c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.690 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000256 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000273 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000258 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000293 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000301 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.691 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000259 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000254 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000296 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000034b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000292 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000253 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.692 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000338 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002dd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000309 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002dc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000255 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.693 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000341 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000314 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000308 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000032b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000316 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000294 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000025b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.694 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000349 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000033b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.695 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000025c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000025d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000355 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.696 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002d9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000002a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000302 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000257 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000025a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000026f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.697 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000593 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000594 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000595 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000596 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000597 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.698 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000599 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x1000000059d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ab (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ad (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005af (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005a8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.702 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ae (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.703 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005ba (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005bc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.704 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: active paths [/] 2023-08-11T14:04:35.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.705 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: pgmap v367: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 112 MiB/s wr, 140 op/s 2023-08-11T14:04:35.706 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:35 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:7700000 of 50000000 tuples (15%) done (elapsed 21.28 s, remaining 116.88 s) 2023-08-11T14:04:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000003 (/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000004 (/volumes/_nogroup/sv_0/.meta) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000027d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000034a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000325 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002fb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000297 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000295 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000025e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002da (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002d3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000342 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000327 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002f9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000310 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000030c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000256 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000273 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000258 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000293 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000301 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000259 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000254 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000296 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000034b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000292 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000253 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000338 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002dd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000309 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002dc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000255 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000341 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000314 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000308 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000032b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000316 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000294 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000025b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000349 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000033b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000025c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000025d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000355 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002d9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000002a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000302 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000257 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000025a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000026f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000593 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000594 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000595 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000596 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000597 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000599 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.810 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.813 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x1000000059d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ab (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.814 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ad (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005af (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005a8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.815 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.816 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.816 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ae (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.816 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005ba (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.818 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005bc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.818 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.818 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.818 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: active paths [/] 2023-08-11T14:04:35.819 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: pgmap v367: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 112 MiB/s wr, 140 op/s 2023-08-11T14:04:35.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:35 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:35.822 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.822 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000003 (/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.822 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000004 (/volumes/_nogroup/sv_0/.meta) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.822 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.822 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.823 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.823 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.823 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.823 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.825 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.825 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.825 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.825 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.826 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000226 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.826 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000227 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.826 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.826 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.827 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.827 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.827 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.827 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.828 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.828 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.828 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.828 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.828 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000224 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.829 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000027d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000034a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000325 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002fb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000223 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000297 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.830 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000295 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000025e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002da (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002d3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000342 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.831 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000327 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002f9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000310 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000030c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000256 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000273 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.832 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000258 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000293 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000301 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000259 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000254 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000296 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.833 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000034b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000292 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000253 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000338 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002dd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000309 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002dc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.834 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000255 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000341 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000314 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000308 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000032b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000316 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000294 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000025b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000349 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000033b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000025c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000025d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000355 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002d9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000002a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000302 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000257 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.837 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000025a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000026f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000593 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000594 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.838 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000595 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000596 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000597 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000599 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.839 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059c (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059f (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059a (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x1000000059d (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ab (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.841 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ad (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.842 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.842 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005af (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.842 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b1 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.842 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005a8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.843 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005aa (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.843 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b2 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.843 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b3 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.843 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.844 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ae (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.844 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.844 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b5 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.845 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b6 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.845 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.845 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b8 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.845 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.845 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005ba (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005bb (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005bc (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613) see mds.i log and `damage ls` output for details 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: active paths [/] 2023-08-11T14:04:35.846 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: scrub summary: idle+waiting paths [/] 2023-08-11T14:04:35.847 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: pgmap v367: 129 pgs: 129 active+clean; 1.2 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 112 MiB/s wr, 140 op/s 2023-08-11T14:04:35.847 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:35 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:36.079 INFO:teuthology.orchestra.run.smithi003.stderr:7800000 of 50000000 tuples (15%) done (elapsed 21.58 s, remaining 116.75 s) 2023-08-11T14:04:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:7900000 of 50000000 tuples (15%) done (elapsed 21.73 s, remaining 115.78 s) 2023-08-11T14:04:36.491 INFO:teuthology.orchestra.run.smithi003.stderr:8000000 of 50000000 tuples (16%) done (elapsed 21.99 s, remaining 115.45 s) 2023-08-11T14:04:36.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:36 smithi103 ceph-mon[130434]: pgmap v368: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 123 MiB/s wr, 155 op/s 2023-08-11T14:04:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:8100000 of 50000000 tuples (16%) done (elapsed 22.20 s, remaining 114.83 s) 2023-08-11T14:04:36.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:36 smithi003 ceph-mon[97195]: pgmap v368: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 123 MiB/s wr, 155 op/s 2023-08-11T14:04:36.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:36 smithi187 ceph-mon[130973]: pgmap v368: 129 pgs: 129 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 123 MiB/s wr, 155 op/s 2023-08-11T14:04:37.000 INFO:teuthology.orchestra.run.smithi003.stderr:8200000 of 50000000 tuples (16%) done (elapsed 22.50 s, remaining 114.70 s) 2023-08-11T14:04:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:8300000 of 50000000 tuples (16%) done (elapsed 22.82 s, remaining 114.65 s) 2023-08-11T14:04:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:8400000 of 50000000 tuples (16%) done (elapsed 23.00 s, remaining 113.89 s) 2023-08-11T14:04:37.793 INFO:teuthology.orchestra.run.smithi003.stderr:8500000 of 50000000 tuples (17%) done (elapsed 23.29 s, remaining 113.72 s) 2023-08-11T14:04:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:8600000 of 50000000 tuples (17%) done (elapsed 23.62 s, remaining 113.70 s) 2023-08-11T14:04:38.292 INFO:teuthology.orchestra.run.smithi003.stderr:8700000 of 50000000 tuples (17%) done (elapsed 23.79 s, remaining 112.94 s) 2023-08-11T14:04:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:8800000 of 50000000 tuples (17%) done (elapsed 24.06 s, remaining 112.62 s) 2023-08-11T14:04:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:38 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:38.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:38 smithi103 ceph-mon[130434]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 134 MiB/s wr, 216 op/s 2023-08-11T14:04:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:38 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:38 smithi003 ceph-mon[97195]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 134 MiB/s wr, 216 op/s 2023-08-11T14:04:38.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:38 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:38.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:38 smithi187 ceph-mon[130973]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 134 MiB/s wr, 216 op/s 2023-08-11T14:04:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:8900000 of 50000000 tuples (17%) done (elapsed 24.35 s, remaining 112.46 s) 2023-08-11T14:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 -- 172.21.15.3:0/541609512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 msgr2=0x7f373410b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 --2- 172.21.15.3:0/541609512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f373410b750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f37300099c0 tx=0x7f3730009cd0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 -- 172.21.15.3:0/541609512 shutdown_connections 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 --2- 172.21.15.3:0/541609512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 0x7f3734113770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 --2- 172.21.15.3:0/541609512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f373410b750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 --2- 172.21.15.3:0/541609512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f373410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.960+0000 7f373c5ff700 1 -- 172.21.15.3:0/541609512 >> 172.21.15.3:0/541609512 conn(0x7f373406aff0 msgr2=0x7f373406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.961+0000 7f373c5ff700 1 -- 172.21.15.3:0/541609512 shutdown_connections 2023-08-11T14:04:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.961+0000 7f373c5ff700 1 -- 172.21.15.3:0/541609512 wait complete. 2023-08-11T14:04:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 Processor -- start 2023-08-11T14:04:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 -- start start 2023-08-11T14:04:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f37341a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 0x7f37341a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341a2b40 con 0x7f373410c520 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341a2cb0 con 0x7f373410b330 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373c5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341a2e70 con 0x7f373410cec0 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373a39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373a39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53596/0 (socket says 172.21.15.3:53596) 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.962+0000 7f373a39b700 1 -- 172.21.15.3:0/1185064690 learned_addr learned my addr 172.21.15.3:0/1185064690 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f37341a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373ab9c700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 0x7f37341a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 -- 172.21.15.3:0/1185064690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 msgr2=0x7f37341a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 0x7f37341a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 -- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 msgr2=0x7f37341a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f37341a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 -- 172.21.15.3:0/1185064690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300095f0 con 0x7f373410b330 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f37341a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.963+0000 7f373a39b700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3730009990 tx=0x7f373000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.964+0000 7f372b7fe700 1 -- 172.21.15.3:0/1185064690 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373001d070 con 0x7f373410b330 2023-08-11T14:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.964+0000 7f372b7fe700 1 -- 172.21.15.3:0/1185064690 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37300054f0 con 0x7f373410b330 2023-08-11T14:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.964+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37341a30f0 con 0x7f373410b330 2023-08-11T14:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.964+0000 7f372b7fe700 1 -- 172.21.15.3:0/1185064690 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730004b00 con 0x7f373410b330 2023-08-11T14:04:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.964+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37341a9910 con 0x7f373410b330 2023-08-11T14:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.968+0000 7f374fed2b80 1 Processor -- start 2023-08-11T14:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 -- start start 2023-08-11T14:04:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f372b7fe700 1 -- 172.21.15.3:0/1185064690 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f373000fb80 con 0x7f373410b330 2023-08-11T14:04:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59ce6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59ce71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59c3f5d0 con 0x563e59ce68a0 2023-08-11T14:04:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59c3cef0 con 0x563e59ce7790 2023-08-11T14:04:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.969+0000 7f374fed2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59c387e0 con 0x563e59ce5f40 2023-08-11T14:04:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53608/0 (socket says 172.21.15.3:53608) 2023-08-11T14:04:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 -- 172.21.15.3:0/2476536752 learned_addr learned my addr 172.21.15.3:0/2476536752 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f373a39b700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59ce71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f373ab9c700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59ce6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 -- 172.21.15.3:0/2476536752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 msgr2=0x563e59ce6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59ce6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 -- 172.21.15.3:0/2476536752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 msgr2=0x563e59ce71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59ce71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.970+0000 7f3739b9a700 1 -- 172.21.15.3:0/2476536752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e59ab85d0 con 0x563e59ce7790 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f373ab9c700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59ce6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f3739b9a700 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3724009de0 tx=0x7f37240116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab54f227fea751 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f3728bfe700 1 -- 172.21.15.3:0/2476536752 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724011e20 con 0x563e59ce7790 2023-08-11T14:04:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f3728bfe700 1 -- 172.21.15.3:0/2476536752 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724012910 con 0x563e59ce7790 2023-08-11T14:04:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f3728bfe700 1 -- 172.21.15.3:0/2476536752 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400f860 con 0x563e59ce7790 2023-08-11T14:04:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.971+0000 7f372b7fe700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 0x7f37200a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f372b7fe700 1 -- 172.21.15.3:0/1185064690 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f37300cc280 con 0x7f373410b330 2023-08-11T14:04:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f373ab9c700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 0x7f37200a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 -- 172.21.15.3:0/2476536752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 msgr2=0x563e59ce7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3724009de0 tx=0x7f37240116c0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 -- 172.21.15.3:0/2476536752 shutdown_connections 2023-08-11T14:04:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59ce7c10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59ce6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 --2- 172.21.15.3:0/2476536752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59ce71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f374fed2b80 1 -- 172.21.15.3:0/2476536752 >> 172.21.15.3:0/2476536752 conn(0x563e59c138f0 msgr2=0x563e59ce2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.972+0000 7f373ab9c700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 0x7f37200a2490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f372c009510 tx=0x7f372c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.973+0000 7f374fed2b80 1 -- 172.21.15.3:0/2476536752 shutdown_connections 2023-08-11T14:04:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.973+0000 7f374fed2b80 1 -- 172.21.15.3:0/2476536752 wait complete. 2023-08-11T14:04:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.974+0000 7f374fed2b80 1 -- start start 2023-08-11T14:04:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.974+0000 7f374fed2b80 1 Processor -- start 2023-08-11T14:04:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce5f40 0x563e59cf8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 0x563e59cff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59bd6a60 con 0x563e59ce5f40 2023-08-11T14:04:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59bd6760 con 0x563e59ce68a0 2023-08-11T14:04:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59aba8c0 con 0x563e59ce7790 2023-08-11T14:04:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f374fed2b80 10 client.0 _ll_register_callbacks cb 0x563e59bd9a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f373ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 0x563e59cff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f373a39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce5f40 0x563e59cf8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53620/0 (socket says 172.21.15.3:53620) 2023-08-11T14:04:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f373ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 0x563e59cff990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42414/0 (socket says 172.21.15.3:42414) 2023-08-11T14:04:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 -- 172.21.15.3:0/4056959902 learned_addr learned my addr 172.21.15.3:0/4056959902 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 -- 172.21.15.3:0/4056959902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 msgr2=0x563e59cff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 0x563e59cff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 -- 172.21.15.3:0/4056959902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce5f40 msgr2=0x563e59cf8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce5f40 0x563e59cf8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.976+0000 7f3739b9a700 1 -- 172.21.15.3:0/4056959902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724009a10 con 0x563e59ce68a0 2023-08-11T14:04:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f3739b9a700 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3724011a10 tx=0x7f3724012cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383d6fe9b207d4a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f371dffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724012660 con 0x563e59ce68a0 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e59cd97e0 con 0x563e59ce68a0 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3724004040 con 0x563e59ce68a0 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.977+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3724004200 con 0x563e59ce68a0 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.978+0000 7f374fed2b80 10 client.25805 fetch_fsmap learned FSMap version 30 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.978+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563e59a64d80 con 0x563e59ce68a0 2023-08-11T14:04:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.978+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e59cd97e0 con 0x563e59ce68a0 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f3724022070 con 0x563e59ce68a0 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 30 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 20 client.25805 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59cf3b30 0x563e59cf5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 4 client.25805 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.979+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e59cb5480 con 0x563e59cf3b30 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.980+0000 7f373a39b700 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59cf3b30 0x563e59cf5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.980+0000 7f371dffb700 10 client.25805 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.980+0000 7f373a39b700 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59cf3b30 0x563e59cf5f50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2595e37310b84218 server_cookie=685aab030fb9ab27 in_seq=0 out_seq=0 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f371dffb700 1 -- 172.21.15.3:0/4056959902 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563e59cb5480 con 0x563e59cf3b30 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f371dffb700 10 client.25805 handle_command_reply: tid=0 2023-08-11T14:04:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 client.25805 shutdown 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 msgr2=0x563e59cf8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3724011a10 tx=0x7f3724012cb0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 shutdown_connections 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce7790 0x563e59cff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce5f40 0x563e59cf8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59cf3b30 0x563e59cf5f50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 --2- 172.21.15.3:0/4056959902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce68a0 0x563e59cf8950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 >> 172.21.15.3:0/4056959902 conn(0x563e59c138f0 msgr2=0x563e59ce1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.981+0000 7f371dffb700 0 client.25805 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.982+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 shutdown_connections 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.982+0000 7f374fed2b80 1 -- 172.21.15.3:0/4056959902 wait complete. 2023-08-11T14:04:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.982+0000 7f374fed2b80 20 client.25805 trim_cache size 0 max 16384 2023-08-11T14:04:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 Processor -- start 2023-08-11T14:04:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 -- start start 2023-08-11T14:04:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59cff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59cf78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59ce1f50 con 0x563e59ce68a0 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59cf9090 con 0x563e59ce7790 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.997+0000 7f374fed2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59cee3d0 con 0x563e59ce5f40 2023-08-11T14:04:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59cff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42416/0 (socket says 172.21.15.3:42416) 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 -- 172.21.15.3:0/1751866837 learned_addr learned my addr 172.21.15.3:0/1751866837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373ab9c700 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59cf78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 -- 172.21.15.3:0/1751866837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 msgr2=0x563e59cf78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59cf78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 -- 172.21.15.3:0/1751866837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 msgr2=0x563e59cff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59cff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 -- 172.21.15.3:0/1751866837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e59a4dc00 con 0x563e59ce5f40 2023-08-11T14:04:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59cff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.998+0000 7f373a39b700 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f373000b580 tx=0x7f373009f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17feb70147470533 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.999+0000 7f37293ff700 1 -- 172.21.15.3:0/1751866837 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730099930 con 0x563e59ce5f40 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.999+0000 7f37293ff700 1 -- 172.21.15.3:0/1751866837 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37300996a0 con 0x563e59ce5f40 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:38.999+0000 7f37293ff700 1 -- 172.21.15.3:0/1751866837 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730099bf0 con 0x563e59ce5f40 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- 172.21.15.3:0/1751866837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 msgr2=0x563e59d14f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f373000b580 tx=0x7f373009f040 comp rx=0 tx=0).stop 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- 172.21.15.3:0/1751866837 shutdown_connections 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce7790 0x563e59cf78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce5f40 0x563e59d14f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1751866837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce68a0 0x563e59cff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- 172.21.15.3:0/1751866837 >> 172.21.15.3:0/1751866837 conn(0x563e59c138f0 msgr2=0x563e59cdd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- 172.21.15.3:0/1751866837 shutdown_connections 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- 172.21.15.3:0/1751866837 wait complete. 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.000+0000 7f374fed2b80 1 -- start start 2023-08-11T14:04:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 Processor -- start 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 0x563e59cfbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce7790 0x563e59cfee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59cff3d0 con 0x563e59ce7790 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59cff540 con 0x563e59ce5f40 2023-08-11T14:04:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e59cff700 con 0x563e59ce68a0 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.001+0000 7f374fed2b80 10 client.0 _ll_register_callbacks cb 0x7f37300cfbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 0x563e59cfbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f373a39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f373a39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53638/0 (socket says 172.21.15.3:53638) 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f373a39b700 1 -- 172.21.15.3:0/1031004602 learned_addr learned my addr 172.21.15.3:0/1031004602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f373ab9c700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce7790 0x563e59cfee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 -- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 msgr2=0x563e59cfb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 -- 172.21.15.3:0/1031004602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce7790 msgr2=0x563e59cfee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce7790 0x563e59cfee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 -- 172.21.15.3:0/1031004602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e59a64d80 con 0x563e59ce68a0 2023-08-11T14:04:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f373a39b700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f3739b9a700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 0x563e59cfbe70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f372400a010 tx=0x7f3724011fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e59012b918c856 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.002+0000 7f371dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:04:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724003880 con 0x563e59ce68a0 2023-08-11T14:04:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3724012910 con 0x563e59ce68a0 2023-08-11T14:04:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f37240039e0 con 0x563e59ce68a0 2023-08-11T14:04:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3724005040 con 0x563e59ce68a0 2023-08-11T14:04:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f374fed2b80 10 client.25816 fetch_fsmap learned FSMap version 30 2023-08-11T14:04:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f37300b1e70 con 0x563e59ce68a0 2023-08-11T14:04:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.003+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724012910 con 0x563e59ce68a0 2023-08-11T14:04:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f37240113e0 con 0x563e59ce68a0 2023-08-11T14:04:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 30 2023-08-11T14:04:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 20 client.25816 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59d60ea0 0x563e59d632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 4 client.25816 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:04:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e59c317c0 con 0x563e59d60ea0 2023-08-11T14:04:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f373a39b700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59d60ea0 0x563e59d632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f373a39b700 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59d60ea0 0x563e59d632c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae23ae8718a247d6 server_cookie=bf8ae873f31d5f32 in_seq=0 out_seq=0 2023-08-11T14:04:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.004+0000 7f371dffb700 10 client.25816 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.038+0000 7f371dffb700 1 -- 172.21.15.3:0/1031004602 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e59c317c0 con 0x563e59d60ea0 2023-08-11T14:04:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.038+0000 7f371dffb700 10 client.25816 handle_command_reply: tid=0 2023-08-11T14:04:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.038+0000 7f374fed2b80 1 client.25816 shutdown 2023-08-11T14:04:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.038+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 msgr2=0x563e59cfbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.038+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 0x563e59cfbe70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f372400a010 tx=0x7f3724011fd0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 shutdown_connections 2023-08-11T14:04:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e59ce7790 0x563e59cfee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e59d60ea0 0x563e59d632c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e59ce5f40 0x563e59cfb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 --2- 172.21.15.3:0/1031004602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e59ce68a0 0x563e59cfbe70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f371dffb700 0 client.25816 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 >> 172.21.15.3:0/1031004602 conn(0x563e59c138f0 msgr2=0x563e59cea670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 shutdown_connections 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 1 -- 172.21.15.3:0/1031004602 wait complete. 2023-08-11T14:04:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.039+0000 7f374fed2b80 20 client.25816 trim_cache size 0 max 16384 2023-08-11T14:04:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:04:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:04:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:04:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:04:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 msgr2=0x7f37200a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 0x7f37200a2490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f372c009510 tx=0x7f372c00b040 comp rx=0 tx=0).stop 2023-08-11T14:04:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 msgr2=0x7f37341a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3730009990 tx=0x7f373000f690 comp rx=0 tx=0).stop 2023-08-11T14:04:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 shutdown_connections 2023-08-11T14:04:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f373410cec0 0x7f37341a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f372009ffd0 0x7f37200a2490 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373410b330 0x7f37341a1120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 --2- 172.21.15.3:0/1185064690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373410c520 0x7f37341a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 >> 172.21.15.3:0/1185064690 conn(0x7f373406aff0 msgr2=0x7f373410a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 shutdown_connections 2023-08-11T14:04:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:39.040+0000 7f373c5ff700 1 -- 172.21.15.3:0/1185064690 wait complete. 2023-08-11T14:04:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:9000000 of 50000000 tuples (18%) done (elapsed 24.56 s, remaining 111.88 s) 2023-08-11T14:04:39.327 INFO:teuthology.orchestra.run.smithi003.stderr:9100000 of 50000000 tuples (18%) done (elapsed 24.83 s, remaining 111.58 s) 2023-08-11T14:04:39.636 INFO:teuthology.orchestra.run.smithi003.stderr:9200000 of 50000000 tuples (18%) done (elapsed 25.14 s, remaining 111.47 s) 2023-08-11T14:04:39.811 INFO:teuthology.orchestra.run.smithi003.stderr:9300000 of 50000000 tuples (18%) done (elapsed 25.31 s, remaining 110.77 s) 2023-08-11T14:04:40.111 INFO:teuthology.orchestra.run.smithi003.stderr:9400000 of 50000000 tuples (18%) done (elapsed 25.61 s, remaining 110.62 s) 2023-08-11T14:04:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:9500000 of 50000000 tuples (19%) done (elapsed 25.94 s, remaining 110.58 s) 2023-08-11T14:04:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:9600000 of 50000000 tuples (19%) done (elapsed 26.07 s, remaining 109.70 s) 2023-08-11T14:04:40.914 INFO:teuthology.orchestra.run.smithi003.stderr:9700000 of 50000000 tuples (19%) done (elapsed 26.41 s, remaining 109.74 s) 2023-08-11T14:04:41.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:41 smithi003 ceph-mon[97195]: pgmap v370: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 MiB/s wr, 191 op/s 2023-08-11T14:04:41.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:41 smithi187 ceph-mon[130973]: pgmap v370: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 MiB/s wr, 191 op/s 2023-08-11T14:04:41.657 INFO:teuthology.orchestra.run.smithi003.stderr:9800000 of 50000000 tuples (19%) done (elapsed 27.16 s, remaining 111.40 s) 2023-08-11T14:04:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:41 smithi103 ceph-mon[130434]: pgmap v370: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 125 MiB/s wr, 191 op/s 2023-08-11T14:04:41.814 INFO:teuthology.orchestra.run.smithi003.stderr:9900000 of 50000000 tuples (19%) done (elapsed 27.31 s, remaining 110.64 s) 2023-08-11T14:04:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:42 smithi003 ceph-mon[97195]: scrub summary: idle 2023-08-11T14:04:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:04:42.269+0000 7f7a41abb700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-08-11T14:04:42.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:42 smithi187 ceph-mon[130973]: scrub summary: idle 2023-08-11T14:04:42.647 INFO:teuthology.orchestra.run.smithi003.stderr:10000000 of 50000000 tuples (20%) done (elapsed 28.15 s, remaining 112.59 s) 2023-08-11T14:04:42.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:42 smithi103 ceph-mon[130434]: scrub summary: idle 2023-08-11T14:04:42.765 INFO:teuthology.orchestra.run.smithi003.stderr:10100000 of 50000000 tuples (20%) done (elapsed 28.27 s, remaining 111.66 s) 2023-08-11T14:04:42.991 INFO:teuthology.orchestra.run.smithi003.stderr:10200000 of 50000000 tuples (20%) done (elapsed 28.49 s, remaining 111.17 s) 2023-08-11T14:04:43.182 INFO:teuthology.orchestra.run.smithi003.stderr:10300000 of 50000000 tuples (20%) done (elapsed 28.68 s, remaining 110.55 s) 2023-08-11T14:04:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:10400000 of 50000000 tuples (20%) done (elapsed 28.88 s, remaining 109.98 s) 2023-08-11T14:04:43.518 INFO:teuthology.orchestra.run.smithi003.stderr:10500000 of 50000000 tuples (21%) done (elapsed 29.02 s, remaining 109.16 s) 2023-08-11T14:04:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:43 smithi003 ceph-mon[97195]: scrub summary: idle 2023-08-11T14:04:43.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:43 smithi003 ceph-mon[97195]: pgmap v371: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 146 MiB/s wr, 193 op/s 2023-08-11T14:04:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:43 smithi003 ceph-mon[97195]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-08-11T14:04:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:43 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:04:43.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:43 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:43.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:43 smithi187 ceph-mon[130973]: scrub summary: idle 2023-08-11T14:04:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:43 smithi187 ceph-mon[130973]: pgmap v371: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 146 MiB/s wr, 193 op/s 2023-08-11T14:04:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:43 smithi187 ceph-mon[130973]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-08-11T14:04:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:43 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:04:43.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:43 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:43 smithi103 ceph-mon[130434]: scrub summary: idle 2023-08-11T14:04:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:43 smithi103 ceph-mon[130434]: pgmap v371: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 146 MiB/s wr, 193 op/s 2023-08-11T14:04:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:43 smithi103 ceph-mon[130434]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-08-11T14:04:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:43 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] up:active 2023-08-11T14:04:43.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:43 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:04:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:10600000 of 50000000 tuples (21%) done (elapsed 29.20 s, remaining 108.53 s) 2023-08-11T14:04:43.882 INFO:teuthology.orchestra.run.smithi003.stderr:10700000 of 50000000 tuples (21%) done (elapsed 29.38 s, remaining 107.92 s) 2023-08-11T14:04:44.038 INFO:teuthology.orchestra.run.smithi003.stderr:10800000 of 50000000 tuples (21%) done (elapsed 29.54 s, remaining 107.21 s) 2023-08-11T14:04:44.203 INFO:teuthology.orchestra.run.smithi003.stderr:10900000 of 50000000 tuples (21%) done (elapsed 29.70 s, remaining 106.55 s) 2023-08-11T14:04:44.419 INFO:teuthology.orchestra.run.smithi003.stderr:11000000 of 50000000 tuples (22%) done (elapsed 29.92 s, remaining 106.08 s) 2023-08-11T14:04:44.627 INFO:teuthology.orchestra.run.smithi003.stderr:11100000 of 50000000 tuples (22%) done (elapsed 30.13 s, remaining 105.58 s) 2023-08-11T14:04:44.820 INFO:teuthology.orchestra.run.smithi003.stderr:11200000 of 50000000 tuples (22%) done (elapsed 30.32 s, remaining 105.04 s) 2023-08-11T14:04:44.956 INFO:teuthology.orchestra.run.smithi003.stderr:11300000 of 50000000 tuples (22%) done (elapsed 30.46 s, remaining 104.30 s) 2023-08-11T14:04:45.167 INFO:teuthology.orchestra.run.smithi003.stderr:11400000 of 50000000 tuples (22%) done (elapsed 30.67 s, remaining 103.84 s) 2023-08-11T14:04:45.356 INFO:teuthology.orchestra.run.smithi003.stderr:11500000 of 50000000 tuples (23%) done (elapsed 30.86 s, remaining 103.30 s) 2023-08-11T14:04:45.503 INFO:teuthology.orchestra.run.smithi003.stderr:11600000 of 50000000 tuples (23%) done (elapsed 31.00 s, remaining 102.63 s) 2023-08-11T14:04:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:45 smithi003 ceph-mon[97195]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 129 MiB/s wr, 183 op/s 2023-08-11T14:04:45.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:45 smithi187 ceph-mon[130973]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 129 MiB/s wr, 183 op/s 2023-08-11T14:04:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:45 smithi103 ceph-mon[130434]: pgmap v372: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 129 MiB/s wr, 183 op/s 2023-08-11T14:04:45.685 INFO:teuthology.orchestra.run.smithi003.stderr:11700000 of 50000000 tuples (23%) done (elapsed 31.19 s, remaining 102.09 s) 2023-08-11T14:04:45.853 INFO:teuthology.orchestra.run.smithi003.stderr:11800000 of 50000000 tuples (23%) done (elapsed 31.35 s, remaining 101.50 s) 2023-08-11T14:04:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:11900000 of 50000000 tuples (23%) done (elapsed 31.53 s, remaining 100.95 s) 2023-08-11T14:04:46.241 INFO:teuthology.orchestra.run.smithi003.stderr:12000000 of 50000000 tuples (24%) done (elapsed 31.74 s, remaining 100.51 s) 2023-08-11T14:04:46.468 INFO:teuthology.orchestra.run.smithi003.stderr:12100000 of 50000000 tuples (24%) done (elapsed 31.97 s, remaining 100.13 s) 2023-08-11T14:04:46.618 INFO:teuthology.orchestra.run.smithi003.stderr:12200000 of 50000000 tuples (24%) done (elapsed 32.12 s, remaining 99.51 s) 2023-08-11T14:04:46.777 INFO:teuthology.orchestra.run.smithi003.stderr:12300000 of 50000000 tuples (24%) done (elapsed 32.28 s, remaining 98.93 s) 2023-08-11T14:04:46.980 INFO:teuthology.orchestra.run.smithi003.stderr:12400000 of 50000000 tuples (24%) done (elapsed 32.48 s, remaining 98.49 s) 2023-08-11T14:04:47.153 INFO:teuthology.orchestra.run.smithi003.stderr:12500000 of 50000000 tuples (25%) done (elapsed 32.65 s, remaining 97.96 s) 2023-08-11T14:04:47.293 INFO:teuthology.orchestra.run.smithi003.stderr:12600000 of 50000000 tuples (25%) done (elapsed 32.79 s, remaining 97.34 s) 2023-08-11T14:04:47.476 INFO:teuthology.orchestra.run.smithi003.stderr:12700000 of 50000000 tuples (25%) done (elapsed 32.98 s, remaining 96.85 s) 2023-08-11T14:04:47.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:47 smithi003 ceph-mon[97195]: pgmap v373: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 118 MiB/s wr, 163 op/s 2023-08-11T14:04:47.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:47 smithi187 ceph-mon[130973]: pgmap v373: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 118 MiB/s wr, 163 op/s 2023-08-11T14:04:47.665 INFO:teuthology.orchestra.run.smithi003.stderr:12800000 of 50000000 tuples (25%) done (elapsed 33.16 s, remaining 96.39 s) 2023-08-11T14:04:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:47 smithi103 ceph-mon[130434]: pgmap v373: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 118 MiB/s wr, 163 op/s 2023-08-11T14:04:47.880 INFO:teuthology.orchestra.run.smithi003.stderr:12900000 of 50000000 tuples (25%) done (elapsed 33.38 s, remaining 96.00 s) 2023-08-11T14:04:48.015 INFO:teuthology.orchestra.run.smithi003.stderr:13000000 of 50000000 tuples (26%) done (elapsed 33.52 s, remaining 95.39 s) 2023-08-11T14:04:48.221 INFO:teuthology.orchestra.run.smithi003.stderr:13100000 of 50000000 tuples (26%) done (elapsed 33.72 s, remaining 94.99 s) 2023-08-11T14:04:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:13200000 of 50000000 tuples (26%) done (elapsed 33.95 s, remaining 94.64 s) 2023-08-11T14:04:48.676 INFO:teuthology.orchestra.run.smithi003.stderr:13300000 of 50000000 tuples (26%) done (elapsed 34.18 s, remaining 94.31 s) 2023-08-11T14:04:48.874 INFO:teuthology.orchestra.run.smithi003.stderr:13400000 of 50000000 tuples (26%) done (elapsed 34.37 s, remaining 93.89 s) 2023-08-11T14:04:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:13500000 of 50000000 tuples (27%) done (elapsed 34.52 s, remaining 93.32 s) 2023-08-11T14:04:49.241 INFO:teuthology.orchestra.run.smithi003.stderr:13600000 of 50000000 tuples (27%) done (elapsed 34.74 s, remaining 92.98 s) 2023-08-11T14:04:49.405 INFO:teuthology.orchestra.run.smithi003.stderr:13700000 of 50000000 tuples (27%) done (elapsed 34.90 s, remaining 92.48 s) 2023-08-11T14:04:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:49 smithi103 ceph-mon[130434]: pgmap v374: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 118 MiB/s wr, 149 op/s 2023-08-11T14:04:49.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:49 smithi187 ceph-mon[130973]: pgmap v374: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 118 MiB/s wr, 149 op/s 2023-08-11T14:04:49.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:49 smithi003 ceph-mon[97195]: pgmap v374: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 118 MiB/s wr, 149 op/s 2023-08-11T14:04:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:13800000 of 50000000 tuples (27%) done (elapsed 35.37 s, remaining 92.78 s) 2023-08-11T14:04:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:13900000 of 50000000 tuples (27%) done (elapsed 36.44 s, remaining 94.65 s) 2023-08-11T14:04:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:14000000 of 50000000 tuples (28%) done (elapsed 36.63 s, remaining 94.20 s) 2023-08-11T14:04:51.328 INFO:teuthology.orchestra.run.smithi003.stderr:14100000 of 50000000 tuples (28%) done (elapsed 36.83 s, remaining 93.77 s) 2023-08-11T14:04:51.526 INFO:teuthology.orchestra.run.smithi003.stderr:14200000 of 50000000 tuples (28%) done (elapsed 37.03 s, remaining 93.35 s) 2023-08-11T14:04:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:51 smithi103 ceph-mon[130434]: pgmap v375: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 104 MiB/s wr, 88 op/s 2023-08-11T14:04:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:14300000 of 50000000 tuples (28%) done (elapsed 37.23 s, remaining 92.94 s) 2023-08-11T14:04:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:51 smithi003 ceph-mon[97195]: pgmap v375: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 104 MiB/s wr, 88 op/s 2023-08-11T14:04:51.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:51 smithi187 ceph-mon[130973]: pgmap v375: 129 pgs: 129 active+clean; 2.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 104 MiB/s wr, 88 op/s 2023-08-11T14:04:51.862 INFO:teuthology.orchestra.run.smithi003.stderr:14400000 of 50000000 tuples (28%) done (elapsed 37.36 s, remaining 92.37 s) 2023-08-11T14:04:52.036 INFO:teuthology.orchestra.run.smithi003.stderr:14500000 of 50000000 tuples (29%) done (elapsed 37.54 s, remaining 91.90 s) 2023-08-11T14:04:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.221+0000 7f1da57ff700 1 -- 172.21.15.3:0/4117343531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da01024c0 msgr2=0x7f1da0108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.221+0000 7f1da57ff700 1 --2- 172.21.15.3:0/4117343531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da01024c0 0x7f1da0108d70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800b0d0 tx=0x7f1d9800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 -- 172.21.15.3:0/4117343531 shutdown_connections 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 --2- 172.21.15.3:0/4117343531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da01024c0 0x7f1da0108d70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 --2- 172.21.15.3:0/4117343531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0100930 0x7f1da0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 --2- 172.21.15.3:0/4117343531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1da0101b20 0x7f1da0101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 -- 172.21.15.3:0/4117343531 >> 172.21.15.3:0/4117343531 conn(0x7f1da00fbe60 msgr2=0x7f1da00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 -- 172.21.15.3:0/4117343531 shutdown_connections 2023-08-11T14:04:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.222+0000 7f1da57ff700 1 -- 172.21.15.3:0/4117343531 wait complete. 2023-08-11T14:04:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.223+0000 7f1da57ff700 1 Processor -- start 2023-08-11T14:04:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.223+0000 7f1da57ff700 1 -- start start 2023-08-11T14:04:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 0x7f1da0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1da01024c0 0x7f1da0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019a470 con 0x7f1da0100930 2023-08-11T14:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019a5e0 con 0x7f1da01024c0 2023-08-11T14:04:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1da57ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da019a7a0 con 0x7f1da0101b20 2023-08-11T14:04:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58948/0 (socket says 172.21.15.3:58948) 2023-08-11T14:04:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.224+0000 7f1d9effd700 1 -- 172.21.15.3:0/513690170 learned_addr learned my addr 172.21.15.3:0/513690170 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 0x7f1da0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/513690170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1da01024c0 msgr2=0x7f1da0199e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:04:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1da01024c0 0x7f1da0199e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 msgr2=0x7f1da0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/513690170 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98009d00 con 0x7f1da0101b20 2023-08-11T14:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.225+0000 7f1d9effd700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 0x7f1da0198f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1d87fff700 1 -- 172.21.15.3:0/513690170 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000cbd0 con 0x7f1da0101b20 2023-08-11T14:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1d87fff700 1 -- 172.21.15.3:0/513690170 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d9000cd30 con 0x7f1da0101b20 2023-08-11T14:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da019aa80 con 0x7f1da0101b20 2023-08-11T14:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1d87fff700 1 -- 172.21.15.3:0/513690170 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90018680 con 0x7f1da0101b20 2023-08-11T14:04:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.226+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da0102f20 con 0x7f1da0101b20 2023-08-11T14:04:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.230+0000 7f1d87fff700 1 -- 172.21.15.3:0/513690170 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1d90010430 con 0x7f1da0101b20 2023-08-11T14:04:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.231+0000 7f1db914bb80 1 Processor -- start 2023-08-11T14:04:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.231+0000 7f1db914bb80 1 -- start start 2023-08-11T14:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.231+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.231+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902afe8a0 0x55b902aff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.231+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 0x55b902affc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1db914bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902a575d0 con 0x55b902aff790 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1db914bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902a54ef0 con 0x55b902afdf40 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1db914bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902a507e0 con 0x55b902afe8a0 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33394/0 (socket says 172.21.15.3:33394) 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 -- 172.21.15.3:0/223739788 learned_addr learned my addr 172.21.15.3:0/223739788 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 0x55b902affc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9effd700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902afe8a0 0x55b902aff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 -- 172.21.15.3:0/223739788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902afe8a0 msgr2=0x55b902aff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902afe8a0 0x55b902aff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 -- 172.21.15.3:0/223739788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 msgr2=0x55b902affc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 0x55b902affc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9f7fe700 1 -- 172.21.15.3:0/223739788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9028d05d0 con 0x55b902afdf40 2023-08-11T14:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 0x55b902affc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.232+0000 7f1d87fff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 0x7f1d880a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009bc0 tx=0x7f1d98009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09b420f1fb21b64 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d87fff700 1 -- 172.21.15.3:0/513690170 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1d90014070 con 0x7f1da0101b20 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d9effd700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 0x7f1d880a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d857fa700 1 -- 172.21.15.3:0/223739788 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800e040 con 0x55b902afdf40 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d857fa700 1 -- 172.21.15.3:0/223739788 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d9800baa0 con 0x55b902afdf40 2023-08-11T14:04:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.233+0000 7f1d857fa700 1 -- 172.21.15.3:0/223739788 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980050d0 con 0x55b902afdf40 2023-08-11T14:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- 172.21.15.3:0/223739788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 msgr2=0x55b902afe360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009bc0 tx=0x7f1d98009bf0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1d9effd700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 0x7f1d880a2410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005950 tx=0x7f1d94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- 172.21.15.3:0/223739788 shutdown_connections 2023-08-11T14:04:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902aff790 0x55b902affc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902afe360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 --2- 172.21.15.3:0/223739788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902afe8a0 0x55b902aff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- 172.21.15.3:0/223739788 >> 172.21.15.3:0/223739788 conn(0x55b902a2b8f0 msgr2=0x55b902afa920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- 172.21.15.3:0/223739788 shutdown_connections 2023-08-11T14:04:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- 172.21.15.3:0/223739788 wait complete. 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.234+0000 7f1db914bb80 1 -- start start 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 Processor -- start 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b07bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b08110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9029eea60 con 0x55b902afdf40 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9029ee760 con 0x55b902afe8a0 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9028d28c0 con 0x55b902aff790 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1db914bb80 10 client.0 _ll_register_callbacks cb 0x55b9029f1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b07bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b07bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58968/0 (socket says 172.21.15.3:58968) 2023-08-11T14:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.235+0000 7f1d9effd700 1 -- 172.21.15.3:0/3911774232 learned_addr learned my addr 172.21.15.3:0/3911774232 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b08110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/3911774232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 msgr2=0x55b902b0f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/3911774232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 msgr2=0x55b902b07bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b07bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 -- 172.21.15.3:0/3911774232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da01096c0 con 0x55b902afe8a0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b08110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b902afef70 tx=0x7f1d900994d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96da85b06d965ccd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d900a0b10 con 0x55b902afe8a0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1d90099690 con 0x55b902afe8a0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.236+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b902af17e0 con 0x55b902afe8a0 2023-08-11T14:04:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d900988f0 con 0x55b902afe8a0 2023-08-11T14:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 10 client.25820 fetch_fsmap learned FSMap version 31 2023-08-11T14:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b90287cd80 con 0x55b902afe8a0 2023-08-11T14:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b902af17e0 con 0x55b902afe8a0 2023-08-11T14:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1d900cdcc0 con 0x55b902afe8a0 2023-08-11T14:04:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 10 client.25820 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 20 client.25820 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b13b20 0x55b902b15f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 4 client.25820 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.237+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b902acd480 con 0x55b902b13b20 2023-08-11T14:04:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.238+0000 7f1d9effd700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b13b20 0x55b902b15f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.238+0000 7f1d9effd700 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b13b20 0x55b902b15f40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1f1b51b5faf65b server_cookie=55bc0635ff88cc26 in_seq=0 out_seq=0 2023-08-11T14:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.238+0000 7f1d82ff5700 10 client.25820 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1d82ff5700 1 -- 172.21.15.3:0/3911774232 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b902acd480 con 0x55b902b13b20 2023-08-11T14:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1d82ff5700 10 client.25820 handle_command_reply: tid=0 2023-08-11T14:04:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 client.25820 shutdown 2023-08-11T14:04:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 msgr2=0x55b902b08110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b08110 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55b902afef70 tx=0x7f1d900994d0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 shutdown_connections 2023-08-11T14:04:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b07bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b13b20 0x55b902b15f40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 --2- 172.21.15.3:0/3911774232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b08110 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 >> 172.21.15.3:0/3911774232 conn(0x55b902a2b8f0 msgr2=0x55b902af9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1d82ff5700 0 client.25820 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 shutdown_connections 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 1 -- 172.21.15.3:0/3911774232 wait complete. 2023-08-11T14:04:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.239+0000 7f1db914bb80 20 client.25820 trim_cache size 0 max 16384 2023-08-11T14:04:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:14600000 of 50000000 tuples (29%) done (elapsed 37.74 s, remaining 91.51 s) 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 Processor -- start 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 -- start start 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afe8a0 0x55b902b0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902afa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902b08ad0 con 0x55b902afe8a0 2023-08-11T14:04:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.255+0000 7f1db914bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902af3ea0 con 0x55b902afdf40 2023-08-11T14:04:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1db914bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902afec80 con 0x55b902aff790 2023-08-11T14:04:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afe8a0 0x55b902b0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33416/0 (socket says 172.21.15.3:33416) 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 -- 172.21.15.3:0/1792941622 learned_addr learned my addr 172.21.15.3:0/1792941622 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902afa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 -- 172.21.15.3:0/1792941622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 msgr2=0x55b902afa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902afa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 -- 172.21.15.3:0/1792941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afe8a0 msgr2=0x55b902b0f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afe8a0 0x55b902b0f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.256+0000 7f1d9effd700 1 -- 172.21.15.3:0/1792941622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b902865c00 con 0x55b902afdf40 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.257+0000 7f1d9effd700 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d94012ac0 tx=0x7f1d9401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78190d0f893333 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.257+0000 7f1d85ffb700 1 -- 172.21.15.3:0/1792941622 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94024040 con 0x55b902afdf40 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.257+0000 7f1d85ffb700 1 -- 172.21.15.3:0/1792941622 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d940044e0 con 0x55b902afdf40 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.257+0000 7f1d85ffb700 1 -- 172.21.15.3:0/1792941622 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94017070 con 0x55b902afdf40 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- 172.21.15.3:0/1792941622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 msgr2=0x55b902affb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d94012ac0 tx=0x7f1d9401a000 comp rx=0 tx=0).stop 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- 172.21.15.3:0/1792941622 shutdown_connections 2023-08-11T14:04:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902afa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afdf40 0x55b902affb70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 --2- 172.21.15.3:0/1792941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afe8a0 0x55b902b0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- 172.21.15.3:0/1792941622 >> 172.21.15.3:0/1792941622 conn(0x55b902a2b8f0 msgr2=0x55b902b02670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- 172.21.15.3:0/1792941622 shutdown_connections 2023-08-11T14:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- 172.21.15.3:0/1792941622 wait complete. 2023-08-11T14:04:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.258+0000 7f1db914bb80 1 -- start start 2023-08-11T14:04:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 Processor -- start 2023-08-11T14:04:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b0edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902b0b870 con 0x55b902afdf40 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902b0b9e0 con 0x55b902afe8a0 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902b0bba0 con 0x55b902aff790 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1db914bb80 10 client.0 _ll_register_callbacks cb 0x55b9029f1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1d9effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58980/0 (socket says 172.21.15.3:58980) 2023-08-11T14:04:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1d9effd700 1 -- 172.21.15.3:0/933218101 learned_addr learned my addr 172.21.15.3:0/933218101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.259+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b0edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9f7fe700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 -- 172.21.15.3:0/933218101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 msgr2=0x55b902b0b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 -- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 msgr2=0x55b902b0edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b0edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 -- 172.21.15.3:0/933218101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d94012e40 con 0x55b902afdf40 2023-08-11T14:04:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b0edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d9effd700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1d940294e0 tx=0x7f1d94029510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff51609df1fa9fb3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d82ff5700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.260+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94024070 con 0x55b902afdf40 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1d94029b20 con 0x55b902afdf40 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d9800baa0 con 0x55b902afdf40 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d94017040 con 0x55b902afdf40 2023-08-11T14:04:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 10 client.15792 fetch_fsmap learned FSMap version 31 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b9028a31f0 con 0x55b902afdf40 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d9800baa0 con 0x55b902afdf40 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1d940049b0 con 0x55b902afdf40 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 20 client.15792 populate_metadata read hostname 'smithi003' 2023-08-11T14:04:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b68c30 0x55b902b6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 4 client.15792 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.261+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b902a497c0 con 0x55b902b68c30 2023-08-11T14:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.262+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b68c30 0x55b902b6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.262+0000 7f1d82ff5700 10 client.15792 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.262+0000 7f1d9e7fc700 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b68c30 0x55b902b6b050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6212767bcbf91f0a server_cookie=735ef0b5eea07532 in_seq=0 out_seq=0 2023-08-11T14:04:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1d82ff5700 1 -- 172.21.15.3:0/933218101 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b902a497c0 con 0x55b902b68c30 2023-08-11T14:04:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1d82ff5700 10 client.15792 handle_command_reply: tid=0 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 client.15792 shutdown 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 msgr2=0x55b902b0e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1d940294e0 tx=0x7f1d94029510 comp rx=0 tx=0).stop 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 shutdown_connections 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b902aff790 0x55b902b0b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b902afdf40 0x55b902b0e890 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55b902b68c30 0x55b902b6b050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 --2- 172.21.15.3:0/933218101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b902afe8a0 0x55b902b0edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 >> 172.21.15.3:0/933218101 conn(0x55b902a2b8f0 msgr2=0x55b902afa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1d82ff5700 0 client.15792 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 shutdown_connections 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 1 -- 172.21.15.3:0/933218101 wait complete. 2023-08-11T14:04:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.297+0000 7f1db914bb80 20 client.15792 trim_cache size 0 max 16384 2023-08-11T14:04:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:04:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:04:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:04:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:04:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 msgr2=0x7f1d880a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 0x7f1d880a2410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005950 tx=0x7f1d94009260 comp rx=0 tx=0).stop 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 msgr2=0x7f1da0198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 0x7f1da0198f90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edb0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 shutdown_connections 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1d8809ff50 0x7f1d880a2410 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1da01024c0 0x7f1da0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1da0100930 0x7f1da0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 --2- 172.21.15.3:0/513690170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1da0101b20 0x7f1da0198f90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:04:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 >> 172.21.15.3:0/513690170 conn(0x7f1da00fbe60 msgr2=0x7f1da00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:04:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 shutdown_connections 2023-08-11T14:04:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:04:52.298+0000 7f1da57ff700 1 -- 172.21.15.3:0/513690170 wait complete. 2023-08-11T14:04:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:14700000 of 50000000 tuples (29%) done (elapsed 37.91 s, remaining 91.04 s) 2023-08-11T14:04:52.571 INFO:teuthology.orchestra.run.smithi003.stderr:14800000 of 50000000 tuples (29%) done (elapsed 38.07 s, remaining 90.55 s) 2023-08-11T14:04:52.743 INFO:teuthology.orchestra.run.smithi003.stderr:14900000 of 50000000 tuples (29%) done (elapsed 38.24 s, remaining 90.09 s) 2023-08-11T14:04:52.947 INFO:teuthology.orchestra.run.smithi003.stderr:15000000 of 50000000 tuples (30%) done (elapsed 38.45 s, remaining 89.71 s) 2023-08-11T14:04:53.125 INFO:teuthology.orchestra.run.smithi003.stderr:15100000 of 50000000 tuples (30%) done (elapsed 38.63 s, remaining 89.27 s) 2023-08-11T14:04:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:15200000 of 50000000 tuples (30%) done (elapsed 38.75 s, remaining 88.73 s) 2023-08-11T14:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:15300000 of 50000000 tuples (30%) done (elapsed 38.98 s, remaining 88.41 s) 2023-08-11T14:04:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:15400000 of 50000000 tuples (30%) done (elapsed 39.16 s, remaining 87.97 s) 2023-08-11T14:04:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:53 smithi103 ceph-mon[130434]: pgmap v376: 129 pgs: 129 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 126 MiB/s wr, 81 op/s 2023-08-11T14:04:53.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:53 smithi003 ceph-mon[97195]: pgmap v376: 129 pgs: 129 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 126 MiB/s wr, 81 op/s 2023-08-11T14:04:53.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:53 smithi187 ceph-mon[130973]: pgmap v376: 129 pgs: 129 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 126 MiB/s wr, 81 op/s 2023-08-11T14:04:53.877 INFO:teuthology.orchestra.run.smithi003.stderr:15500000 of 50000000 tuples (31%) done (elapsed 39.38 s, remaining 87.65 s) 2023-08-11T14:04:53.997 INFO:teuthology.orchestra.run.smithi003.stderr:15600000 of 50000000 tuples (31%) done (elapsed 39.50 s, remaining 87.10 s) 2023-08-11T14:04:54.190 INFO:teuthology.orchestra.run.smithi003.stderr:15700000 of 50000000 tuples (31%) done (elapsed 39.69 s, remaining 86.71 s) 2023-08-11T14:04:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:15800000 of 50000000 tuples (31%) done (elapsed 39.87 s, remaining 86.31 s) 2023-08-11T14:04:54.563 INFO:teuthology.orchestra.run.smithi003.stderr:15900000 of 50000000 tuples (31%) done (elapsed 40.06 s, remaining 85.92 s) 2023-08-11T14:04:54.676 INFO:teuthology.orchestra.run.smithi003.stderr:16000000 of 50000000 tuples (32%) done (elapsed 40.18 s, remaining 85.37 s) 2023-08-11T14:04:54.900 INFO:teuthology.orchestra.run.smithi003.stderr:16100000 of 50000000 tuples (32%) done (elapsed 40.40 s, remaining 85.06 s) 2023-08-11T14:04:55.082 INFO:teuthology.orchestra.run.smithi003.stderr:16200000 of 50000000 tuples (32%) done (elapsed 40.58 s, remaining 84.67 s) 2023-08-11T14:04:55.229 INFO:teuthology.orchestra.run.smithi003.stderr:16300000 of 50000000 tuples (32%) done (elapsed 40.73 s, remaining 84.21 s) 2023-08-11T14:04:55.469 INFO:teuthology.orchestra.run.smithi003.stderr:16400000 of 50000000 tuples (32%) done (elapsed 40.97 s, remaining 83.94 s) 2023-08-11T14:04:55.637 INFO:teuthology.orchestra.run.smithi003.stderr:16500000 of 50000000 tuples (33%) done (elapsed 41.14 s, remaining 83.52 s) 2023-08-11T14:04:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:55 smithi103 ceph-mon[130434]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 MiB/s wr, 84 op/s 2023-08-11T14:04:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:55 smithi003 ceph-mon[97195]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 MiB/s wr, 84 op/s 2023-08-11T14:04:55.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:55 smithi187 ceph-mon[130973]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 119 MiB/s wr, 84 op/s 2023-08-11T14:04:55.850 INFO:teuthology.orchestra.run.smithi003.stderr:16600000 of 50000000 tuples (33%) done (elapsed 41.35 s, remaining 83.20 s) 2023-08-11T14:04:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:16700000 of 50000000 tuples (33%) done (elapsed 41.52 s, remaining 82.80 s) 2023-08-11T14:04:56.209 INFO:teuthology.orchestra.run.smithi003.stderr:16800000 of 50000000 tuples (33%) done (elapsed 41.71 s, remaining 82.42 s) 2023-08-11T14:04:56.354 INFO:teuthology.orchestra.run.smithi003.stderr:16900000 of 50000000 tuples (33%) done (elapsed 41.85 s, remaining 81.97 s) 2023-08-11T14:04:56.535 INFO:teuthology.orchestra.run.smithi003.stderr:17000000 of 50000000 tuples (34%) done (elapsed 42.04 s, remaining 81.60 s) 2023-08-11T14:04:56.737 INFO:teuthology.orchestra.run.smithi003.stderr:17100000 of 50000000 tuples (34%) done (elapsed 42.24 s, remaining 81.26 s) 2023-08-11T14:04:56.943 INFO:teuthology.orchestra.run.smithi003.stderr:17200000 of 50000000 tuples (34%) done (elapsed 42.44 s, remaining 80.94 s) 2023-08-11T14:04:57.145 INFO:teuthology.orchestra.run.smithi003.stderr:17300000 of 50000000 tuples (34%) done (elapsed 42.65 s, remaining 80.61 s) 2023-08-11T14:04:57.256 INFO:teuthology.orchestra.run.smithi003.stderr:17400000 of 50000000 tuples (34%) done (elapsed 42.76 s, remaining 80.11 s) 2023-08-11T14:04:57.493 INFO:teuthology.orchestra.run.smithi003.stderr:17500000 of 50000000 tuples (35%) done (elapsed 42.99 s, remaining 79.85 s) 2023-08-11T14:04:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:57 smithi103 ceph-mon[130434]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 MiB/s wr, 70 op/s 2023-08-11T14:04:57.686 INFO:teuthology.orchestra.run.smithi003.stderr:17600000 of 50000000 tuples (35%) done (elapsed 43.19 s, remaining 79.50 s) 2023-08-11T14:04:57.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:57 smithi003 ceph-mon[97195]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 MiB/s wr, 70 op/s 2023-08-11T14:04:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:57 smithi187 ceph-mon[130973]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 MiB/s wr, 70 op/s 2023-08-11T14:04:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:17700000 of 50000000 tuples (35%) done (elapsed 43.71 s, remaining 79.76 s) 2023-08-11T14:04:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:04:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:04:58.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:04:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:17800000 of 50000000 tuples (35%) done (elapsed 44.53 s, remaining 80.56 s) 2023-08-11T14:04:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:17900000 of 50000000 tuples (35%) done (elapsed 44.82 s, remaining 80.37 s) 2023-08-11T14:04:59.476 INFO:teuthology.orchestra.run.smithi003.stderr:18000000 of 50000000 tuples (36%) done (elapsed 44.98 s, remaining 79.96 s) 2023-08-11T14:04:59.651 INFO:teuthology.orchestra.run.smithi003.stderr:18100000 of 50000000 tuples (36%) done (elapsed 45.15 s, remaining 79.58 s) 2023-08-11T14:04:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:59 smithi103 ceph-mon[130434]: pgmap v379: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 103 MiB/s wr, 76 op/s 2023-08-11T14:04:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:04:59 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:59 smithi003 ceph-mon[97195]: pgmap v379: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 103 MiB/s wr, 76 op/s 2023-08-11T14:04:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:04:59 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:59 smithi187 ceph-mon[130973]: pgmap v379: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 103 MiB/s wr, 76 op/s 2023-08-11T14:04:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:04:59 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:04:59.859 INFO:teuthology.orchestra.run.smithi003.stderr:18200000 of 50000000 tuples (36%) done (elapsed 45.36 s, remaining 79.25 s) 2023-08-11T14:05:00.053 INFO:teuthology.orchestra.run.smithi003.stderr:18300000 of 50000000 tuples (36%) done (elapsed 45.55 s, remaining 78.91 s) 2023-08-11T14:05:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:18400000 of 50000000 tuples (36%) done (elapsed 45.73 s, remaining 78.54 s) 2023-08-11T14:05:00.368 INFO:teuthology.orchestra.run.smithi003.stderr:18500000 of 50000000 tuples (37%) done (elapsed 45.87 s, remaining 78.10 s) 2023-08-11T14:05:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:18600000 of 50000000 tuples (37%) done (elapsed 46.07 s, remaining 77.77 s) 2023-08-11T14:05:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:05:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:05:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:05:00.731 INFO:teuthology.orchestra.run.smithi003.stderr:18700000 of 50000000 tuples (37%) done (elapsed 46.23 s, remaining 77.38 s) 2023-08-11T14:05:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:05:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:05:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:05:00.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:05:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:05:00.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:05:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:18800000 of 50000000 tuples (37%) done (elapsed 46.43 s, remaining 77.05 s) 2023-08-11T14:05:01.087 INFO:teuthology.orchestra.run.smithi003.stderr:18900000 of 50000000 tuples (37%) done (elapsed 46.59 s, remaining 76.66 s) 2023-08-11T14:05:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:19000000 of 50000000 tuples (38%) done (elapsed 46.78 s, remaining 76.33 s) 2023-08-11T14:05:01.465 INFO:teuthology.orchestra.run.smithi003.stderr:19100000 of 50000000 tuples (38%) done (elapsed 46.96 s, remaining 75.98 s) 2023-08-11T14:05:01.660 INFO:teuthology.orchestra.run.smithi003.stderr:19200000 of 50000000 tuples (38%) done (elapsed 47.16 s, remaining 75.65 s) 2023-08-11T14:05:01.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:01 smithi103 ceph-mon[130434]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 MiB/s wr, 66 op/s 2023-08-11T14:05:01.774 INFO:teuthology.orchestra.run.smithi003.stderr:19300000 of 50000000 tuples (38%) done (elapsed 47.27 s, remaining 75.20 s) 2023-08-11T14:05:01.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:01 smithi003 ceph-mon[97195]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 MiB/s wr, 66 op/s 2023-08-11T14:05:01.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:01 smithi187 ceph-mon[130973]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 98 MiB/s wr, 66 op/s 2023-08-11T14:05:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:19400000 of 50000000 tuples (38%) done (elapsed 47.47 s, remaining 74.87 s) 2023-08-11T14:05:02.143 INFO:teuthology.orchestra.run.smithi003.stderr:19500000 of 50000000 tuples (39%) done (elapsed 47.64 s, remaining 74.52 s) 2023-08-11T14:05:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:19600000 of 50000000 tuples (39%) done (elapsed 47.84 s, remaining 74.20 s) 2023-08-11T14:05:02.525 INFO:teuthology.orchestra.run.smithi003.stderr:19700000 of 50000000 tuples (39%) done (elapsed 48.02 s, remaining 73.87 s) 2023-08-11T14:05:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:19800000 of 50000000 tuples (39%) done (elapsed 48.24 s, remaining 73.57 s) 2023-08-11T14:05:02.872 INFO:teuthology.orchestra.run.smithi003.stderr:19900000 of 50000000 tuples (39%) done (elapsed 48.37 s, remaining 73.17 s) 2023-08-11T14:05:03.098 INFO:teuthology.orchestra.run.smithi003.stderr:20000000 of 50000000 tuples (40%) done (elapsed 48.60 s, remaining 72.90 s) 2023-08-11T14:05:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:20100000 of 50000000 tuples (40%) done (elapsed 48.77 s, remaining 72.55 s) 2023-08-11T14:05:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:20200000 of 50000000 tuples (40%) done (elapsed 48.92 s, remaining 72.18 s) 2023-08-11T14:05:03.611 INFO:teuthology.orchestra.run.smithi003.stderr:20300000 of 50000000 tuples (40%) done (elapsed 49.11 s, remaining 71.85 s) 2023-08-11T14:05:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:03 smithi103 ceph-mon[130434]: pgmap v381: 129 pgs: 129 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-08-11T14:05:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:20400000 of 50000000 tuples (40%) done (elapsed 49.26 s, remaining 71.48 s) 2023-08-11T14:05:03.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:03 smithi003 ceph-mon[97195]: pgmap v381: 129 pgs: 129 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-08-11T14:05:03.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:03 smithi187 ceph-mon[130973]: pgmap v381: 129 pgs: 129 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-08-11T14:05:03.964 INFO:teuthology.orchestra.run.smithi003.stderr:20500000 of 50000000 tuples (41%) done (elapsed 49.46 s, remaining 71.18 s) 2023-08-11T14:05:04.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-08-11T14:05:04.161 INFO:teuthology.orchestra.run.smithi003.stderr:20600000 of 50000000 tuples (41%) done (elapsed 49.66 s, remaining 70.87 s) 2023-08-11T14:05:04.283 INFO:teuthology.orchestra.run.smithi003.stderr:20700000 of 50000000 tuples (41%) done (elapsed 49.78 s, remaining 70.47 s) 2023-08-11T14:05:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 -- 172.21.15.3:0/4217082251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234073380 msgr2=0x7f8234073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 --2- 172.21.15.3:0/4217082251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234073380 0x7f8234073800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8228009990 tx=0x7f8228009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 -- 172.21.15.3:0/4217082251 shutdown_connections 2023-08-11T14:05:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 --2- 172.21.15.3:0/4217082251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82341087f0 0x7f823410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 --2- 172.21.15.3:0/4217082251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8234074960 0x7f8234072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 --2- 172.21.15.3:0/4217082251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234073380 0x7f8234073800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 -- 172.21.15.3:0/4217082251 >> 172.21.15.3:0/4217082251 conn(0x7f82340fbea0 msgr2=0x7f82340fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 -- 172.21.15.3:0/4217082251 shutdown_connections 2023-08-11T14:05:04.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.390+0000 7f8238bff700 1 -- 172.21.15.3:0/4217082251 wait complete. 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.392+0000 7f8238bff700 1 Processor -- start 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.392+0000 7f8238bff700 1 -- start start 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.392+0000 7f8238bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8234073380 0x7f8234198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.392+0000 7f8238bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8238bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82341087f0 0x7f823419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8238bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82341a03e0 con 0x7f82341087f0 2023-08-11T14:05:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8238bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82341a0520 con 0x7f8234073380 2023-08-11T14:05:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8238bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82341a06b0 con 0x7f8234074960 2023-08-11T14:05:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40304/0 (socket says 172.21.15.3:40304) 2023-08-11T14:05:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 -- 172.21.15.3:0/2077453563 learned_addr learned my addr 172.21.15.3:0/2077453563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8232d9d700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82341087f0 0x7f823419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f823259c700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8234073380 0x7f8234198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 -- 172.21.15.3:0/2077453563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8234073380 msgr2=0x7f8234198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8234073380 0x7f8234198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 -- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82341087f0 msgr2=0x7f823419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82341087f0 0x7f823419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.393+0000 7f8231d9b700 1 -- 172.21.15.3:0/2077453563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280095f0 con 0x7f8234074960 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f8231d9b700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8228004710 tx=0x7f8228009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f82237fe700 1 -- 172.21.15.3:0/2077453563 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822801d070 con 0x7f8234074960 2023-08-11T14:05:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f82237fe700 1 -- 172.21.15.3:0/2077453563 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8228021410 con 0x7f8234074960 2023-08-11T14:05:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82341a0930 con 0x7f8234074960 2023-08-11T14:05:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f82237fe700 1 -- 172.21.15.3:0/2077453563 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228017650 con 0x7f8234074960 2023-08-11T14:05:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.394+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82341a0db0 con 0x7f8234074960 2023-08-11T14:05:04.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.399+0000 7f82237fe700 1 -- 172.21.15.3:0/2077453563 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f8228021580 con 0x7f8234074960 2023-08-11T14:05:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.399+0000 7f824c5e9b80 1 Processor -- start 2023-08-11T14:05:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 -- start start 2023-08-11T14:05:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 0x555771055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x5557710568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771056df0 0x555771057270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770fae630 con 0x555771056df0 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770fabf50 con 0x555771055ad0 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770fa7840 con 0x555771056430 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f8232d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 0x555771055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f823259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x5557710568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f8232d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 0x555771055ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38132/0 (socket says 172.21.15.3:38132) 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f8232d9d700 1 -- 172.21.15.3:0/2047702573 learned_addr learned my addr 172.21.15.3:0/2047702573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.400+0000 7f8231d9b700 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771056df0 0x555771057270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 -- 172.21.15.3:0/2047702573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 msgr2=0x555771055ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 0x555771055ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 -- 172.21.15.3:0/2047702573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771056df0 msgr2=0x555771057270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771056df0 0x555771057270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 -- 172.21.15.3:0/2047702573 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555770e27630 con 0x555771056430 2023-08-11T14:05:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f823259c700 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x5557710568b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8224009cb0 tx=0x7f8224011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3ae44201d73271 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f8220ff9700 1 -- 172.21.15.3:0/2047702573 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224011d20 con 0x555771056430 2023-08-11T14:05:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f8220ff9700 1 -- 172.21.15.3:0/2047702573 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8224011e80 con 0x555771056430 2023-08-11T14:05:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f8220ff9700 1 -- 172.21.15.3:0/2047702573 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224005280 con 0x555771056430 2023-08-11T14:05:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.401+0000 7f82237fe700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 0x7f821c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f82237fe700 1 -- 172.21.15.3:0/2077453563 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f82280cc940 con 0x7f8234074960 2023-08-11T14:05:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f8232d9d700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 0x7f821c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 -- 172.21.15.3:0/2047702573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 msgr2=0x5557710568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x5557710568b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8224009cb0 tx=0x7f8224011440 comp rx=0 tx=0).stop 2023-08-11T14:05:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 -- 172.21.15.3:0/2047702573 shutdown_connections 2023-08-11T14:05:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771056df0 0x555771057270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771055ad0 0x555771055ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/2047702573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x5557710568b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.402+0000 7f824c5e9b80 1 -- 172.21.15.3:0/2047702573 >> 172.21.15.3:0/2047702573 conn(0x555770f82950 msgr2=0x555771050420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.403+0000 7f824c5e9b80 1 -- 172.21.15.3:0/2047702573 shutdown_connections 2023-08-11T14:05:04.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.403+0000 7f8232d9d700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 0x7f821c0a2490 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f822c0075c0 tx=0x7f822c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.403+0000 7f824c5e9b80 1 -- 172.21.15.3:0/2047702573 wait complete. 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.403+0000 7f824c5e9b80 1 -- start start 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 Processor -- start 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770f45ac0 con 0x555771055ad0 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770f457c0 con 0x555771056430 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.404+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555770e29920 con 0x555771056df0 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f824c5e9b80 10 client.0 _ll_register_callbacks cb 0x555770f48ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38148/0 (socket says 172.21.15.3:38148) 2023-08-11T14:05:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 -- 172.21.15.3:0/304999141 learned_addr learned my addr 172.21.15.3:0/304999141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36484/0 (socket says 172.21.15.3:36484) 2023-08-11T14:05:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8232d9d700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 -- 172.21.15.3:0/304999141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 msgr2=0x555771066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 -- 172.21.15.3:0/304999141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 msgr2=0x55577105f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f8231d9b700 1 -- 172.21.15.3:0/304999141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82240098e0 con 0x555771056430 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.405+0000 7f823259c700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8231d9b700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f822800b2e0 tx=0x7f8228021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e26b2a5a49c0da server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822809f070 con 0x555771056430 2023-08-11T14:05:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557710244e0 con 0x555771056430 2023-08-11T14:05:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f822809a7a0 con 0x555771056430 2023-08-11T14:05:04.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228099e70 con 0x555771056430 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f824c5e9b80 10 client.25838 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555770dd3de0 con 0x555771056430 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.406+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557710244e0 con 0x555771056430 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f82280a1990 con 0x555771056430 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 20 client.25838 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55577106abb0 0x55577106cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 4 client.25838 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555770fa0820 con 0x55577106abb0 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.407+0000 7f823259c700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55577106abb0 0x55577106cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.408+0000 7f823259c700 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55577106abb0 0x55577106cfd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa116cd9b3c0d7a9 server_cookie=f046446f69589f54 in_seq=0 out_seq=0 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.408+0000 7f8219ffb700 10 client.25838 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.408+0000 7f8219ffb700 1 -- 172.21.15.3:0/304999141 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555770fa0820 con 0x55577106abb0 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.408+0000 7f8219ffb700 10 client.25838 handle_command_reply: tid=0 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.408+0000 7f824c5e9b80 1 client.25838 shutdown 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 msgr2=0x55577105f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f822800b2e0 tx=0x7f8228021b60 comp rx=0 tx=0).stop 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 shutdown_connections 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771066640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x55577105f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55577106abb0 0x55577106cfd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/304999141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x55577105f600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f8219ffb700 0 client.25838 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 >> 172.21.15.3:0/304999141 conn(0x555770f82950 msgr2=0x55577104acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 shutdown_connections 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 1 -- 172.21.15.3:0/304999141 wait complete. 2023-08-11T14:05:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.409+0000 7f824c5e9b80 20 client.25838 trim_cache size 0 max 16384 2023-08-11T14:05:04.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 Processor -- start 2023-08-11T14:05:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 -- start start 2023-08-11T14:05:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x55577105a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 0x555771066600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557710512f0 con 0x555771055ad0 2023-08-11T14:05:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577105d9b0 con 0x555771056df0 2023-08-11T14:05:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577106a6f0 con 0x555771056430 2023-08-11T14:05:04.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f8232d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 0x555771066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36498/0 (socket says 172.21.15.3:36498) 2023-08-11T14:05:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f823259c700 1 -- 172.21.15.3:0/1499201724 learned_addr learned my addr 172.21.15.3:0/1499201724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.427+0000 7f8231d9b700 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x55577105a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 -- 172.21.15.3:0/1499201724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 msgr2=0x55577105a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x55577105a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 -- 172.21.15.3:0/1499201724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 msgr2=0x555771066600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 0x555771066600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 -- 172.21.15.3:0/1499201724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555770dbcc60 con 0x555771055ad0 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f8232d9d700 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 0x555771066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f823259c700 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f82240092a0 tx=0x7f8224009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cce8d1805e976bb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.428+0000 7f82217fa700 1 -- 172.21.15.3:0/1499201724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224005ce0 con 0x555771055ad0 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f82217fa700 1 -- 172.21.15.3:0/1499201724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8224013e60 con 0x555771055ad0 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f82217fa700 1 -- 172.21.15.3:0/1499201724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224003680 con 0x555771055ad0 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 -- 172.21.15.3:0/1499201724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 msgr2=0x555771059c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f82240092a0 tx=0x7f8224009440 comp rx=0 tx=0).stop 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 -- 172.21.15.3:0/1499201724 shutdown_connections 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056df0 0x555771066600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771059c10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/1499201724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056430 0x55577105a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 -- 172.21.15.3:0/1499201724 >> 172.21.15.3:0/1499201724 conn(0x555770f82950 msgr2=0x555771066f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 -- 172.21.15.3:0/1499201724 shutdown_connections 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.429+0000 7f824c5e9b80 1 -- 172.21.15.3:0/1499201724 wait complete. 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.430+0000 7f824c5e9b80 1 -- start start 2023-08-11T14:05:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.430+0000 7f824c5e9b80 1 Processor -- start 2023-08-11T14:05:04.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.430+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.430+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x555771062a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f824c5e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771065a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555771065fb0 con 0x555771055ad0 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555771066120 con 0x555771056430 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f824c5e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557710662e0 con 0x555771056df0 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f824c5e9b80 10 client.0 _ll_register_callbacks cb 0x555770f48ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36500/0 (socket says 172.21.15.3:36500) 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 -- 172.21.15.3:0/305621418 learned_addr learned my addr 172.21.15.3:0/305621418 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f8231d9b700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x555771062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f8232d9d700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771065a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 -- 172.21.15.3:0/305621418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 msgr2=0x555771065a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771065a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 -- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 msgr2=0x555771062a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x555771062a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.431+0000 7f823259c700 1 -- 172.21.15.3:0/305621418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c0164d0 con 0x555771055ad0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f823259c700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8224011480 tx=0x7f82240040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a074877803044277 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f8219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224004570 con 0x555771055ad0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555771048a90 con 0x555771055ad0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8224004a60 con 0x555771055ad0 2023-08-11T14:05:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8224004c20 con 0x555771055ad0 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f824c5e9b80 10 client.15810 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f822401d040 con 0x555771055ad0 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.432+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555771048a90 con 0x555771055ad0 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f8224031030 con 0x555771055ad0 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 20 client.15810 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557710bf6c0 0x5557710c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 4 client.15810 mds_command: new command op to 24569 tid=0[{"prefix": "scrub status"}] 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555770f61050 con 0x5557710bf6c0 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f8231d9b700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557710bf6c0 0x5557710c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f8231d9b700 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557710bf6c0 0x5557710c1ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f38dfb06123783 server_cookie=26bf1f7080565310 in_seq=0 out_seq=0 2023-08-11T14:05:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.433+0000 7f8219ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f8219ffb700 1 -- 172.21.15.3:0/305621418 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555770f61050 con 0x5557710bf6c0 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f8219ffb700 10 client.15810 handle_command_reply: tid=0 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 client.15810 shutdown 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 msgr2=0x555771062510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8224011480 tx=0x7f82240040d0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 shutdown_connections 2023-08-11T14:05:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555771056df0 0x555771065a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555771055ad0 0x555771062510 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5557710bf6c0 0x5557710c1ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 --2- 172.21.15.3:0/305621418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555771056430 0x555771062a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f8219ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 >> 172.21.15.3:0/305621418 conn(0x555770f82950 msgr2=0x555771050390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 shutdown_connections 2023-08-11T14:05:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 1 -- 172.21.15.3:0/305621418 wait complete. 2023-08-11T14:05:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.434+0000 7f824c5e9b80 20 client.15810 trim_cache size 0 max 16384 2023-08-11T14:05:04.445 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:05:04.446 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2023-08-11T14:05:04.446 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2023-08-11T14:05:04.446 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 msgr2=0x7f821c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 0x7f821c0a2490 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f822c0075c0 tx=0x7f822c00b040 comp rx=0 tx=0).stop 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 msgr2=0x7f8234198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8228004710 tx=0x7f8228009580 comp rx=0 tx=0).stop 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 shutdown_connections 2023-08-11T14:05:04.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82341087f0 0x7f823419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f821c09ffd0 0x7f821c0a2490 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8234073380 0x7f8234198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 --2- 172.21.15.3:0/2077453563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234074960 0x7f8234198da0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 >> 172.21.15.3:0/2077453563 conn(0x7f82340fbea0 msgr2=0x7f82340fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 shutdown_connections 2023-08-11T14:05:04.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.435+0000 7f8238bff700 1 -- 172.21.15.3:0/2077453563 wait complete. 2023-08-11T14:05:04.455 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-08-11T14:05:04.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:05:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:20800000 of 50000000 tuples (41%) done (elapsed 50.01 s, remaining 70.21 s) 2023-08-11T14:05:04.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.677+0000 7ff59679d700 1 -- 172.21.15.3:0/697410933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff590100380 msgr2=0x7ff590100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.677+0000 7ff59679d700 1 --2- 172.21.15.3:0/697410933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff590100380 0x7ff590100800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff588009fc0 tx=0x7ff58800b330 comp rx=0 tx=0).stop 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 -- 172.21.15.3:0/697410933 shutdown_connections 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 --2- 172.21.15.3:0/697410933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff590100380 0x7ff590100800 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 --2- 172.21.15.3:0/697410933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5900ff060 0x7ff5900ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 --2- 172.21.15.3:0/697410933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 0x7ff5900ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 -- 172.21.15.3:0/697410933 >> 172.21.15.3:0/697410933 conn(0x7ff5900fabe0 msgr2=0x7ff5900fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 -- 172.21.15.3:0/697410933 shutdown_connections 2023-08-11T14:05:04.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.678+0000 7ff59679d700 1 -- 172.21.15.3:0/697410933 wait complete. 2023-08-11T14:05:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 Processor -- start 2023-08-11T14:05:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 -- start start 2023-08-11T14:05:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 0x7ff590198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff590100380 0x7ff59019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a01b0 con 0x7ff5900ff060 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a02f0 con 0x7ff590100380 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.679+0000 7ff59679d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5901a0480 con 0x7ff5900ff9c0 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36526/0 (socket says 172.21.15.3:36526) 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 -- 172.21.15.3:0/473720055 learned_addr learned my addr 172.21.15.3:0/473720055 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff594f9a700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 0x7ff590198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff595f9c700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff590100380 0x7ff59019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 -- 172.21.15.3:0/473720055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 msgr2=0x7ff590198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 0x7ff590198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 -- 172.21.15.3:0/473720055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff590100380 msgr2=0x7ff59019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff590100380 0x7ff59019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 -- 172.21.15.3:0/473720055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0095f0 con 0x7ff5900ff060 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.680+0000 7ff59579b700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00ec10 tx=0x7ff58c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.681+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00cc90 con 0x7ff5900ff060 2023-08-11T14:05:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.681+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58c0044e0 con 0x7ff5900ff060 2023-08-11T14:05:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.681+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff580009520 con 0x7ff5900ff060 2023-08-11T14:05:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.681+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c010430 con 0x7ff5900ff060 2023-08-11T14:05:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.681+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff588009c40 con 0x7ff5900ff060 2023-08-11T14:05:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.682+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff590109e40 con 0x7ff5900ff060 2023-08-11T14:05:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:20900000 of 50000000 tuples (41%) done (elapsed 50.18 s, remaining 69.87 s) 2023-08-11T14:05:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.684+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff58c003fb0 con 0x7ff5900ff060 2023-08-11T14:05:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.685+0000 7ff5867fc700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 0x7ff57c0a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.685+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff58c014070 con 0x7ff5900ff060 2023-08-11T14:05:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.686+0000 7ff594f9a700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 0x7ff57c0a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.686+0000 7ff594f9a700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 0x7ff57c0a2500 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff580009df0 tx=0x7ff580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.689+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ff58c095510 con 0x7ff5900ff060 2023-08-11T14:05:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:21000000 of 50000000 tuples (42%) done (elapsed 50.41 s, remaining 69.62 s) 2023-08-11T14:05:04.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.956+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff59010e1d0 con 0x7ff5900ff060 2023-08-11T14:05:04.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.956+0000 7ff5867fc700 1 -- 172.21.15.3:0/473720055 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11374 (secure 0 0 0) 0x7ff58c098b30 con 0x7ff5900ff060 2023-08-11T14:05:04.958 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:05:04.958 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:04:42.270149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:05:04.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 msgr2=0x7ff57c0a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 0x7ff57c0a2500 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff580009df0 tx=0x7ff580009260 comp rx=0 tx=0).stop 2023-08-11T14:05:04.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 msgr2=0x7ff590198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:04.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00ec10 tx=0x7ff58c00ef20 comp rx=0 tx=0).stop 2023-08-11T14:05:04.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 shutdown_connections 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff57c0a0040 0x7ff57c0a2500 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.959+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff590100380 0x7ff59019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.960+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5900ff060 0x7ff590198630 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.960+0000 7ff59679d700 1 --2- 172.21.15.3:0/473720055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5900ff9c0 0x7ff590198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.960+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 >> 172.21.15.3:0/473720055 conn(0x7ff5900fabe0 msgr2=0x7ff5900fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.960+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 shutdown_connections 2023-08-11T14:05:04.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:04.960+0000 7ff59679d700 1 -- 172.21.15.3:0/473720055 wait complete. 2023-08-11T14:05:04.963 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 31 2023-08-11T14:05:04.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-08-11T14:05:05.063 INFO:teuthology.orchestra.run.smithi003.stderr:21100000 of 50000000 tuples (42%) done (elapsed 50.56 s, remaining 69.25 s) 2023-08-11T14:05:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.201+0000 7f2439b9d700 1 -- 172.21.15.3:0/1196250190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2434101260 msgr2=0x7f2434103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.201+0000 7f2439b9d700 1 --2- 172.21.15.3:0/1196250190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2434101260 0x7f2434103650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2428009990 tx=0x7f2428009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 -- 172.21.15.3:0/1196250190 shutdown_connections 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 --2- 172.21.15.3:0/1196250190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24341064c0 0x7f24341088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 --2- 172.21.15.3:0/1196250190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2434101260 0x7f2434103650 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 --2- 172.21.15.3:0/1196250190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434103b90 0x7f2434105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 -- 172.21.15.3:0/1196250190 >> 172.21.15.3:0/1196250190 conn(0x7f24340fabd0 msgr2=0x7f24340fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 -- 172.21.15.3:0/1196250190 shutdown_connections 2023-08-11T14:05:05.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.202+0000 7f2439b9d700 1 -- 172.21.15.3:0/1196250190 wait complete. 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 Processor -- start 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 -- start start 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 0x7f2434198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 0x7f2434198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 0x7f243419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24341a0100 con 0x7f24341064c0 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24341a0240 con 0x7f2434103b90 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.203+0000 7f2439b9d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24341a03d0 con 0x7f2434101260 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f243939c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 0x7f243419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 0x7f2434198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 0x7f2434198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 0x7f2434198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40366/0 (socket says 172.21.15.3:40366) 2023-08-11T14:05:05.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2438b9b700 1 -- 172.21.15.3:0/397665456 learned_addr learned my addr 172.21.15.3:0/397665456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 -- 172.21.15.3:0/397665456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 msgr2=0x7f2434198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 0x7f2434198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 -- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 msgr2=0x7f243419fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 0x7f243419fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 -- 172.21.15.3:0/397665456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280095f0 con 0x7f2434103b90 2023-08-11T14:05:05.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f243939c700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 0x7f243419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2433fff700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 0x7f2434198ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9f0 tx=0x7f242000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.204+0000 7f2431ffb700 1 -- 172.21.15.3:0/397665456 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000c9c0 con 0x7f2434103b90 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.205+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24341a06b0 con 0x7f2434103b90 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.205+0000 7f2431ffb700 1 -- 172.21.15.3:0/397665456 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2420010430 con 0x7f2434103b90 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.205+0000 7f2431ffb700 1 -- 172.21.15.3:0/397665456 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420018680 con 0x7f2434103b90 2023-08-11T14:05:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.205+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24341a0c10 con 0x7f2434103b90 2023-08-11T14:05:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.208+0000 7f2431ffb700 1 -- 172.21.15.3:0/397665456 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2420010860 con 0x7f2434103b90 2023-08-11T14:05:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.209+0000 7f2431ffb700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 0x7f24240a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.210+0000 7f2431ffb700 1 -- 172.21.15.3:0/397665456 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2420014070 con 0x7f2434103b90 2023-08-11T14:05:05.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.210+0000 7f2438b9b700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 0x7f24240a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.210+0000 7f2438b9b700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 0x7f24240a2530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2428006010 tx=0x7f242800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.211+0000 7f244e710b80 1 Processor -- start 2023-08-11T14:05:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 -- start start 2023-08-11T14:05:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d30838cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30838e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3082e5630 con 0x55d30838d470 2023-08-11T14:05:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3082e2f50 con 0x55d30838cb10 2023-08-11T14:05:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f244e710b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3082de840 con 0x55d30838de90 2023-08-11T14:05:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f2433fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f2433fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36546/0 (socket says 172.21.15.3:36546) 2023-08-11T14:05:05.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f2433fff700 1 -- 172.21.15.3:0/3965427839 learned_addr learned my addr 172.21.15.3:0/3965427839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.212+0000 7f2438b9b700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d30838cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f243939c700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30838e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 -- 172.21.15.3:0/3965427839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 msgr2=0x55d30838e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30838e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 -- 172.21.15.3:0/3965427839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 msgr2=0x55d30838cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d30838cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 -- 172.21.15.3:0/3965427839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d30815e630 con 0x55d30838d470 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2438b9b700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d30838cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f2433fff700 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2420098cf0 tx=0x7f24200986c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5496a5578a5445c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f241effd700 1 -- 172.21.15.3:0/3965427839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420098d60 con 0x55d30838d470 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f241effd700 1 -- 172.21.15.3:0/3965427839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f242009cb90 con 0x55d30838d470 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.213+0000 7f241effd700 1 -- 172.21.15.3:0/3965427839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420010650 con 0x55d30838d470 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 -- 172.21.15.3:0/3965427839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 msgr2=0x55d30838d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2420098cf0 tx=0x7f24200986c0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 -- 172.21.15.3:0/3965427839 shutdown_connections 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30838e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d30838cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 --2- 172.21.15.3:0/3965427839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30838d950 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.214+0000 7f244e710b80 1 -- 172.21.15.3:0/3965427839 >> 172.21.15.3:0/3965427839 conn(0x55d3082b9950 msgr2=0x55d3083873e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.215+0000 7f244e710b80 1 -- 172.21.15.3:0/3965427839 shutdown_connections 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.215+0000 7f244e710b80 1 -- 172.21.15.3:0/3965427839 wait complete. 2023-08-11T14:05:05.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.215+0000 7f244e710b80 1 -- start start 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.215+0000 7f244e710b80 1 Processor -- start 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 0x55d3083967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30839d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30827cac0 con 0x55d30838cb10 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30827c7c0 con 0x55d30838d470 2023-08-11T14:05:05.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d308160920 con 0x55d30838de90 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f244e710b80 10 client.0 _ll_register_callbacks cb 0x55d30827fab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36550/0 (socket says 172.21.15.3:36550) 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f2438b9b700 1 -- 172.21.15.3:0/1325192410 learned_addr learned my addr 172.21.15.3:0/1325192410 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f243939c700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30839d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.216+0000 7f2433fff700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 0x55d3083967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 -- 172.21.15.3:0/1325192410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 msgr2=0x55d30839d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30839d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 -- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 msgr2=0x55d3083967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 0x55d3083967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 -- 172.21.15.3:0/1325192410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24341094c0 con 0x55d30838cb10 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2433fff700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 0x55d3083967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2438b9b700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f24280057e0 tx=0x7f2428020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a78bd96b011c5f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2413fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:05:05.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242802d040 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d30837fa90 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f242800f660 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f242800f820 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f244e710b80 10 client.15828 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.217+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f24341a0c10 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d30837fa90 con 0x55d30838cb10 2023-08-11T14:05:05.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f24280329c0 con 0x55d30838cb10 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 10 client.15828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 20 client.15828 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083a20e0 0x55d3083a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 4 client.15828 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:05.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d30835b4e0 con 0x55d3083a20e0 2023-08-11T14:05:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.218+0000 7f2433fff700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083a20e0 0x55d3083a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.219+0000 7f2433fff700 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083a20e0 0x55d3083a4500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15b6117c71941b69 server_cookie=28512c20b10c0f8c in_seq=0 out_seq=0 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.219+0000 7f2413fff700 10 client.15828 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f2413fff700 1 -- 172.21.15.3:0/1325192410 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d30835b4e0 con 0x55d3083a20e0 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f2413fff700 10 client.15828 handle_command_reply: tid=0 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 client.15828 shutdown 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 msgr2=0x55d3083962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f24280057e0 tx=0x7f2428020000 comp rx=0 tx=0).stop 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 shutdown_connections 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083a20e0 0x55d3083a4500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838de90 0x55d30839d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838cb10 0x55d3083962b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f2413fff700 0 client.15828 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 --2- 172.21.15.3:0/1325192410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838d470 0x55d3083967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 >> 172.21.15.3:0/1325192410 conn(0x55d3082b9950 msgr2=0x55d308381cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 shutdown_connections 2023-08-11T14:05:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.220+0000 7f244e710b80 1 -- 172.21.15.3:0/1325192410 wait complete. 2023-08-11T14:05:05.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.221+0000 7f244e710b80 20 client.15828 trim_cache size 0 max 16384 2023-08-11T14:05:05.235 INFO:teuthology.orchestra.run.smithi003.stderr:21200000 of 50000000 tuples (42%) done (elapsed 50.74 s, remaining 68.92 s) 2023-08-11T14:05:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.234+0000 7f244e710b80 1 Processor -- start 2023-08-11T14:05:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.234+0000 7f244e710b80 1 -- start start 2023-08-11T14:05:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30839d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 0x55d308380d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d308381240 con 0x55d30838d470 2023-08-11T14:05:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d308391910 con 0x55d30838de90 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f244e710b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d308394d60 con 0x55d30838cb10 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f2433fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30839d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f243939c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 0x55d308380d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40392/0 (socket says 172.21.15.3:40392) 2023-08-11T14:05:05.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.235+0000 7f2438b9b700 1 -- 172.21.15.3:0/1539687766 learned_addr learned my addr 172.21.15.3:0/1539687766 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 -- 172.21.15.3:0/1539687766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 msgr2=0x55d30838cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 -- 172.21.15.3:0/1539687766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 msgr2=0x55d308380d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 0x55d308380d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 -- 172.21.15.3:0/1539687766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d30810ade0 con 0x55d30838d470 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f243939c700 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 0x55d308380d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:05.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2438b9b700 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f2433fff700 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30839d7f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9c0 tx=0x7f242001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad1191f404cbec4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f241f7fe700 1 -- 172.21.15.3:0/1539687766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420098d60 con 0x55d30838d470 2023-08-11T14:05:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f241f7fe700 1 -- 172.21.15.3:0/1539687766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24200cb3f0 con 0x55d30838d470 2023-08-11T14:05:05.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.236+0000 7f241f7fe700 1 -- 172.21.15.3:0/1539687766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200a06e0 con 0x55d30838d470 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 -- 172.21.15.3:0/1539687766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 msgr2=0x55d30839d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30839d7f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9c0 tx=0x7f242001f000 comp rx=0 tx=0).stop 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 -- 172.21.15.3:0/1539687766 shutdown_connections 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838de90 0x55d308380d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838cb10 0x55d30838cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 --2- 172.21.15.3:0/1539687766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838d470 0x55d30839d7f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 -- 172.21.15.3:0/1539687766 >> 172.21.15.3:0/1539687766 conn(0x55d3082b9950 msgr2=0x55d3083873b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 -- 172.21.15.3:0/1539687766 shutdown_connections 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.237+0000 7f244e710b80 1 -- 172.21.15.3:0/1539687766 wait complete. 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 -- start start 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 Processor -- start 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838d470 0x55d308399380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838de90 0x55d30839c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30839c8e0 con 0x55d30838de90 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30839ca50 con 0x55d30838cb10 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d30839cc10 con 0x55d30838d470 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f244e710b80 10 client.0 _ll_register_callbacks cb 0x7f2428005390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.238+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f243939c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838de90 0x55d30839c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38252/0 (socket says 172.21.15.3:38252) 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 -- 172.21.15.3:0/373879259 learned_addr learned my addr 172.21.15.3:0/373879259 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2433fff700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838d470 0x55d308399380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 -- 172.21.15.3:0/373879259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838d470 msgr2=0x55d308399380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838d470 0x55d308399380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 -- 172.21.15.3:0/373879259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838de90 msgr2=0x55d30839c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838de90 0x55d30839c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 -- 172.21.15.3:0/373879259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24341a0c10 con 0x55d30838cb10 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2438b9b700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2428032cf0 tx=0x7f242800bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9b466419e9e0ad server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2413fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.239+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242802d050 con 0x55d30838cb10 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f242009cb90 con 0x55d30838cb10 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2428032630 con 0x55d30838cb10 2023-08-11T14:05:05.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f242800b2f0 con 0x55d30838cb10 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 10 client.25868 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f242c00e6e0 con 0x55d30838cb10 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f242009cb90 con 0x55d30838cb10 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f242800f460 con 0x55d30838cb10 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 10 client.25868 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 10 client.25868 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-08-11T14:05:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 20 client.25868 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083f61e0 0x55d3083f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 4 client.25868 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:05:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.240+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 --> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3082d7820 con 0x55d3083f61e0 2023-08-11T14:05:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f2433fff700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083f61e0 0x55d3083f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f2433fff700 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083f61e0 0x55d3083f8600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=812d86cc2325ab04 server_cookie=b0de25f7cda7a44d in_seq=0 out_seq=0 2023-08-11T14:05:05.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f2413fff700 10 client.25868 ms_handle_connect on v2:172.21.15.103:6834/1763153778 2023-08-11T14:05:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f2413fff700 1 -- 172.21.15.3:0/373879259 <== mds.4 v2:172.21.15.103:6834/1763153778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3082d7820 con 0x55d3083f61e0 2023-08-11T14:05:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f2413fff700 10 client.25868 handle_command_reply: tid=0 2023-08-11T14:05:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.241+0000 7f244e710b80 1 client.25868 shutdown 2023-08-11T14:05:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 msgr2=0x55d308398e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2428032cf0 tx=0x7f242800bf00 comp rx=0 tx=0).stop 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 shutdown_connections 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:6834/1763153778,v1:172.21.15.103:6835/1763153778] conn(0x55d3083f61e0 0x55d3083f8600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d30838de90 0x55d30839c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d30838cb10 0x55d308398e40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 --2- 172.21.15.3:0/373879259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d30838d470 0x55d308399380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f2413fff700 0 client.25868 ms_handle_reset on v2:172.21.15.103:6834/1763153778 2023-08-11T14:05:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 >> 172.21.15.3:0/373879259 conn(0x55d3082b9950 msgr2=0x55d308394270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 shutdown_connections 2023-08-11T14:05:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 1 -- 172.21.15.3:0/373879259 wait complete. 2023-08-11T14:05:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f244e710b80 20 client.25868 trim_cache size 0 max 16384 2023-08-11T14:05:05.254 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:05:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 msgr2=0x7f24240a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 0x7f24240a2530 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2428006010 tx=0x7f242800b260 comp rx=0 tx=0).stop 2023-08-11T14:05:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 msgr2=0x7f2434198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.242+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 0x7f2434198ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9f0 tx=0x7f242000ed00 comp rx=0 tx=0).stop 2023-08-11T14:05:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 shutdown_connections 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24341064c0 0x7f243419fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2434101260 0x7f2434198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f24240a0070 0x7f24240a2530 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 --2- 172.21.15.3:0/397665456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2434103b90 0x7f2434198ac0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 >> 172.21.15.3:0/397665456 conn(0x7f24340fabd0 msgr2=0x7f24340ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 shutdown_connections 2023-08-11T14:05:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.243+0000 7f2439b9d700 1 -- 172.21.15.3:0/397665456 wait complete. 2023-08-11T14:05:05.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-08-11T14:05:05.452 INFO:teuthology.orchestra.run.smithi003.stderr:21300000 of 50000000 tuples (42%) done (elapsed 50.95 s, remaining 68.65 s) 2023-08-11T14:05:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 -- 172.21.15.3:0/699244677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc101b00 msgr2=0x7f92bc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 --2- 172.21.15.3:0/699244677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc101b00 0x7f92bc103f90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009990 tx=0x7f92b0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 -- 172.21.15.3:0/699244677 shutdown_connections 2023-08-11T14:05:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 --2- 172.21.15.3:0/699244677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc1044d0 0x7f92bc104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 --2- 172.21.15.3:0/699244677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 --2- 172.21.15.3:0/699244677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc101b00 0x7f92bc103f90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 -- 172.21.15.3:0/699244677 >> 172.21.15.3:0/699244677 conn(0x7f92bc0fad70 msgr2=0x7f92bc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 -- 172.21.15.3:0/699244677 shutdown_connections 2023-08-11T14:05:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.460+0000 7f92c0fff700 1 -- 172.21.15.3:0/699244677 wait complete. 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.461+0000 7f92c0fff700 1 Processor -- start 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 -- start start 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc101b00 0x7f92bc19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc1044d0 0x7f92bc19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc19e780 con 0x7f92bc1044d0 2023-08-11T14:05:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc19e8f0 con 0x7f92bc0ff190 2023-08-11T14:05:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.462+0000 7f92c0fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc19eab0 con 0x7f92bc101b00 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92bad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc1044d0 0x7f92bc19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38264/0 (socket says 172.21.15.3:38264) 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 -- 172.21.15.3:0/4083856169 learned_addr learned my addr 172.21.15.3:0/4083856169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc101b00 0x7f92bc19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 -- 172.21.15.3:0/4083856169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc101b00 msgr2=0x7f92bc19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc101b00 0x7f92bc19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 -- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc1044d0 msgr2=0x7f92bc19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc1044d0 0x7f92bc19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.463+0000 7f92ba59c700 1 -- 172.21.15.3:0/4083856169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b00095f0 con 0x7f92bc0ff190 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92ba59c700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00e9a0 tx=0x7f92ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92ab7fe700 1 -- 172.21.15.3:0/4083856169 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac009790 con 0x7f92bc0ff190 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92ab7fe700 1 -- 172.21.15.3:0/4083856169 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac010430 con 0x7f92bc0ff190 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92ab7fe700 1 -- 172.21.15.3:0/4083856169 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac018680 con 0x7f92bc0ff190 2023-08-11T14:05:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc19ed90 con 0x7f92bc0ff190 2023-08-11T14:05:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.464+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc1a54a0 con 0x7f92bc0ff190 2023-08-11T14:05:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.468+0000 7f92ab7fe700 1 -- 172.21.15.3:0/4083856169 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f92ac0105a0 con 0x7f92bc0ff190 2023-08-11T14:05:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.469+0000 7f92ab7fe700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 0x7f92a40a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.469+0000 7f92ab7fe700 1 -- 172.21.15.3:0/4083856169 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f92ac014070 con 0x7f92bc0ff190 2023-08-11T14:05:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.469+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 0x7f92a40a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.470+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 0x7f92a40a2490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b2e0 tx=0x7f92b0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.470+0000 7f92d48a6b80 1 Processor -- start 2023-08-11T14:05:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 -- start start 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f60141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6014c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5f6c5d0 con 0x55f1f6014790 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5f69ef0 con 0x55f1f6012f40 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5f657e0 con 0x55f1f60138a0 2023-08-11T14:05:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.471+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38280/0 (socket says 172.21.15.3:38280) 2023-08-11T14:05:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 -- 172.21.15.3:0/1437161981 learned_addr learned my addr 172.21.15.3:0/1437161981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f60141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92bad9d700 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6014c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 -- 172.21.15.3:0/1437161981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 msgr2=0x55f1f60141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f60141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 -- 172.21.15.3:0/1437161981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 msgr2=0x55f1f6014c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6014c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 -- 172.21.15.3:0/1437161981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1f5de55d0 con 0x55f1f6012f40 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92ac098d50 tx=0x7f92ac098820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8756481e4616492 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92a8ff9700 1 -- 172.21.15.3:0/1437161981 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac09cbf0 con 0x55f1f6012f40 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92a8ff9700 1 -- 172.21.15.3:0/1437161981 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac0a0bd0 con 0x55f1f6012f40 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.472+0000 7f92a8ff9700 1 -- 172.21.15.3:0/1437161981 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0982b0 con 0x55f1f6012f40 2023-08-11T14:05:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.473+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1437161981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 msgr2=0x55f1f6013360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.473+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92ac098d50 tx=0x7f92ac098820 comp rx=0 tx=0).stop 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1437161981 shutdown_connections 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6014c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f6013360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1437161981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f60141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1437161981 >> 172.21.15.3:0/1437161981 conn(0x55f1f5f408f0 msgr2=0x55f1f600f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1437161981 shutdown_connections 2023-08-11T14:05:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1437161981 wait complete. 2023-08-11T14:05:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.474+0000 7f92d48a6b80 1 -- start start 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.475+0000 7f92d48a6b80 1 Processor -- start 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.475+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f601d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 0x55f1f60241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5f03a60 con 0x55f1f6012f40 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5f03760 con 0x55f1f6014790 2023-08-11T14:05:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f5de78c0 con 0x55f1f60138a0 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92d48a6b80 10 client.0 _ll_register_callbacks cb 0x55f1f5f06a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36600/0 (socket says 172.21.15.3:36600) 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 -- 172.21.15.3:0/882014420 learned_addr learned my addr 172.21.15.3:0/882014420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92bad9d700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 0x55f1f60241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f601d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 -- 172.21.15.3:0/882014420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 msgr2=0x55f1f601d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f601d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 -- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 msgr2=0x55f1f60241c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 0x55f1f60241c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92ba59c700 1 -- 172.21.15.3:0/882014420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc1a54a0 con 0x55f1f6012f40 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.476+0000 7f92bad9d700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 0x55f1f60241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92ba59c700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f92ac010cc0 tx=0x7f92ac010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5e90384b3e7b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:05:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0984b0 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f92ac0a0740 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1f60067e0 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ac0a0900 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92d48a6b80 10 client.15846 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.477+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f1f5d91d80 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1f60067e0 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f92ac0b5070 con 0x55f1f6012f40 2023-08-11T14:05:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 20 client.15846 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f6028780 0x55f1f602aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 4 client.15846 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1f5fe2480 con 0x55f1f6028780 2023-08-11T14:05:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.478+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f6028780 0x55f1f602aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.479+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f6028780 0x55f1f602aba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d9cb832c19dd27 server_cookie=e88b5a5507855c3d in_seq=0 out_seq=0 2023-08-11T14:05:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.479+0000 7f92a1ffb700 10 client.15846 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92a1ffb700 1 -- 172.21.15.3:0/882014420 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f1f5fe2480 con 0x55f1f6028780 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92a1ffb700 10 client.15846 handle_command_reply: tid=0 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 client.15846 shutdown 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 msgr2=0x55f1f601cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f92ac010cc0 tx=0x7f92ac010aa0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 shutdown_connections 2023-08-11T14:05:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f6028780 0x55f1f602aba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6014790 0x55f1f60241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6012f40 0x55f1f601cc40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/882014420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f60138a0 0x55f1f601d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 >> 172.21.15.3:0/882014420 conn(0x55f1f5f408f0 msgr2=0x55f1f600e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92a1ffb700 0 client.15846 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 shutdown_connections 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.480+0000 7f92d48a6b80 1 -- 172.21.15.3:0/882014420 wait complete. 2023-08-11T14:05:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.481+0000 7f92d48a6b80 20 client.15846 trim_cache size 0 max 16384 2023-08-11T14:05:05.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 -- 172.21.15.3:0/788446237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa140103d30 msgr2=0x7fa140104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 --2- 172.21.15.3:0/788446237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa140103d30 0x7fa140104190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa1340099c0 tx=0x7fa134009cd0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 -- 172.21.15.3:0/788446237 shutdown_connections 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 --2- 172.21.15.3:0/788446237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1401046d0 0x7fa14010af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 --2- 172.21.15.3:0/788446237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa140102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 --2- 172.21.15.3:0/788446237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa140103d30 0x7fa140104190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 -- 172.21.15.3:0/788446237 >> 172.21.15.3:0/788446237 conn(0x7fa1400fe0d0 msgr2=0x7fa1401004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 -- 172.21.15.3:0/788446237 shutdown_connections 2023-08-11T14:05:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.500+0000 7f92d48a6b80 1 Processor -- start 2023-08-11T14:05:05.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.500+0000 7fa1457ff700 1 -- 172.21.15.3:0/788446237 wait complete. 2023-08-11T14:05:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.500+0000 7f92d48a6b80 1 -- start start 2023-08-11T14:05:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f60138a0 0x55f1f6017bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6024180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f6008c90 con 0x55f1f6014790 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f6013fd0 con 0x55f1f60138a0 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f601a910 con 0x55f1f6012f40 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92b9d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f60138a0 0x55f1f6017bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92bad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6024180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40472/0 (socket says 172.21.15.3:40472) 2023-08-11T14:05:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 -- 172.21.15.3:0/3111085634 learned_addr learned my addr 172.21.15.3:0/3111085634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 -- 172.21.15.3:0/3111085634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f60138a0 msgr2=0x55f1f6017bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f60138a0 0x55f1f6017bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 -- 172.21.15.3:0/3111085634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 msgr2=0x55f1f6024180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.501+0000 7f92ba59c700 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6024180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92ba59c700 1 -- 172.21.15.3:0/3111085634 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1f5d7ac00 con 0x55f1f6012f40 2023-08-11T14:05:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92bad9d700 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6024180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:05.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 Processor -- start 2023-08-11T14:05:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92ba59c700 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92ac098ac0 tx=0x7f92ac098b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5463a31ce1465a8b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 -- start start 2023-08-11T14:05:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92a97fa700 1 -- 172.21.15.3:0/3111085634 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00ca10 con 0x55f1f6012f40 2023-08-11T14:05:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92a97fa700 1 -- 172.21.15.3:0/3111085634 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac00ce50 con 0x55f1f6012f40 2023-08-11T14:05:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.502+0000 7f92a97fa700 1 -- 172.21.15.3:0/3111085634 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0982b0 con 0x55f1f6012f40 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa14019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 0x7fa14019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a47e0 con 0x7fa140103d30 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a4950 con 0x7fa1401046d0 2023-08-11T14:05:05.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.502+0000 7fa1457ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401a4b10 con 0x7fa140102b40 2023-08-11T14:05:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38314/0 (socket says 172.21.15.3:38314) 2023-08-11T14:05:05.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 -- 172.21.15.3:0/2975968478 learned_addr learned my addr 172.21.15.3:0/2975968478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 -- 172.21.15.3:0/3111085634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 msgr2=0x55f1f60176b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92ac098ac0 tx=0x7f92ac098b90 comp rx=0 tx=0).stop 2023-08-11T14:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 -- 172.21.15.3:0/3111085634 shutdown_connections 2023-08-11T14:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f6014790 0x55f1f6024180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6012f40 0x55f1f60176b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/3111085634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f60138a0 0x55f1f6017bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 -- 172.21.15.3:0/3111085634 >> 172.21.15.3:0/3111085634 conn(0x55f1f5f408f0 msgr2=0x55f1f601ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 0x7fa14019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 -- 172.21.15.3:0/3111085634 shutdown_connections 2023-08-11T14:05:05.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 -- 172.21.15.3:0/2975968478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 msgr2=0x7fa14019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13effd700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa14019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.503+0000 7f92d48a6b80 1 -- 172.21.15.3:0/3111085634 wait complete. 2023-08-11T14:05:05.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa14019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 -- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 msgr2=0x7fa14019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 0x7fa14019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13f7fe700 1 -- 172.21.15.3:0/2975968478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1340095f0 con 0x7fa1401046d0 2023-08-11T14:05:05.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 0x7fa14019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:05.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.503+0000 7fa13effd700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa14019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.504+0000 7f92d48a6b80 1 -- start start 2023-08-11T14:05:05.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa13800ef30 tx=0x7fa13800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.504+0000 7f92d48a6b80 1 Processor -- start 2023-08-11T14:05:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa127fff700 1 -- 172.21.15.3:0/2975968478 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138018070 con 0x7fa1401046d0 2023-08-11T14:05:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa127fff700 1 -- 172.21.15.3:0/2975968478 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa13800f040 con 0x7fa1401046d0 2023-08-11T14:05:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa127fff700 1 -- 172.21.15.3:0/2975968478 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138003dc0 con 0x7fa1401046d0 2023-08-11T14:05:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1401a4d90 con 0x7fa1401046d0 2023-08-11T14:05:05.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.504+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a51c0 con 0x7fa1401046d0 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f60138a0 0x55f1f6022900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 0x55f1f6022e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f6023590 con 0x55f1f60138a0 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f6023700 con 0x55f1f6012f40 2023-08-11T14:05:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1f60238c0 con 0x55f1f6014790 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92d48a6b80 10 client.0 _ll_register_callbacks cb 0x7f92bc19d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38322/0 (socket says 172.21.15.3:38322) 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 -- 172.21.15.3:0/1166621946 learned_addr learned my addr 172.21.15.3:0/1166621946 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f60138a0 0x55f1f6022900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92bad9d700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 0x55f1f6022e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 -- 172.21.15.3:0/1166621946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 msgr2=0x55f1f6022e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 0x55f1f6022e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 -- 172.21.15.3:0/1166621946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f60138a0 msgr2=0x55f1f6022900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.505+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f60138a0 0x55f1f6022900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92ba59c700 1 -- 172.21.15.3:0/1166621946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1f5d91d80 con 0x55f1f6012f40 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92bad9d700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 0x55f1f6022e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92ba59c700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92ac097110 tx=0x7f92ac0984e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899ce22f0f19c749 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00ca10 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92ac0a0bd0 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f92ac097d20 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92ac095100 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92d48a6b80 10 client.25892 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.506+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f1f5db81f0 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92ac0a0bd0 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f92ac0b5070 con 0x55f1f6012f40 2023-08-11T14:05:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 10 client.25892 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 20 client.25892 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f607cac0 0x55f1f607eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 4 client.25892 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.507+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1f5f5e7c0 con 0x55f1f607cac0 2023-08-11T14:05:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.508+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f607cac0 0x55f1f607eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.508+0000 7fa127fff700 1 -- 172.21.15.3:0/2975968478 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa13800caa0 con 0x7fa1401046d0 2023-08-11T14:05:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.508+0000 7f92b9d9b700 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f607cac0 0x55f1f607eee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3249224df72e97 server_cookie=353c19bb248adaf5 in_seq=0 out_seq=0 2023-08-11T14:05:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.508+0000 7f92a1ffb700 10 client.25892 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.510+0000 7fa127fff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 0x7fa1280a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.510+0000 7fa127fff700 1 -- 172.21.15.3:0/2975968478 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa1380cafc0 con 0x7fa1401046d0 2023-08-11T14:05:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.510+0000 7fa13effd700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 0x7fa1280a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.510+0000 7fa159048b80 1 Processor -- start 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.510+0000 7fa159048b80 1 -- start start 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b56def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b56e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b4c6630 con 0x56117b56e430 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b4c3f50 con 0x56117b56dad0 2023-08-11T14:05:05.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa159048b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b4bf840 con 0x56117b56edf0 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13effd700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 0x7fa1280a2490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa14019db40 tx=0x7fa130009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40510/0 (socket says 172.21.15.3:40510) 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 -- 172.21.15.3:0/464762411 learned_addr learned my addr 172.21.15.3:0/464762411 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b56e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13effd700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b56def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 -- 172.21.15.3:0/464762411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 msgr2=0x56117b56def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b56def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 -- 172.21.15.3:0/464762411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 msgr2=0x56117b56e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b56e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.511+0000 7fa13f7fe700 1 -- 172.21.15.3:0/464762411 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56117b33f630 con 0x56117b56edf0 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.512+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b56e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.512+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1380986b0 tx=0x7fa138098330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f617b6b6c55e4477 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.512+0000 7fa1257fa700 1 -- 172.21.15.3:0/464762411 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138098e20 con 0x56117b56edf0 2023-08-11T14:05:05.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.512+0000 7fa1257fa700 1 -- 172.21.15.3:0/464762411 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa138098720 con 0x56117b56edf0 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.512+0000 7fa1257fa700 1 -- 172.21.15.3:0/464762411 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138096bd0 con 0x56117b56edf0 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 -- 172.21.15.3:0/464762411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 msgr2=0x56117b56f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1380986b0 tx=0x7fa138098330 comp rx=0 tx=0).stop 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 -- 172.21.15.3:0/464762411 shutdown_connections 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b56f270 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b56def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 --2- 172.21.15.3:0/464762411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b56e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 -- 172.21.15.3:0/464762411 >> 172.21.15.3:0/464762411 conn(0x56117b49a950 msgr2=0x56117b568420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 -- 172.21.15.3:0/464762411 shutdown_connections 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.513+0000 7fa159048b80 1 -- 172.21.15.3:0/464762411 wait complete. 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.514+0000 7fa159048b80 1 -- start start 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.514+0000 7fa159048b80 1 Processor -- start 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.514+0000 7fa159048b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b577140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.514+0000 7fa159048b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa159048b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56edf0 0x56117b57e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa159048b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b45dac0 con 0x56117b56edf0 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa159048b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b45d7c0 con 0x56117b56dad0 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa159048b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b341920 con 0x56117b56e430 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa159048b80 10 client.0 _ll_register_callbacks cb 0x56117b460ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa13e7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa13effd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b577140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa13e7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40518/0 (socket says 172.21.15.3:40518) 2023-08-11T14:05:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa13e7fc700 1 -- 172.21.15.3:0/1384771018 learned_addr learned my addr 172.21.15.3:0/1384771018 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.515+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56edf0 0x56117b57e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.516+0000 7fa13e7fc700 1 -- 172.21.15.3:0/1384771018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 msgr2=0x56117b577140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.516+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b577140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.516+0000 7fa13e7fc700 1 -- 172.21.15.3:0/1384771018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56edf0 msgr2=0x56117b57e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.516+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56edf0 0x56117b57e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.516+0000 7fa13e7fc700 1 -- 172.21.15.3:0/1384771018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a51c0 con 0x56117b56e430 2023-08-11T14:05:05.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.517+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa134009fd0 tx=0x7fa13400be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273750afa447ccc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.519+0000 7fa122ff5700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.520+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13401d050 con 0x56117b56e430 2023-08-11T14:05:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.520+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa134005560 con 0x56117b56e430 2023-08-11T14:05:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.520+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56117b560a90 con 0x56117b56e430 2023-08-11T14:05:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.521+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa134005720 con 0x56117b56e430 2023-08-11T14:05:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.521+0000 7fa159048b80 10 client.25900 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.521+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56117b2ebde0 con 0x56117b56e430 2023-08-11T14:05:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56117b560a90 con 0x56117b56e430 2023-08-11T14:05:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa134023ae0 con 0x56117b56e430 2023-08-11T14:05:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 10 client.25900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-08-11T14:05:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 20 client.25900 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b582fe0 0x56117b585400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 4 client.25900 mds_command: new command op to 24511 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.524+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56117b53c4e0 con 0x56117b582fe0 2023-08-11T14:05:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.525+0000 7fa13effd700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b582fe0 0x56117b585400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.526+0000 7fa13effd700 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b582fe0 0x56117b585400 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86b039dd9c8f683a server_cookie=e65d1bfebc41117f in_seq=0 out_seq=0 2023-08-11T14:05:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.526+0000 7fa122ff5700 10 client.25900 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.527+0000 7fa122ff5700 1 -- 172.21.15.3:0/1384771018 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56117b53c4e0 con 0x56117b582fe0 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.527+0000 7fa122ff5700 10 client.25900 handle_command_reply: tid=0 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.527+0000 7fa159048b80 1 client.25900 shutdown 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 msgr2=0x56117b577680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa134009fd0 tx=0x7fa13400be70 comp rx=0 tx=0).stop 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 shutdown_connections 2023-08-11T14:05:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b582fe0 0x56117b585400 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56edf0 0x56117b57e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56dad0 0x56117b577140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 --2- 172.21.15.3:0/1384771018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56e430 0x56117b577680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.528+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 >> 172.21.15.3:0/1384771018 conn(0x56117b49a950 msgr2=0x56117b562cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.529+0000 7fa122ff5700 0 client.25900 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:05:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.529+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 shutdown_connections 2023-08-11T14:05:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.530+0000 7fa159048b80 1 -- 172.21.15.3:0/1384771018 wait complete. 2023-08-11T14:05:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.530+0000 7fa159048b80 20 client.25900 trim_cache size 0 max 16384 2023-08-11T14:05:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.534+0000 7f92a1ffb700 1 -- 172.21.15.3:0/1166621946 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1f5f5e7c0 con 0x55f1f607cac0 2023-08-11T14:05:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.534+0000 7f92a1ffb700 10 client.25892 handle_command_reply: tid=0 2023-08-11T14:05:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.534+0000 7f92d48a6b80 1 client.25892 shutdown 2023-08-11T14:05:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 msgr2=0x55f1f60223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92ac097110 tx=0x7f92ac0984e0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 shutdown_connections 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1f6014790 0x55f1f6022e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f1f607cac0 0x55f1f607eee0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f6012f40 0x55f1f60223c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 --2- 172.21.15.3:0/1166621946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f1f60138a0 0x55f1f6022900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92a1ffb700 0 client.25892 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 >> 172.21.15.3:0/1166621946 conn(0x55f1f5f408f0 msgr2=0x55f1f601ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 shutdown_connections 2023-08-11T14:05:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.535+0000 7f92d48a6b80 1 -- 172.21.15.3:0/1166621946 wait complete. 2023-08-11T14:05:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92d48a6b80 20 client.25892 trim_cache size 0 max 16384 2023-08-11T14:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:05:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 msgr2=0x7f92a40a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 0x7f92a40a2490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92b000b2e0 tx=0x7f92b0005fb0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 msgr2=0x7f92bc19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00e9a0 tx=0x7f92ac00ecb0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 shutdown_connections 2023-08-11T14:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92bc1044d0 0x7f92bc19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f92a409ffd0 0x7f92a40a2490 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92bc0ff190 0x7f92bc19cd90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 --2- 172.21.15.3:0/4083856169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92bc101b00 0x7f92bc19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 >> 172.21.15.3:0/4083856169 conn(0x7f92bc0fad70 msgr2=0x7f92bc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.536+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 shutdown_connections 2023-08-11T14:05:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:05.537+0000 7f92c0fff700 1 -- 172.21.15.3:0/4083856169 wait complete. 2023-08-11T14:05:05.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:05 smithi003 ceph-mon[97195]: pgmap v382: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 116 MiB/s wr, 80 op/s 2023-08-11T14:05:05.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:05 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/473720055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:05:05.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.547+0000 7fa159048b80 1 Processor -- start 2023-08-11T14:05:05.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 -- start start 2023-08-11T14:05:05.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56dad0 0x56117b571d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b57e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 0x56117b572290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b582ad0 con 0x56117b56e430 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b568ed0 con 0x56117b56edf0 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa159048b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b578130 con 0x56117b56dad0 2023-08-11T14:05:05.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa13e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b57e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa13effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56dad0 0x56117b571d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 0x56117b572290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa13e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b57e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36670/0 (socket says 172.21.15.3:36670) 2023-08-11T14:05:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.548+0000 7fa13e7fc700 1 -- 172.21.15.3:0/1526911370 learned_addr learned my addr 172.21.15.3:0/1526911370 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 -- 172.21.15.3:0/1526911370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56dad0 msgr2=0x56117b571d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56dad0 0x56117b571d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 -- 172.21.15.3:0/1526911370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 msgr2=0x56117b57e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b57e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 -- 172.21.15.3:0/1526911370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56117b2d4c60 con 0x56117b56edf0 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 0x56117b572290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa138094c20 tx=0x7fa138096860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbca4449496c2790 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa125ffb700 1 -- 172.21.15.3:0/1526911370 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138098e20 con 0x56117b56edf0 2023-08-11T14:05:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa125ffb700 1 -- 172.21.15.3:0/1526911370 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1380a04a0 con 0x56117b56edf0 2023-08-11T14:05:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.549+0000 7fa125ffb700 1 -- 172.21.15.3:0/1526911370 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13809c120 con 0x56117b56edf0 2023-08-11T14:05:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 -- 172.21.15.3:0/1526911370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 msgr2=0x56117b572290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 0x56117b572290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa138094c20 tx=0x7fa138096860 comp rx=0 tx=0).stop 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 -- 172.21.15.3:0/1526911370 shutdown_connections 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56edf0 0x56117b572290 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56dad0 0x56117b571d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 --2- 172.21.15.3:0/1526911370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56e430 0x56117b57e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 -- 172.21.15.3:0/1526911370 >> 172.21.15.3:0/1526911370 conn(0x56117b49a950 msgr2=0x56117b568280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 -- 172.21.15.3:0/1526911370 shutdown_connections 2023-08-11T14:05:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.550+0000 7fa159048b80 1 -- 172.21.15.3:0/1526911370 wait complete. 2023-08-11T14:05:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.551+0000 7fa159048b80 1 -- start start 2023-08-11T14:05:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.551+0000 7fa159048b80 1 Processor -- start 2023-08-11T14:05:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 0x56117b57a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 0x56117b57a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b57d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b57de10 con 0x56117b56dad0 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b57df80 con 0x56117b56e430 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117b57e140 con 0x56117b56edf0 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa159048b80 10 client.0 _ll_register_callbacks cb 0x7fa1380097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa13effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 0x56117b57a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa13effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 0x56117b57a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36680/0 (socket says 172.21.15.3:36680) 2023-08-11T14:05:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa13effd700 1 -- 172.21.15.3:0/4110461905 learned_addr learned my addr 172.21.15.3:0/4110461905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 0x56117b57a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.552+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b57d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 -- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 msgr2=0x56117b57d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b57d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 -- 172.21.15.3:0/4110461905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 msgr2=0x56117b57a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 0x56117b57a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 -- 172.21.15.3:0/4110461905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56117b33f630 con 0x56117b56e430 2023-08-11T14:05:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13f7fe700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b57d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa13e7fc700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 0x56117b57a8b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa134009fd0 tx=0x7fa134016db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba15c310ed52af0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.553+0000 7fa122ff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:05.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13401d040 con 0x56117b56e430 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa134000e40 con 0x56117b56e430 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa138098720 con 0x56117b56e430 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa134005a00 con 0x56117b56e430 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa159048b80 10 client.25904 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.554+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56117b312250 con 0x56117b56e430 2023-08-11T14:05:05.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa138098720 con 0x56117b56e430 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fa13400fa10 con 0x56117b56e430 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 10 client.25904 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 10 client.25904 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 20 client.25904 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b5d7700 0x56117b5d9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 4 client.25904 mds_command: new command op to 24511 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:05:05.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.555+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 --> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56117b4b8820 con 0x56117b5d7700 2023-08-11T14:05:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa13effd700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b5d7700 0x56117b5d9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa13effd700 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b5d7700 0x56117b5d9b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86512ad6dc647ad2 server_cookie=31ce06d1530be372 in_seq=0 out_seq=0 2023-08-11T14:05:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa122ff5700 10 client.25904 ms_handle_connect on v2:172.21.15.187:6836/1876247524 2023-08-11T14:05:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa122ff5700 1 -- 172.21.15.3:0/4110461905 <== mds.3 v2:172.21.15.187:6836/1876247524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56117b4b8820 con 0x56117b5d7700 2023-08-11T14:05:05.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa122ff5700 10 client.25904 handle_command_reply: tid=0 2023-08-11T14:05:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa159048b80 1 client.25904 shutdown 2023-08-11T14:05:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 msgr2=0x56117b57a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.556+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 0x56117b57a8b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa134009fd0 tx=0x7fa134016db0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 shutdown_connections 2023-08-11T14:05:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:6836/1876247524,v1:172.21.15.187:6837/1876247524] conn(0x56117b5d7700 0x56117b5d9b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56117b56edf0 0x56117b57d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56117b56dad0 0x56117b57a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa122ff5700 0 client.25904 ms_handle_reset on v2:172.21.15.187:6836/1876247524 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 --2- 172.21.15.3:0/4110461905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56117b56e430 0x56117b57a8b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 >> 172.21.15.3:0/4110461905 conn(0x56117b49a950 msgr2=0x56117b571d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 shutdown_connections 2023-08-11T14:05:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 1 -- 172.21.15.3:0/4110461905 wait complete. 2023-08-11T14:05:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.557+0000 7fa159048b80 20 client.25904 trim_cache size 0 max 16384 2023-08-11T14:05:05.571 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:05:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 msgr2=0x7fa1280a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 0x7fa1280a2490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa14019db40 tx=0x7fa130009260 comp rx=0 tx=0).stop 2023-08-11T14:05:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 msgr2=0x7fa1401a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa13800ef30 tx=0x7fa13800c450 comp rx=0 tx=0).stop 2023-08-11T14:05:05.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 shutdown_connections 2023-08-11T14:05:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa12809ffd0 0x7fa1280a2490 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1401046d0 0x7fa1401a4210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa140102b40 0x7fa14019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 --2- 172.21.15.3:0/2975968478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa140103d30 0x7fa14019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 >> 172.21.15.3:0/2975968478 conn(0x7fa1400fe0d0 msgr2=0x7fa140100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 shutdown_connections 2023-08-11T14:05:05.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.558+0000 7fa1457ff700 1 -- 172.21.15.3:0/2975968478 wait complete. 2023-08-11T14:05:05.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-08-11T14:05:05.656 INFO:teuthology.orchestra.run.smithi003.stderr:21400000 of 50000000 tuples (42%) done (elapsed 51.16 s, remaining 68.37 s) 2023-08-11T14:05:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:05 smithi103 ceph-mon[130434]: pgmap v382: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 116 MiB/s wr, 80 op/s 2023-08-11T14:05:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:05 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/473720055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:05:05.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:05 smithi187 ceph-mon[130973]: pgmap v382: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 116 MiB/s wr, 80 op/s 2023-08-11T14:05:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:05 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/473720055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:05:05.849 INFO:teuthology.orchestra.run.smithi003.stderr:21500000 of 50000000 tuples (43%) done (elapsed 51.35 s, remaining 68.07 s) 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4158421087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 msgr2=0x7fdd48103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4158421087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48103780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009990 tx=0x7fdd38009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4158421087 shutdown_connections 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4158421087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4158421087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48103780 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4158421087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd481060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4158421087 >> 172.21.15.3:0/4158421087 conn(0x7fdd480fad80 msgr2=0x7fdd480fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4158421087 shutdown_connections 2023-08-11T14:05:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.894+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4158421087 wait complete. 2023-08-11T14:05:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.896+0000 7fdd4e9ff700 1 Processor -- start 2023-08-11T14:05:05.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.896+0000 7fdd4e9ff700 1 -- start start 2023-08-11T14:05:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.896+0000 7fdd4e9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.896+0000 7fdd4e9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd48195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4e9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48191c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38384/0 (socket says 172.21.15.3:38384) 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 -- 172.21.15.3:0/4147167740 learned_addr learned my addr 172.21.15.3:0/4147167740 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48192430 con 0x7fdd48103cc0 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481925a0 con 0x7fdd48101390 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48192760 con 0x7fdd481065f0 2023-08-11T14:05:05.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd477fe700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd48195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 -- 172.21.15.3:0/4147167740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 msgr2=0x7fdd48191c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48191c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48191c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 -- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 msgr2=0x7fdd48195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd48195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd47fff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd380095f0 con 0x7fdd48101390 2023-08-11T14:05:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd477fe700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd48195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.897+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48191c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b680 tx=0x7fdd38005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd457fa700 1 -- 172.21.15.3:0/4147167740 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3801d070 con 0x7fdd48101390 2023-08-11T14:05:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd481a4d40 con 0x7fdd48101390 2023-08-11T14:05:05.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd457fa700 1 -- 172.21.15.3:0/4147167740 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd3800fbc0 con 0x7fdd48101390 2023-08-11T14:05:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd457fa700 1 -- 172.21.15.3:0/4147167740 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38017650 con 0x7fdd48101390 2023-08-11T14:05:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.898+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd481a5160 con 0x7fdd48101390 2023-08-11T14:05:05.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.902+0000 7fdd457fa700 1 -- 172.21.15.3:0/4147167740 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fdd38022470 con 0x7fdd48101390 2023-08-11T14:05:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.903+0000 7fdd62262b80 1 Processor -- start 2023-08-11T14:05:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.903+0000 7fdd62262b80 1 -- start start 2023-08-11T14:05:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.903+0000 7fdd457fa700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 0x7fdd300a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd457fa700 1 -- 172.21.15.3:0/4147167740 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fdd380ccce0 con 0x7fdd48101390 2023-08-11T14:05:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 0x56035c4e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2430 0x56035c4e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c43a630 con 0x56035c4e2430 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c437f50 con 0x56035c4e2df0 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd62262b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c433840 con 0x56035c4e1ad0 2023-08-11T14:05:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd4cf9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd4cf9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38400/0 (socket says 172.21.15.3:38400) 2023-08-11T14:05:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd4cf9c700 1 -- 172.21.15.3:0/1640841958 learned_addr learned my addr 172.21.15.3:0/1640841958 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.904+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 0x56035c4e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd477fe700 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2430 0x56035c4e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 -- 172.21.15.3:0/1640841958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 msgr2=0x56035c4e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 -- 172.21.15.3:0/1640841958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2430 msgr2=0x56035c4e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2430 0x56035c4e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 -- 172.21.15.3:0/1640841958 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035c2b3630 con 0x56035c4e1ad0 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 0x7fdd300a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 0x56035c4e1ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd38000c00 tx=0x7fdd380057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c66f3b4e47976b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd2e7fc700 1 -- 172.21.15.3:0/1640841958 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38022ea0 con 0x56035c4e1ad0 2023-08-11T14:05:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd2e7fc700 1 -- 172.21.15.3:0/1640841958 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd380a00f0 con 0x56035c4e1ad0 2023-08-11T14:05:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.905+0000 7fdd2e7fc700 1 -- 172.21.15.3:0/1640841958 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd380987e0 con 0x56035c4e1ad0 2023-08-11T14:05:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 -- 172.21.15.3:0/1640841958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 msgr2=0x56035c4e1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 0x56035c4e1ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd38000c00 tx=0x7fdd380057f0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 0x7fdd300a2410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdd400060b0 tx=0x7fdd4000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 -- 172.21.15.3:0/1640841958 shutdown_connections 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e1ad0 0x56035c4e1ef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1640841958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2430 0x56035c4e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.906+0000 7fdd62262b80 1 -- 172.21.15.3:0/1640841958 >> 172.21.15.3:0/1640841958 conn(0x56035c40e950 msgr2=0x56035c4dc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.907+0000 7fdd62262b80 1 -- 172.21.15.3:0/1640841958 shutdown_connections 2023-08-11T14:05:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.907+0000 7fdd62262b80 1 -- 172.21.15.3:0/1640841958 wait complete. 2023-08-11T14:05:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.907+0000 7fdd62262b80 1 -- start start 2023-08-11T14:05:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 Processor -- start 2023-08-11T14:05:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c3d1ac0 con 0x56035c4e2df0 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c3d17c0 con 0x56035c4e1ad0 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c2b5920 con 0x56035c4e2430 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd62262b80 10 client.0 _ll_register_callbacks cb 0x56035c3d4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.908+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38414/0 (socket says 172.21.15.3:38414) 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd47fff700 1 -- 172.21.15.3:0/1695794743 learned_addr learned my addr 172.21.15.3:0/1695794743 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 -- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 msgr2=0x56035c4ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 -- 172.21.15.3:0/1695794743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 msgr2=0x56035c4f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd477fe700 1 -- 172.21.15.3:0/1695794743 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd481a5160 con 0x56035c4e2430 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.909+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd477fe700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4eb2d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c023680 tx=0x7fdd3c023990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe72cda44ca9f4a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd277fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004460 con 0x56035c4e2430 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56035c4d4a90 con 0x56035c4e2430 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fdd3c003680 con 0x56035c4e2430 2023-08-11T14:05:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd3c003840 con 0x56035c4e2430 2023-08-11T14:05:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd62262b80 10 client.25912 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.910+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56035c25fde0 con 0x56035c4e2430 2023-08-11T14:05:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.911+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56035c4d4a90 con 0x56035c4e2430 2023-08-11T14:05:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.911+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdd3c02ca70 con 0x56035c4e2430 2023-08-11T14:05:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 10 client.25912 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 10 client.25912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 20 client.25912 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c4f6b60 0x56035c4f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 4 client.25912 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56035c4b04e0 con 0x56035c4f6b60 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.912+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c4f6b60 0x56035c4f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.913+0000 7fdd47fff700 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c4f6b60 0x56035c4f8f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8824c1b3a9c12f94 server_cookie=c4e4e5a68bb1f015 in_seq=0 out_seq=0 2023-08-11T14:05:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.913+0000 7fdd277fe700 10 client.25912 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.913+0000 7fdd277fe700 1 -- 172.21.15.3:0/1695794743 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56035c4b04e0 con 0x56035c4f6b60 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.913+0000 7fdd277fe700 10 client.25912 handle_command_reply: tid=0 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.913+0000 7fdd62262b80 1 client.25912 shutdown 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 msgr2=0x56035c4eb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4eb2d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c023680 tx=0x7fdd3c023990 comp rx=0 tx=0).stop 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 shutdown_connections 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c4f6b60 0x56035c4f8f80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 --2- 172.21.15.3:0/1695794743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4eb2d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 >> 172.21.15.3:0/1695794743 conn(0x56035c40e950 msgr2=0x56035c4d6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd277fe700 0 client.25912 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 shutdown_connections 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.914+0000 7fdd62262b80 1 -- 172.21.15.3:0/1695794743 wait complete. 2023-08-11T14:05:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.915+0000 7fdd62262b80 20 client.25912 trim_cache size 0 max 16384 2023-08-11T14:05:05.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 Processor -- start 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 -- start start 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e1ad0 0x56035c4e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4d8de0 con 0x56035c4e1ad0 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4ddae0 con 0x56035c4e2df0 2023-08-11T14:05:05.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd62262b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4ebee0 con 0x56035c4e2430 2023-08-11T14:05:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd4cf9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd4cf9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38420/0 (socket says 172.21.15.3:38420) 2023-08-11T14:05:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd4cf9c700 1 -- 172.21.15.3:0/3923621720 learned_addr learned my addr 172.21.15.3:0/3923621720 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.938+0000 7fdd47fff700 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e1ad0 0x56035c4e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd477fe700 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 -- 172.21.15.3:0/3923621720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 msgr2=0x56035c4f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 -- 172.21.15.3:0/3923621720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e1ad0 msgr2=0x56035c4e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e1ad0 0x56035c4e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 -- 172.21.15.3:0/3923621720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56035c248c60 con 0x56035c4e2df0 2023-08-11T14:05:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.939+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009430 tx=0x7fdd40009070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e40e581d45814b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.940+0000 7fdd2effd700 1 -- 172.21.15.3:0/3923621720 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4000e030 con 0x56035c4e2df0 2023-08-11T14:05:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.940+0000 7fdd2effd700 1 -- 172.21.15.3:0/3923621720 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4001e050 con 0x56035c4e2df0 2023-08-11T14:05:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.940+0000 7fdd2effd700 1 -- 172.21.15.3:0/3923621720 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40018650 con 0x56035c4e2df0 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 -- 172.21.15.3:0/3923621720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 msgr2=0x56035c4e31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009430 tx=0x7fdd40009070 comp rx=0 tx=0).stop 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 -- 172.21.15.3:0/3923621720 shutdown_connections 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e2df0 0x56035c4e31d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e1ad0 0x56035c4e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 --2- 172.21.15.3:0/3923621720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.941+0000 7fdd62262b80 1 -- 172.21.15.3:0/3923621720 >> 172.21.15.3:0/3923621720 conn(0x56035c40e950 msgr2=0x56035c4e9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.942+0000 7fdd62262b80 1 -- 172.21.15.3:0/3923621720 shutdown_connections 2023-08-11T14:05:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.942+0000 7fdd62262b80 1 -- 172.21.15.3:0/3923621720 wait complete. 2023-08-11T14:05:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.942+0000 7fdd62262b80 1 -- start start 2023-08-11T14:05:05.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.943+0000 7fdd62262b80 1 Processor -- start 2023-08-11T14:05:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4f1570 con 0x56035c4e2df0 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4f16e0 con 0x56035c4e1ad0 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56035c4f18a0 con 0x56035c4e2430 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd62262b80 10 client.0 _ll_register_callbacks cb 0x7fdd3c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd477fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38436/0 (socket says 172.21.15.3:38436) 2023-08-11T14:05:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 -- 172.21.15.3:0/4059834913 learned_addr learned my addr 172.21.15.3:0/4059834913 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd4cf9c700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 -- 172.21.15.3:0/4059834913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 msgr2=0x56035c4f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 -- 172.21.15.3:0/4059834913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 msgr2=0x56035c4f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.947+0000 7fdd47fff700 1 -- 172.21.15.3:0/4059834913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd400090e0 con 0x56035c4e1ad0 2023-08-11T14:05:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd47fff700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd3809a140 tx=0x7fdd38005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c7c7688ecd7d2c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd277fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3809e270 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fdd3800fd30 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd380a00f0 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd380987e0 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.948+0000 7fdd62262b80 10 client.25922 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56035c286250 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd380a00f0 con 0x56035c4e1ad0 2023-08-11T14:05:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdd380a1d30 con 0x56035c4e1ad0 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 10 client.25922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 20 client.25922 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c55bf70 0x56035c55e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 4 client.25922 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.949+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 --> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56035c42c820 con 0x56035c55bf70 2023-08-11T14:05:05.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd477fe700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c55bf70 0x56035c55e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd277fe700 10 client.25922 ms_handle_connect on v2:172.21.15.103:6836/4115864783 2023-08-11T14:05:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd477fe700 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c55bf70 0x56035c55e390 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b133601e830c2745 server_cookie=2b49dc2b4f06626 in_seq=0 out_seq=0 2023-08-11T14:05:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd277fe700 1 -- 172.21.15.3:0/4059834913 <== mds.2 v2:172.21.15.103:6836/4115864783 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56035c42c820 con 0x56035c55bf70 2023-08-11T14:05:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd277fe700 10 client.25922 handle_command_reply: tid=0 2023-08-11T14:05:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.950+0000 7fdd62262b80 1 client.25922 shutdown 2023-08-11T14:05:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 msgr2=0x56035c4ee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd3809a140 tx=0x7fdd38005ad0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 shutdown_connections 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:6836/4115864783,v1:172.21.15.103:6837/4115864783] conn(0x56035c55bf70 0x56035c55e390 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56035c4e2df0 0x56035c4f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035c4e1ad0 0x56035c4ee2b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd277fe700 0 client.25922 ms_handle_reset on v2:172.21.15.103:6836/4115864783 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 --2- 172.21.15.3:0/4059834913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56035c4e2430 0x56035c4f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 >> 172.21.15.3:0/4059834913 conn(0x56035c40e950 msgr2=0x56035c4e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 shutdown_connections 2023-08-11T14:05:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 1 -- 172.21.15.3:0/4059834913 wait complete. 2023-08-11T14:05:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.951+0000 7fdd62262b80 20 client.25922 trim_cache size 0 max 16384 2023-08-11T14:05:05.958 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:05:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 msgr2=0x7fdd300a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 0x7fdd300a2410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdd400060b0 tx=0x7fdd4000b040 comp rx=0 tx=0).stop 2023-08-11T14:05:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 msgr2=0x7fdd48195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b680 tx=0x7fdd38005ef0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 shutdown_connections 2023-08-11T14:05:05.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd481065f0 0x7fdd48191c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fdd3009ff50 0x7fdd300a2410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd48101390 0x7fdd48195100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 --2- 172.21.15.3:0/4147167740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd48103cc0 0x7fdd48195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:05.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 >> 172.21.15.3:0/4147167740 conn(0x7fdd480fad80 msgr2=0x7fdd480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:05.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 shutdown_connections 2023-08-11T14:05:05.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:05.952+0000 7fdd4e9ff700 1 -- 172.21.15.3:0/4147167740 wait complete. 2023-08-11T14:05:05.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-08-11T14:05:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:21600000 of 50000000 tuples (43%) done (elapsed 51.85 s, remaining 68.17 s) 2023-08-11T14:05:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 -- 172.21.15.3:0/3936928299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364106f40 msgr2=0x7fd364107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/3936928299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364106f40 0x7fd364107360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a9a0 tx=0x7fd34c01acb0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 -- 172.21.15.3:0/3936928299 shutdown_connections 2023-08-11T14:05:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/3936928299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108a50 0x7fd36410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/3936928299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364106f40 0x7fd364107360 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a9a0 tx=0x7fd34c01acb0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/3936928299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364108090 0x7fd364108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.353+0000 7fd36a3ff700 1 -- 172.21.15.3:0/3936928299 >> 172.21.15.3:0/3936928299 conn(0x7fd364076280 msgr2=0x7fd3640786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.354+0000 7fd36a3ff700 1 -- 172.21.15.3:0/3936928299 shutdown_connections 2023-08-11T14:05:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.354+0000 7fd36a3ff700 1 -- 172.21.15.3:0/3936928299 wait complete. 2023-08-11T14:05:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 Processor -- start 2023-08-11T14:05:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 -- start start 2023-08-11T14:05:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 0x7fd364199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364199920 0x7fd36419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.356+0000 7fd36a3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36419c550 con 0x7fd364199920 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd36a3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640791a0 con 0x7fd364108090 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd36a3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364079360 con 0x7fd364108a50 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38454/0 (socket says 172.21.15.3:38454) 2023-08-11T14:05:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 -- 172.21.15.3:0/1101617382 learned_addr learned my addr 172.21.15.3:0/1101617382 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd36899c700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364199920 0x7fd36419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 0x7fd364199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 -- 172.21.15.3:0/1101617382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 msgr2=0x7fd364199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 0x7fd364199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 -- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364199920 msgr2=0x7fd36419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364199920 0x7fd36419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.357+0000 7fd363fff700 1 -- 172.21.15.3:0/1101617382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c01a600 con 0x7fd364108090 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 0x7fd364199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd363fff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a8f0 tx=0x7fd34c01a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd361ffb700 1 -- 172.21.15.3:0/1101617382 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c01a070 con 0x7fd364108090 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd361ffb700 1 -- 172.21.15.3:0/1101617382 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34c032420 con 0x7fd364108090 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd361ffb700 1 -- 172.21.15.3:0/1101617382 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c028650 con 0x7fd364108090 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640795e0 con 0x7fd364108090 2023-08-11T14:05:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.358+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd364079aa0 con 0x7fd364108090 2023-08-11T14:05:06.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.362+0000 7fd361ffb700 1 -- 172.21.15.3:0/1101617382 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd34c032590 con 0x7fd364108090 2023-08-11T14:05:06.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.365+0000 7fd361ffb700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 0x7fd3500a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.365+0000 7fd37dcabb80 1 Processor -- start 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.365+0000 7fd361ffb700 1 -- 172.21.15.3:0/1101617382 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd34c0dd680 con 0x7fd364108090 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.366+0000 7fd37dcabb80 1 -- start start 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.366+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.366+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 0x563aba2ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2cedf0 0x563aba2cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba226630 con 0x563aba2cdad0 2023-08-11T14:05:06.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba223f50 con 0x563aba2ce430 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba21f840 con 0x563aba2cedf0 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 0x7fd3500a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd363fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd363fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36782/0 (socket says 172.21.15.3:36782) 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd363fff700 1 -- 172.21.15.3:0/4127873094 learned_addr learned my addr 172.21.15.3:0/4127873094 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.367+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 0x563aba2ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd36899c700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2cedf0 0x563aba2cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 -- 172.21.15.3:0/4127873094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2cedf0 msgr2=0x563aba2cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2cedf0 0x563aba2cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 -- 172.21.15.3:0/4127873094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 msgr2=0x563aba2ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 0x563aba2ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 -- 172.21.15.3:0/4127873094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aba09f630 con 0x563aba2cdad0 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 0x563aba2ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.368+0000 7fd363fff700 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0050b0 tx=0x7fd34c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06c4815683298df server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.369+0000 7fd35a7fc700 1 -- 172.21.15.3:0/4127873094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0aade0 con 0x563aba2cdad0 2023-08-11T14:05:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.369+0000 7fd35a7fc700 1 -- 172.21.15.3:0/4127873094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34c0aec10 con 0x563aba2cdad0 2023-08-11T14:05:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.369+0000 7fd35a7fc700 1 -- 172.21.15.3:0/4127873094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0daf10 con 0x563aba2cdad0 2023-08-11T14:05:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.370+0000 7fd37dcabb80 1 -- 172.21.15.3:0/4127873094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 msgr2=0x563aba2cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.370+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0050b0 tx=0x7fd34c032c30 comp rx=0 tx=0).stop 2023-08-11T14:05:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 0x7fd3500a2560 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd354009fd0 tx=0x7fd3540096f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd37dcabb80 1 -- 172.21.15.3:0/4127873094 shutdown_connections 2023-08-11T14:05:06.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2cedf0 0x563aba2cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cdad0 0x563aba2cdef0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/4127873094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2ce430 0x563aba2ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.371+0000 7fd37dcabb80 1 -- 172.21.15.3:0/4127873094 >> 172.21.15.3:0/4127873094 conn(0x563aba1fa950 msgr2=0x563aba2c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.374+0000 7fd37dcabb80 1 -- 172.21.15.3:0/4127873094 shutdown_connections 2023-08-11T14:05:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.374+0000 7fd37dcabb80 1 -- 172.21.15.3:0/4127873094 wait complete. 2023-08-11T14:05:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.375+0000 7fd37dcabb80 1 -- start start 2023-08-11T14:05:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.377+0000 7fd37dcabb80 1 Processor -- start 2023-08-11T14:05:06.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba1bdac0 con 0x563aba2cedf0 2023-08-11T14:05:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba1bd7c0 con 0x563aba2cdad0 2023-08-11T14:05:06.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba0a1920 con 0x563aba2ce430 2023-08-11T14:05:06.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.378+0000 7fd37dcabb80 10 client.0 _ll_register_callbacks cb 0x563aba1c0ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:06.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.379+0000 7fd36899c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.379+0000 7fd36899c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36790/0 (socket says 172.21.15.3:36790) 2023-08-11T14:05:06.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.379+0000 7fd36899c700 1 -- 172.21.15.3:0/273237461 learned_addr learned my addr 172.21.15.3:0/273237461 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.380+0000 7fd36899c700 1 -- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 msgr2=0x563aba2d73c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:05:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.380+0000 7fd36899c700 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2d73c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.380+0000 7fd36899c700 1 -- 172.21.15.3:0/273237461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 msgr2=0x563aba2d6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.380+0000 7fd36899c700 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.380+0000 7fd36899c700 1 -- 172.21.15.3:0/273237461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd364079aa0 con 0x563aba2cedf0 2023-08-11T14:05:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.381+0000 7fd36899c700 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007ba0 tx=0x7fd35c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a279a99b38f80ba7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.381+0000 7fd3437fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:05:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.381+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00e040 con 0x563aba2cedf0 2023-08-11T14:05:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.381+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd35c004bb0 con 0x563aba2cedf0 2023-08-11T14:05:06.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.381+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aba2c0a90 con 0x563aba2cedf0 2023-08-11T14:05:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.382+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd35c004d90 con 0x563aba2cedf0 2023-08-11T14:05:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.382+0000 7fd37dcabb80 10 client.15894 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:06.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.382+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563aba04bde0 con 0x563aba2cedf0 2023-08-11T14:05:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.383+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563aba2c0a90 con 0x563aba2cedf0 2023-08-11T14:05:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.383+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd35c016a80 con 0x563aba2cedf0 2023-08-11T14:05:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-08-11T14:05:06.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 20 client.15894 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba2e2cd0 0x563aba2e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 4 client.15894 mds_command: new command op to 24532 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aba29c4e0 con 0x563aba2e2cd0 2023-08-11T14:05:06.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.384+0000 7fd363fff700 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba2e2cd0 0x563aba2e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.385+0000 7fd363fff700 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba2e2cd0 0x563aba2e50f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bc31eb9ff21c68c server_cookie=23146cca179377ac in_seq=0 out_seq=0 2023-08-11T14:05:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.385+0000 7fd3437fe700 10 client.15894 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:05:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.386+0000 7fd3437fe700 1 -- 172.21.15.3:0/273237461 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563aba29c4e0 con 0x563aba2e2cd0 2023-08-11T14:05:06.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.386+0000 7fd3437fe700 10 client.15894 handle_command_reply: tid=0 2023-08-11T14:05:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.386+0000 7fd37dcabb80 1 client.15894 shutdown 2023-08-11T14:05:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.386+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 msgr2=0x563aba2de400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.386+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007ba0 tx=0x7fd35c007c80 comp rx=0 tx=0).stop 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 shutdown_connections 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba2e2cd0 0x563aba2e50f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2de400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2d6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/273237461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 >> 172.21.15.3:0/273237461 conn(0x563aba1fa950 msgr2=0x563aba2c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd3437fe700 0 client.15894 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:05:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 shutdown_connections 2023-08-11T14:05:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.387+0000 7fd37dcabb80 1 -- 172.21.15.3:0/273237461 wait complete. 2023-08-11T14:05:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.388+0000 7fd37dcabb80 20 client.15894 trim_cache size 0 max 16384 2023-08-11T14:05:06.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 Processor -- start 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 -- start start 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2c4de0 con 0x563aba2cedf0 2023-08-11T14:05:06.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2c9510 con 0x563aba2cdad0 2023-08-11T14:05:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.409+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2e00c0 con 0x563aba2ce430 2023-08-11T14:05:06.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40630/0 (socket says 172.21.15.3:40630) 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 -- 172.21.15.3:0/2525834443 learned_addr learned my addr 172.21.15.3:0/2525834443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd36899c700 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 -- 172.21.15.3:0/2525834443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 msgr2=0x563aba2e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 -- 172.21.15.3:0/2525834443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 msgr2=0x563aba2decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.410+0000 7fd35b7fe700 1 -- 172.21.15.3:0/2525834443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aba034c60 con 0x563aba2ce430 2023-08-11T14:05:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.411+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd354009aa0 tx=0x7fd3540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d409196ff2cc77e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.411+0000 7fd35affd700 1 -- 172.21.15.3:0/2525834443 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540373c0 con 0x563aba2ce430 2023-08-11T14:05:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.411+0000 7fd35affd700 1 -- 172.21.15.3:0/2525834443 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd354034cf0 con 0x563aba2ce430 2023-08-11T14:05:06.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.411+0000 7fd35affd700 1 -- 172.21.15.3:0/2525834443 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354043d70 con 0x563aba2ce430 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- 172.21.15.3:0/2525834443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 msgr2=0x563aba2de3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd354009aa0 tx=0x7fd3540047c0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- 172.21.15.3:0/2525834443 shutdown_connections 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/2525834443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2de3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- 172.21.15.3:0/2525834443 >> 172.21.15.3:0/2525834443 conn(0x563aba1fa950 msgr2=0x563aba2d65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- 172.21.15.3:0/2525834443 shutdown_connections 2023-08-11T14:05:06.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- 172.21.15.3:0/2525834443 wait complete. 2023-08-11T14:05:06.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.412+0000 7fd37dcabb80 1 -- start start 2023-08-11T14:05:06.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 Processor -- start 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2dd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2ddd50 con 0x563aba2cedf0 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2ddec0 con 0x563aba2cdad0 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aba2de080 con 0x563aba2ce430 2023-08-11T14:05:06.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.413+0000 7fd37dcabb80 10 client.0 _ll_register_callbacks cb 0x7fd35c007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40632/0 (socket says 172.21.15.3:40632) 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 -- 172.21.15.3:0/149217291 learned_addr learned my addr 172.21.15.3:0/149217291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd363fff700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 -- 172.21.15.3:0/149217291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 msgr2=0x563aba2da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 -- 172.21.15.3:0/149217291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 msgr2=0x563aba2dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 -- 172.21.15.3:0/149217291 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35403f040 con 0x563aba2ce430 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd35b7fe700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd354043870 tx=0x7fd354034b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a27bcd8a52060ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.414+0000 7fd3437fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354043420 con 0x563aba2ce430 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd34c0aec10 con 0x563aba2ce430 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd3540373c0 con 0x563aba2ce430 2023-08-11T14:05:06.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd354043620 con 0x563aba2ce430 2023-08-11T14:05:06.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd37dcabb80 10 client.25930 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:06.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.415+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563aba072250 con 0x563aba2ce430 2023-08-11T14:05:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34c0aec10 con 0x563aba2ce430 2023-08-11T14:05:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fd35404c450 con 0x563aba2ce430 2023-08-11T14:05:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd37dcabb80 10 client.25930 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd37dcabb80 10 client.25930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd37dcabb80 20 client.25930 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba337af0 0x563aba339f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.416+0000 7fd37dcabb80 4 client.25930 mds_command: new command op to 24532 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.417+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 --> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563aba218820 con 0x563aba337af0 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.417+0000 7fd363fff700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba337af0 0x563aba339f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.417+0000 7fd363fff700 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba337af0 0x563aba339f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb382d4a3d6db8cd server_cookie=7507443a8c0568b2 in_seq=0 out_seq=0 2023-08-11T14:05:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.417+0000 7fd3437fe700 10 client.25930 ms_handle_connect on v2:172.21.15.187:6838/1685844974 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd3437fe700 1 -- 172.21.15.3:0/149217291 <== mds.1 v2:172.21.15.187:6838/1685844974 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25183 (crc 0 0 0) 0x563aba218820 con 0x563aba337af0 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd3437fe700 10 client.25930 handle_command_reply: tid=0 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 client.25930 shutdown 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 msgr2=0x563aba2da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd354043870 tx=0x7fd354034b10 comp rx=0 tx=0).stop 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 shutdown_connections 2023-08-11T14:05:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:6838/1685844974,v1:172.21.15.187:6839/1685844974] conn(0x563aba337af0 0x563aba339f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563aba2cedf0 0x563aba2dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563aba2cdad0 0x563aba2da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 --2- 172.21.15.3:0/149217291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563aba2ce430 0x563aba2da7f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 >> 172.21.15.3:0/149217291 conn(0x563aba1fa950 msgr2=0x563aba2d1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.418+0000 7fd3437fe700 0 client.25930 ms_handle_reset on v2:172.21.15.187:6838/1685844974 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.419+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 shutdown_connections 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.419+0000 7fd37dcabb80 1 -- 172.21.15.3:0/149217291 wait complete. 2023-08-11T14:05:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.419+0000 7fd37dcabb80 20 client.25930 trim_cache size 0 max 16384 2023-08-11T14:05:06.425 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-08-11T14:05:06.425 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.425 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.425 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 37221692, 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629291, 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init" 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 47224092, 2023-08-11T14:05:06.426 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629224, 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691" 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 47898387, 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628327, 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000" 2023-08-11T14:05:06.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 79860770, 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628323, 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control" 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 93409659, 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629214, 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712" 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.429 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 145181039, 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629212, 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655" 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 171378835, 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629233, 2023-08-11T14:05:06.430 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603" 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 184949457, 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628455, 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695" 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 187896587, 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627795, 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets" 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.432 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 264205600, 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629202, 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm" 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 294516734, 2023-08-11T14:05:06.433 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628435, 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847" 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 335267248, 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629211, 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702" 2023-08-11T14:05:06.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 361129383, 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629229, 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838" 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.435 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 379563255, 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628439, 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061" 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.436 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 380152640, 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628452, 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676" 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 385550828, 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627779, 2023-08-11T14:05:06.437 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388" 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 412698461, 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629236, 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm" 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.438 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 577886125, 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628609, 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm" 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.439 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 663829480, 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628552, 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114" 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 670207408, 2023-08-11T14:05:06.440 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628505, 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232" 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 692513702, 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629295, 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393" 2023-08-11T14:05:06.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 705426549, 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628403, 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592" 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.442 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 824725268, 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628413, 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100" 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 842337898, 2023-08-11T14:05:06.443 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628453, 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677" 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 856050291, 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628676, 2023-08-11T14:05:06.444 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm" 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 862902496, 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628506, 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233" 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.445 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 863786518, 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628376, 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136" 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.446 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 869439095, 2023-08-11T14:05:06.447 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629228, 2023-08-11T14:05:06.447 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663" 2023-08-11T14:05:06.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.447 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.447 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 898258250, 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629297, 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm" 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 903627480, 2023-08-11T14:05:06.448 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629208, 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668" 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 943744433, 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629209, 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm" 2023-08-11T14:05:06.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 945148456, 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628583, 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm" 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.450 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1009190267, 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628619, 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm" 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.451 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1012219690, 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628373, 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964" 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.452 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1040273801, 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629234, 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764" 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1075258953, 2023-08-11T14:05:06.453 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628537, 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965" 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1112158257, 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629204, 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467" 2023-08-11T14:05:06.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1124869463, 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629232, 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247" 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.455 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1189540869, 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628546, 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002" 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.456 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1305817988, 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628324, 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000" 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1319087724, 2023-08-11T14:05:06.457 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629218, 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164" 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1407902751, 2023-08-11T14:05:06.458 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628377, 2023-08-11T14:05:06.459 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm" 2023-08-11T14:05:06.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.459 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.463 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.463 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1434321753, 2023-08-11T14:05:06.464 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628566, 2023-08-11T14:05:06.464 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm" 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1435225946, 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628434, 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846" 2023-08-11T14:05:06.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.466 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.466 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.466 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1487719064, 2023-08-11T14:05:06.466 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628379, 2023-08-11T14:05:06.466 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm" 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1489925564, 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628382, 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396" 2023-08-11T14:05:06.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1588463786, 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629219, 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm" 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.468 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1618266806, 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629203, 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575" 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.469 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1730787422, 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629289, 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat" 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1809850571, 2023-08-11T14:05:06.470 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629240, 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm" 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1813454236, 2023-08-11T14:05:06.471 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628539, 2023-08-11T14:05:06.472 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593" 2023-08-11T14:05:06.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.472 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.472 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1877206482, 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629285, 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000" 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.473 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1893918535, 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629215, 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608" 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 1981833737, 2023-08-11T14:05:06.474 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629287, 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log" 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2000396982, 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628675, 2023-08-11T14:05:06.475 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm" 2023-08-11T14:05:06.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.476 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.476 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.476 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2009805970, 2023-08-11T14:05:06.476 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629222, 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113" 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2055544194, 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629290, 2023-08-11T14:05:06.477 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init" 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2064530221, 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629221, 2023-08-11T14:05:06.478 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682" 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2145784339, 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628378, 2023-08-11T14:05:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260" 2023-08-11T14:05:06.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.480 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.480 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.480 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2162057774, 2023-08-11T14:05:06.480 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628436, 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966" 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2195304920, 2023-08-11T14:05:06.481 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629223, 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604" 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2235325653, 2023-08-11T14:05:06.482 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629237, 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754" 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2309627812, 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627794, 2023-08-11T14:05:06.483 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members" 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2340440622, 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629220, 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111" 2023-08-11T14:05:06.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2370009688, 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628381, 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214" 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.485 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2370757710, 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628560, 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm" 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.486 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2384832646, 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629241, 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259" 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2393600460, 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628438, 2023-08-11T14:05:06.487 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060" 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2436846549, 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629244, 2023-08-11T14:05:06.488 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606" 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2466993924, 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629216, 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831" 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.489 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2479609972, 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628475, 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397" 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.490 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2552325905, 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628617, 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm" 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2619508485, 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628470, 2023-08-11T14:05:06.491 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671" 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2672219054, 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629230, 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836" 2023-08-11T14:05:06.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2713182625, 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627780, 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_0/.meta" 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.493 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2819983223, 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629227, 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685" 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.494 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2923613342, 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628545, 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001" 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2926683809, 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629245, 2023-08-11T14:05:06.495 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613" 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2929217585, 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628325, 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000" 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.496 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 2982137391, 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629217, 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm" 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.497 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3000523535, 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628374, 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213" 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.498 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3065061800, 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628603, 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm" 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3065430256, 2023-08-11T14:05:06.499 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628581, 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm" 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3087424254, 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628437, 2023-08-11T14:05:06.500 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967" 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3088885316, 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629238, 2023-08-11T14:05:06.501 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199" 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3142137616, 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629225, 2023-08-11T14:05:06.502 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm" 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3196603362, 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628509, 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698" 2023-08-11T14:05:06.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3200383143, 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628471, 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672" 2023-08-11T14:05:06.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3222237151, 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628629, 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map" 2023-08-11T14:05:06.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3298054286, 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629207, 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674" 2023-08-11T14:05:06.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.507 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.507 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.507 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3337320886, 2023-08-11T14:05:06.507 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628371, 2023-08-11T14:05:06.507 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262" 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3401711241, 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627804, 2023-08-11T14:05:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots" 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3404121910, 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628326, 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000" 2023-08-11T14:05:06.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3425400516, 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628553, 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115" 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.510 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3456439680, 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629242, 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207" 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3463964084, 2023-08-11T14:05:06.511 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629231, 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548" 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3479506675, 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628399, 2023-08-11T14:05:06.512 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000" 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3480102253, 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629243, 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610" 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.513 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3486648913, 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628610, 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm" 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.514 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3511062747, 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628587, 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm" 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3526545898, 2023-08-11T14:05:06.515 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628508, 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697" 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3564465358, 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628556, 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm" 2023-08-11T14:05:06.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3608305195, 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629210, 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110" 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.517 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3672278187, 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628618, 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm" 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.518 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3716965156, 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628600, 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm" 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3737238258, 2023-08-11T14:05:06.519 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629226, 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440" 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3948353003, 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628499, 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137" 2023-08-11T14:05:06.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 3953142008, 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628380, 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261" 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4058882836, 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629239, 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455" 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.522 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4059596266, 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511627805, 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings" 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4111413533, 2023-08-11T14:05:06.523 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629206, 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119" 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4137408644, 2023-08-11T14:05:06.524 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628564, 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm" 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4144499169, 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629205, 2023-08-11T14:05:06.525 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm" 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4166520507, 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628372, 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm" 2023-08-11T14:05:06.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4173354683, 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628677, 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm" 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.527 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4195827933, 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628375, 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm" 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.528 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4205932086, 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629213, 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597" 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4248481033, 2023-08-11T14:05:06.529 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628454, 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694" 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4248500185, 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511629235, 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080" 2023-08-11T14:05:06.530 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:05:06.531 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-08-11T14:05:06.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 msgr2=0x7fd3500a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 0x7fd3500a2560 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd354009fd0 tx=0x7fd3540096f0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 msgr2=0x7fd364196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a8f0 tx=0x7fd34c01a590 comp rx=0 tx=0).stop 2023-08-11T14:05:06.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 shutdown_connections 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd364199920 0x7fd36419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd3500a00a0 0x7fd3500a2560 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd364108090 0x7fd364196df0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 --2- 172.21.15.3:0/1101617382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd364108a50 0x7fd364199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.420+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 >> 172.21.15.3:0/1101617382 conn(0x7fd364076280 msgr2=0x7fd364078580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.421+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 shutdown_connections 2023-08-11T14:05:06.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.421+0000 7fd36a3ff700 1 -- 172.21.15.3:0/1101617382 wait complete. 2023-08-11T14:05:06.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-08-11T14:05:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:21700000 of 50000000 tuples (43%) done (elapsed 52.05 s, remaining 67.88 s) 2023-08-11T14:05:06.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:06 smithi103 ceph-mon[130434]: pgmap v383: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 100 MiB/s wr, 61 op/s 2023-08-11T14:05:06.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:06 smithi187 ceph-mon[130973]: pgmap v383: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 100 MiB/s wr, 61 op/s 2023-08-11T14:05:06.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:06 smithi003 ceph-mon[97195]: pgmap v383: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 100 MiB/s wr, 61 op/s 2023-08-11T14:05:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 -- 172.21.15.3:0/3041686182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 msgr2=0x7f0534100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 --2- 172.21.15.3:0/3041686182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534100cd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0528009990 tx=0x7f0528009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 -- 172.21.15.3:0/3041686182 shutdown_connections 2023-08-11T14:05:06.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 --2- 172.21.15.3:0/3041686182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f0534107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 --2- 172.21.15.3:0/3041686182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 0x7f05340ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 --2- 172.21.15.3:0/3041686182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534100cd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.853+0000 7f0538d9d700 1 -- 172.21.15.3:0/3041686182 >> 172.21.15.3:0/3041686182 conn(0x7f05340fabd0 msgr2=0x7f05340fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.854+0000 7f0538d9d700 1 -- 172.21.15.3:0/3041686182 shutdown_connections 2023-08-11T14:05:06.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.854+0000 7f0538d9d700 1 -- 172.21.15.3:0/3041686182 wait complete. 2023-08-11T14:05:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.855+0000 7f0538d9d700 1 Processor -- start 2023-08-11T14:05:06.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.855+0000 7f0538d9d700 1 -- start start 2023-08-11T14:05:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 0x7f05341985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f053419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341a0130 con 0x7f0534101210 2023-08-11T14:05:06.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341a0270 con 0x7f0534100870 2023-08-11T14:05:06.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.856+0000 7f0538d9d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341a0400 con 0x7f05340ff680 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f05337fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 0x7f05341985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38504/0 (socket says 172.21.15.3:38504) 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 -- 172.21.15.3:0/1999814476 learned_addr learned my addr 172.21.15.3:0/1999814476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0533fff700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f053419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 -- 172.21.15.3:0/1999814476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 msgr2=0x7f05341985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 0x7f05341985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 -- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 msgr2=0x7f053419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f053419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 -- 172.21.15.3:0/1999814476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280095f0 con 0x7f0534100870 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0533fff700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f053419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.857+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0528005b40 tx=0x7f052800f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.858+0000 7f0530ff9700 1 -- 172.21.15.3:0/1999814476 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052801d070 con 0x7f0534100870 2023-08-11T14:05:06.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.858+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05341a0680 con 0x7f0534100870 2023-08-11T14:05:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.858+0000 7f0530ff9700 1 -- 172.21.15.3:0/1999814476 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0528005da0 con 0x7f0534100870 2023-08-11T14:05:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.858+0000 7f0530ff9700 1 -- 172.21.15.3:0/1999814476 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05280049d0 con 0x7f0534100870 2023-08-11T14:05:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.858+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341a0b00 con 0x7f0534100870 2023-08-11T14:05:06.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.861+0000 7f0530ff9700 1 -- 172.21.15.3:0/1999814476 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f05280052c0 con 0x7f0534100870 2023-08-11T14:05:06.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.863+0000 7f054d8a9b80 1 Processor -- start 2023-08-11T14:05:06.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 -- start start 2023-08-11T14:05:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x556174396950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561742ee630 con 0x556174396470 2023-08-11T14:05:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561742ebf50 con 0x556174395b10 2023-08-11T14:05:06.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.864+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561742e7840 con 0x556174396e90 2023-08-11T14:05:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.865+0000 7f0533fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.865+0000 7f0533fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38514/0 (socket says 172.21.15.3:38514) 2023-08-11T14:05:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.865+0000 7f0533fff700 1 -- 172.21.15.3:0/3833590106 learned_addr learned my addr 172.21.15.3:0/3833590106 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.866+0000 7f0533fff700 1 -- 172.21.15.3:0/3833590106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 msgr2=0x5561743972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.868+0000 7f05337fe700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x556174396950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.869+0000 7f0532ffd700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.869+0000 7f0530ff9700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 0x7f051c0a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 -- 172.21.15.3:0/3833590106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 msgr2=0x556174396950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x556174396950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 -- 172.21.15.3:0/3833590106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556174167630 con 0x556174395b10 2023-08-11T14:05:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f05337fe700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x556174396950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:06.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0530ff9700 1 -- 172.21.15.3:0/1999814476 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f05280cc860 con 0x7f0534100870 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f052c009440 tx=0x7f052c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed09a72e75955c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0533fff700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 0x7f051c0a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0519ffb700 1 -- 172.21.15.3:0/3833590106 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c01a070 con 0x556174395b10 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.870+0000 7f0519ffb700 1 -- 172.21.15.3:0/3833590106 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c00f040 con 0x556174395b10 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.871+0000 7f0519ffb700 1 -- 172.21.15.3:0/3833590106 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00e070 con 0x556174395b10 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.871+0000 7f054d8a9b80 1 -- 172.21.15.3:0/3833590106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 msgr2=0x556174395f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.871+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f052c009440 tx=0x7f052c009920 comp rx=0 tx=0).stop 2023-08-11T14:05:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.871+0000 7f0533fff700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 0x7f051c0a2530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f052c019000 tx=0x7f052c014c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 -- 172.21.15.3:0/3833590106 shutdown_connections 2023-08-11T14:05:06.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x556174395f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/3833590106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x556174396950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 -- 172.21.15.3:0/3833590106 >> 172.21.15.3:0/3833590106 conn(0x5561742c2950 msgr2=0x5561743903e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 -- 172.21.15.3:0/3833590106 shutdown_connections 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.872+0000 7f054d8a9b80 1 -- 172.21.15.3:0/3833590106 wait complete. 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.873+0000 7f054d8a9b80 1 -- start start 2023-08-11T14:05:06.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.873+0000 7f054d8a9b80 1 Processor -- start 2023-08-11T14:05:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x55617439f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x55617439f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556174285ac0 con 0x556174396470 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561742857c0 con 0x556174395b10 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556174169920 con 0x556174396e90 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.874+0000 7f054d8a9b80 10 client.0 _ll_register_callbacks cb 0x556174288ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f05337fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x55617439f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f05337fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x55617439f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38526/0 (socket says 172.21.15.3:38526) 2023-08-11T14:05:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f05337fe700 1 -- 172.21.15.3:0/2401179699 learned_addr learned my addr 172.21.15.3:0/2401179699 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0532ffd700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x55617439f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 -- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 msgr2=0x55617439f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x55617439f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 -- 172.21.15.3:0/2401179699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 msgr2=0x55617439f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x55617439f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0533fff700 1 -- 172.21.15.3:0/2401179699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0090f0 con 0x556174396e90 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.875+0000 7f0532ffd700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x55617439f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.876+0000 7f0533fff700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743a6870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f052c009440 tx=0x7f052c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb219239c58abfe8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.876+0000 7f0512ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.876+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00e030 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.876+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556174388a90 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.876+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f052c004a70 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f052c0095e0 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 10 client.25939 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556174113de0 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556174388a90 con 0x556174396e90 2023-08-11T14:05:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f052c02b070 con 0x556174396e90 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 20 client.25939 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ab0a0 0x5561743ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 4 client.25939 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.877+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561743644e0 con 0x5561743ab0a0 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.878+0000 7f05337fe700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ab0a0 0x5561743ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.878+0000 7f0512ffd700 10 client.25939 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:06.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.880+0000 7f05337fe700 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ab0a0 0x5561743ad4c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b929f103fc93345 server_cookie=ed99778a368c337 in_seq=0 out_seq=0 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.880+0000 7f0512ffd700 1 -- 172.21.15.3:0/2401179699 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5561743644e0 con 0x5561743ab0a0 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.880+0000 7f0512ffd700 10 client.25939 handle_command_reply: tid=0 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.880+0000 7f054d8a9b80 1 client.25939 shutdown 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 msgr2=0x5561743a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743a6870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f052c009440 tx=0x7f052c01a000 comp rx=0 tx=0).stop 2023-08-11T14:05:06.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 shutdown_connections 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174396e90 0x5561743a6870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ab0a0 0x5561743ad4c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174395b10 0x55617439f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/2401179699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396470 0x55617439f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 >> 172.21.15.3:0/2401179699 conn(0x5561742c2950 msgr2=0x55617438acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.881+0000 7f0512ffd700 0 client.25939 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:06.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.882+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 shutdown_connections 2023-08-11T14:05:06.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.882+0000 7f054d8a9b80 1 -- 172.21.15.3:0/2401179699 wait complete. 2023-08-11T14:05:06.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.883+0000 7f054d8a9b80 20 client.25939 trim_cache size 0 max 16384 2023-08-11T14:05:06.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.900+0000 7f054d8a9b80 1 Processor -- start 2023-08-11T14:05:06.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.900+0000 7f054d8a9b80 1 -- start start 2023-08-11T14:05:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x556174397270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x55617439e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556174388a70 con 0x556174396e90 2023-08-11T14:05:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556174391130 con 0x556174396470 2023-08-11T14:05:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55617439ff70 con 0x556174395b10 2023-08-11T14:05:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38530/0 (socket says 172.21.15.3:38530) 2023-08-11T14:05:06.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.901+0000 7f0532ffd700 1 -- 172.21.15.3:0/1764867884 learned_addr learned my addr 172.21.15.3:0/1764867884 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x556174397270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 -- 172.21.15.3:0/1764867884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 msgr2=0x5561743a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 -- 172.21.15.3:0/1764867884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 msgr2=0x55617439e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x55617439e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 -- 172.21.15.3:0/1764867884 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561740fcc60 con 0x556174395b10 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.902+0000 7f05337fe700 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x556174397270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5561743add30 tx=0x7f0524009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8325b818376758 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.903+0000 7f051a7fc700 1 -- 172.21.15.3:0/1764867884 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524018040 con 0x556174395b10 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.903+0000 7f051a7fc700 1 -- 172.21.15.3:0/1764867884 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052401dd00 con 0x556174395b10 2023-08-11T14:05:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.903+0000 7f051a7fc700 1 -- 172.21.15.3:0/1764867884 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524005280 con 0x556174395b10 2023-08-11T14:05:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1764867884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 msgr2=0x556174397270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x556174397270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5561743add30 tx=0x7f0524009d40 comp rx=0 tx=0).stop 2023-08-11T14:05:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1764867884 shutdown_connections 2023-08-11T14:05:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x55617439e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x556174397270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1764867884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1764867884 >> 172.21.15.3:0/1764867884 conn(0x5561742c2950 msgr2=0x556174390390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1764867884 shutdown_connections 2023-08-11T14:05:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.904+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1764867884 wait complete. 2023-08-11T14:05:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.905+0000 7f054d8a9b80 1 -- start start 2023-08-11T14:05:06.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.905+0000 7f054d8a9b80 1 Processor -- start 2023-08-11T14:05:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x5561743a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561743a5ea0 con 0x556174396e90 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561743a6010 con 0x556174396470 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561743a6180 con 0x556174395b10 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.906+0000 7f054d8a9b80 10 client.0 _ll_register_callbacks cb 0x7f052c009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.907+0000 7f05337fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.907+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.907+0000 7f0532ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:38538/0 (socket says 172.21.15.3:38538) 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.907+0000 7f0532ffd700 1 -- 172.21.15.3:0/1262408128 learned_addr learned my addr 172.21.15.3:0/1262408128 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.907+0000 7f05337fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40688/0 (socket says 172.21.15.3:40688) 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 -- 172.21.15.3:0/1262408128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 msgr2=0x5561743a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 -- 172.21.15.3:0/1262408128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 msgr2=0x5561743a5960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x5561743a5960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 -- 172.21.15.3:0/1262408128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0524009b20 con 0x556174396470 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f05337fe700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0532ffd700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0528099cc0 tx=0x7f0528099920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ebcc66ddc4332d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.908+0000 7f0512ffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.909+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052809a420 con 0x556174396470 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.909+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f052c00f040 con 0x556174396470 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.909+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f052809ddf0 con 0x556174396470 2023-08-11T14:05:06.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.910+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05280a1d40 con 0x556174396470 2023-08-11T14:05:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 10 client.25952 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55617413a250 con 0x556174396470 2023-08-11T14:05:06.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c00f040 con 0x556174396470 2023-08-11T14:05:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0528094cd0 con 0x556174396470 2023-08-11T14:05:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 10 client.25952 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:06.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 20 client.25952 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.911+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ff820 0x556174401c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.912+0000 7f054d8a9b80 4 client.25952 mds_command: new command op to 24569 tid=0[{"prefix": "damage ls"}] 2023-08-11T14:05:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.912+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5561742e0820 con 0x5561743ff820 2023-08-11T14:05:06.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.912+0000 7f05337fe700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ff820 0x556174401c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.921+0000 7f0512ffd700 10 client.25952 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.921+0000 7f05337fe700 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ff820 0x556174401c40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378e67d36e4bf9ec server_cookie=ffd8c778e2c06c8c in_seq=0 out_seq=0 2023-08-11T14:05:06.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.922+0000 7f0512ffd700 1 -- 172.21.15.3:0/1262408128 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5561742e0820 con 0x5561743ff820 2023-08-11T14:05:06.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.922+0000 7f0512ffd700 10 client.25952 handle_command_reply: tid=0 2023-08-11T14:05:06.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.922+0000 7f054d8a9b80 1 client.25952 shutdown 2023-08-11T14:05:06.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.922+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 msgr2=0x5561743a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.922+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0528099cc0 tx=0x7f0528099920 comp rx=0 tx=0).stop 2023-08-11T14:05:06.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 shutdown_connections 2023-08-11T14:05:06.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556174396e90 0x5561743a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556174395b10 0x5561743a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5561743ff820 0x556174401c40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 --2- 172.21.15.3:0/1262408128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556174396470 0x5561743a2940 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0528099cc0 tx=0x7f0528099920 comp rx=0 tx=0).stop 2023-08-11T14:05:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 >> 172.21.15.3:0/1262408128 conn(0x5561742c2950 msgr2=0x556174390280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f0512ffd700 0 client.25952 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 shutdown_connections 2023-08-11T14:05:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.923+0000 7f054d8a9b80 1 -- 172.21.15.3:0/1262408128 wait complete. 2023-08-11T14:05:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.924+0000 7f054d8a9b80 20 client.25952 trim_cache size 0 max 16384 2023-08-11T14:05:06.929 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2023-08-11T14:05:06.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 msgr2=0x7f051c0a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 0x7f051c0a2530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f052c019000 tx=0x7f052c014c50 comp rx=0 tx=0).stop 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 msgr2=0x7f0534198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0528005b40 tx=0x7f052800f670 comp rx=0 tx=0).stop 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 shutdown_connections 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0534101210 0x7f053419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05340ff680 0x7f05341985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f051c0a0070 0x7f051c0a2530 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 --2- 172.21.15.3:0/1999814476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0534100870 0x7f0534198af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:06.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 >> 172.21.15.3:0/1999814476 conn(0x7f05340fabd0 msgr2=0x7f05340fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:06.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 shutdown_connections 2023-08-11T14:05:06.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:05:06.925+0000 7f0538d9d700 1 -- 172.21.15.3:0/1999814476 wait complete. 2023-08-11T14:05:06.970 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 37221692, 'ino': 1099511629291, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/pg_internal.init'}, {'damage_type': 'backtrace', 'id': 47224092, 'ino': 1099511629224, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2691'}, {'damage_type': 'backtrace', 'id': 47898387, 'ino': 1099511628327, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members/0000'}, {'damage_type': 'backtrace', 'id': 79860770, 'ino': 1099511628323, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_control'}, {'damage_type': 'backtrace', 'id': 93409659, 'ino': 1099511629214, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3712'}, {'damage_type': 'backtrace', 'id': 145181039, 'ino': 1099511629212, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2655'}, {'damage_type': 'backtrace', 'id': 171378835, 'ino': 1099511629233, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603'}, {'damage_type': 'backtrace', 'id': 184949457, 'ino': 1099511628455, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2695'}, {'damage_type': 'backtrace', 'id': 187896587, 'ino': 1099511627795, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets'}, {'damage_type': 'backtrace', 'id': 264205600, 'ino': 1099511629202, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3603_vm'}, {'damage_type': 'backtrace', 'id': 294516734, 'ino': 1099511628435, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2847'}, {'damage_type': 'backtrace', 'id': 335267248, 'ino': 1099511629211, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2702'}, {'damage_type': 'backtrace', 'id': 361129383, 'ino': 1099511629229, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2838'}, {'damage_type': 'backtrace', 'id': 379563255, 'ino': 1099511628439, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4061'}, {'damage_type': 'backtrace', 'id': 380152640, 'ino': 1099511628452, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2676'}, {'damage_type': 'backtrace', 'id': 385550828, 'ino': 1099511627779, 'path': '/volumes/_nogroup/sv_0/a00788a1-1990-44f3-ba7e-f12ca9c8c388'}, {'damage_type': 'backtrace', 'id': 412698461, 'ino': 1099511629236, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3394_vm'}, {'damage_type': 'backtrace', 'id': 577886125, 'ino': 1099511628609, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_vm'}, {'damage_type': 'backtrace', 'id': 663829480, 'ino': 1099511628552, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6114'}, {'damage_type': 'backtrace', 'id': 670207408, 'ino': 1099511628505, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1232'}, {'damage_type': 'backtrace', 'id': 692513702, 'ino': 1099511629295, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16393'}, {'damage_type': 'backtrace', 'id': 705426549, 'ino': 1099511628403, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592'}, {'damage_type': 'backtrace', 'id': 824725268, 'ino': 1099511628413, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100'}, {'damage_type': 'backtrace', 'id': 842337898, 'ino': 1099511628453, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2677'}, {'damage_type': 'backtrace', 'id': 856050291, 'ino': 1099511628676, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_fsm'}, {'damage_type': 'backtrace', 'id': 862902496, 'ino': 1099511628506, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1233'}, {'damage_type': 'backtrace', 'id': 863786518, 'ino': 1099511628376, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136'}, {'damage_type': 'backtrace', 'id': 869439095, 'ino': 1099511629228, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2663'}, {'damage_type': 'backtrace', 'id': 898258250, 'ino': 1099511629297, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/16387_fsm'}, {'damage_type': 'backtrace', 'id': 903627480, 'ino': 1099511629208, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2668'}, {'damage_type': 'backtrace', 'id': 943744433, 'ino': 1099511629209, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2603_fsm'}, {'damage_type': 'backtrace', 'id': 945148456, 'ino': 1099511628583, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214_vm'}, {'damage_type': 'backtrace', 'id': 1009190267, 'ino': 1099511628619, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846_vm'}, {'damage_type': 'backtrace', 'id': 1012219690, 'ino': 1099511628373, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964'}, {'damage_type': 'backtrace', 'id': 1040273801, 'ino': 1099511629234, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3764'}, {'damage_type': 'backtrace', 'id': 1075258953, 'ino': 1099511628537, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2965'}, {'damage_type': 'backtrace', 'id': 1112158257, 'ino': 1099511629204, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3467'}, {'damage_type': 'backtrace', 'id': 1124869463, 'ino': 1099511629232, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1247'}, {'damage_type': 'backtrace', 'id': 1189540869, 'ino': 1099511628546, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6002'}, {'damage_type': 'backtrace', 'id': 1305817988, 'ino': 1099511628324, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_xact/0000'}, {'damage_type': 'backtrace', 'id': 1319087724, 'ino': 1099511629218, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3164'}, {'damage_type': 'backtrace', 'id': 1407902751, 'ino': 1099511628377, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_fsm'}, {'damage_type': 'backtrace', 'id': 1434321753, 'ino': 1099511628566, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2964_vm'}, {'damage_type': 'backtrace', 'id': 1435225946, 'ino': 1099511628434, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2846'}, {'damage_type': 'backtrace', 'id': 1487719064, 'ino': 1099511628379, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_fsm'}, {'damage_type': 'backtrace', 'id': 1489925564, 'ino': 1099511628382, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396'}, {'damage_type': 'backtrace', 'id': 1588463786, 'ino': 1099511629219, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3501_vm'}, {'damage_type': 'backtrace', 'id': 1618266806, 'ino': 1099511629203, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3575'}, {'damage_type': 'backtrace', 'id': 1730787422, 'ino': 1099511629289, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_stat_tmp/global.stat'}, {'damage_type': 'backtrace', 'id': 1809850571, 'ino': 1099511629240, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12182_fsm'}, {'damage_type': 'backtrace', 'id': 1813454236, 'ino': 1099511628539, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3593'}, {'damage_type': 'backtrace', 'id': 1877206482, 'ino': 1099511629285, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_notify/0000'}, {'damage_type': 'backtrace', 'id': 1893918535, 'ino': 1099511629215, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2608'}, {'damage_type': 'backtrace', 'id': 1981833737, 'ino': 1099511629287, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/log/postgresql-Fri.log'}, {'damage_type': 'backtrace', 'id': 2000396982, 'ino': 1099511628675, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261_fsm'}, {'damage_type': 'backtrace', 'id': 2009805970, 'ino': 1099511629222, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/113'}, {'damage_type': 'backtrace', 'id': 2055544194, 'ino': 1099511629290, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_internal.init'}, {'damage_type': 'backtrace', 'id': 2064530221, 'ino': 1099511629221, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2682'}, {'damage_type': 'backtrace', 'id': 2145784339, 'ino': 1099511628378, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260'}, {'damage_type': 'backtrace', 'id': 2162057774, 'ino': 1099511628436, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966'}, {'damage_type': 'backtrace', 'id': 2195304920, 'ino': 1099511629223, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3604'}, {'damage_type': 'backtrace', 'id': 2235325653, 'ino': 1099511629237, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2754'}, {'damage_type': 'backtrace', 'id': 2309627812, 'ino': 1099511627794, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/members'}, {'damage_type': 'backtrace', 'id': 2340440622, 'ino': 1099511629220, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6111'}, {'damage_type': 'backtrace', 'id': 2370009688, 'ino': 1099511628381, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1214'}, {'damage_type': 'backtrace', 'id': 2370757710, 'ino': 1099511628560, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000_vm'}, {'damage_type': 'backtrace', 'id': 2384832646, 'ino': 1099511629241, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/1259'}, {'damage_type': 'backtrace', 'id': 2393600460, 'ino': 1099511628438, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060'}, {'damage_type': 'backtrace', 'id': 2436846549, 'ino': 1099511629244, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3606'}, {'damage_type': 'backtrace', 'id': 2466993924, 'ino': 1099511629216, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2831'}, {'damage_type': 'backtrace', 'id': 2479609972, 'ino': 1099511628475, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2397'}, {'damage_type': 'backtrace', 'id': 2552325905, 'ino': 1099511628617, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/4060_vm'}, {'damage_type': 'backtrace', 'id': 2619508485, 'ino': 1099511628470, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2671'}, {'damage_type': 'backtrace', 'id': 2672219054, 'ino': 1099511629230, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2836'}, {'damage_type': 'backtrace', 'id': 2713182625, 'ino': 1099511627780, 'path': '/volumes/_nogroup/sv_0/.meta'}, {'damage_type': 'backtrace', 'id': 2819983223, 'ino': 1099511629227, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2685'}, {'damage_type': 'backtrace', 'id': 2923613342, 'ino': 1099511628545, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6001'}, {'damage_type': 'backtrace', 'id': 2926683809, 'ino': 1099511629245, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2613'}, {'damage_type': 'backtrace', 'id': 2929217585, 'ino': 1099511628325, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_subtrans/0000'}, {'damage_type': 'backtrace', 'id': 2982137391, 'ino': 1099511629217, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12187_vm'}, {'damage_type': 'backtrace', 'id': 3000523535, 'ino': 1099511628374, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213'}, {'damage_type': 'backtrace', 'id': 3065061800, 'ino': 1099511628603, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_vm'}, {'damage_type': 'backtrace', 'id': 3065430256, 'ino': 1099511628581, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_vm'}, {'damage_type': 'backtrace', 'id': 3087424254, 'ino': 1099511628437, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2967'}, {'damage_type': 'backtrace', 'id': 3088885316, 'ino': 1099511629238, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12199'}, {'damage_type': 'backtrace', 'id': 3142137616, 'ino': 1099511629225, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2605_vm'}, {'damage_type': 'backtrace', 'id': 3196603362, 'ino': 1099511628509, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2698'}, {'damage_type': 'backtrace', 'id': 3200383143, 'ino': 1099511628471, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2672'}, {'damage_type': 'backtrace', 'id': 3222237151, 'ino': 1099511628629, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/pg_filenode.map'}, {'damage_type': 'backtrace', 'id': 3298054286, 'ino': 1099511629207, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2674'}, {'damage_type': 'backtrace', 'id': 3337320886, 'ino': 1099511628371, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262'}, {'damage_type': 'backtrace', 'id': 3401711241, 'ino': 1099511627804, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/snapshots'}, {'damage_type': 'backtrace', 'id': 3404121910, 'ino': 1099511628326, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_multixact/offsets/0000'}, {'damage_type': 'backtrace', 'id': 3425400516, 'ino': 1099511628553, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6115'}, {'damage_type': 'backtrace', 'id': 3456439680, 'ino': 1099511629242, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/12207'}, {'damage_type': 'backtrace', 'id': 3463964084, 'ino': 1099511629231, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/548'}, {'damage_type': 'backtrace', 'id': 3479506675, 'ino': 1099511628399, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6000'}, {'damage_type': 'backtrace', 'id': 3480102253, 'ino': 1099511629243, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2610'}, {'damage_type': 'backtrace', 'id': 3486648913, 'ino': 1099511628610, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1260_vm'}, {'damage_type': 'backtrace', 'id': 3511062747, 'ino': 1099511628587, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1136_vm'}, {'damage_type': 'backtrace', 'id': 3526545898, 'ino': 1099511628508, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2697'}, {'damage_type': 'backtrace', 'id': 3564465358, 'ino': 1099511628556, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/6100_vm'}, {'damage_type': 'backtrace', 'id': 3608305195, 'ino': 1099511629210, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/6110'}, {'damage_type': 'backtrace', 'id': 3672278187, 'ino': 1099511628618, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2966_vm'}, {'damage_type': 'backtrace', 'id': 3716965156, 'ino': 1099511628600, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_vm'}, {'damage_type': 'backtrace', 'id': 3737238258, 'ino': 1099511629226, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3440'}, {'damage_type': 'backtrace', 'id': 3948353003, 'ino': 1099511628499, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1137'}, {'damage_type': 'backtrace', 'id': 3953142008, 'ino': 1099511628380, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1261'}, {'damage_type': 'backtrace', 'id': 4058882836, 'ino': 1099511629239, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3455'}, {'damage_type': 'backtrace', 'id': 4059596266, 'ino': 1099511627805, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/pg_logical/mappings'}, {'damage_type': 'backtrace', 'id': 4111413533, 'ino': 1099511629206, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3119'}, {'damage_type': 'backtrace', 'id': 4137408644, 'ino': 1099511628564, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/3592_vm'}, {'damage_type': 'backtrace', 'id': 4144499169, 'ino': 1099511629205, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/2617_fsm'}, {'damage_type': 'backtrace', 'id': 4166520507, 'ino': 1099511628372, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1262_fsm'}, {'damage_type': 'backtrace', 'id': 4173354683, 'ino': 1099511628677, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2396_fsm'}, {'damage_type': 'backtrace', 'id': 4195827933, 'ino': 1099511628375, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/1213_fsm'}, {'damage_type': 'backtrace', 'id': 4205932086, 'ino': 1099511629213, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3597'}, {'damage_type': 'backtrace', 'id': 4248481033, 'ino': 1099511628454, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/global/2694'}, {'damage_type': 'backtrace', 'id': 4248500185, 'ino': 1099511629235, 'path': '/volumes/_nogroup/sv_1/62cd9d2c-a60b-4f8a-ba60-005c555889b1/postgres/data/base/12347/3080'}] 2023-08-11T14:05:06.971 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-08-11T14:05:07.289 INFO:teuthology.orchestra.run.smithi003.stderr:21800000 of 50000000 tuples (43%) done (elapsed 52.79 s, remaining 68.29 s) 2023-08-11T14:05:07.701 INFO:teuthology.orchestra.run.smithi003.stderr:21900000 of 50000000 tuples (43%) done (elapsed 53.20 s, remaining 68.26 s) 2023-08-11T14:05:07.884 INFO:teuthology.orchestra.run.smithi003.stderr:22000000 of 50000000 tuples (44%) done (elapsed 53.38 s, remaining 67.94 s) 2023-08-11T14:05:07.998 INFO:teuthology.orchestra.run.smithi003.stderr:22100000 of 50000000 tuples (44%) done (elapsed 53.50 s, remaining 67.54 s) 2023-08-11T14:05:08.218 INFO:teuthology.orchestra.run.smithi003.stderr:22200000 of 50000000 tuples (44%) done (elapsed 53.72 s, remaining 67.27 s) 2023-08-11T14:05:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:22300000 of 50000000 tuples (44%) done (elapsed 53.92 s, remaining 66.98 s) 2023-08-11T14:05:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:22400000 of 50000000 tuples (44%) done (elapsed 54.06 s, remaining 66.61 s) 2023-08-11T14:05:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:22500000 of 50000000 tuples (45%) done (elapsed 54.26 s, remaining 66.32 s) 2023-08-11T14:05:08.913 INFO:teuthology.orchestra.run.smithi003.stderr:22600000 of 50000000 tuples (45%) done (elapsed 54.41 s, remaining 65.97 s) 2023-08-11T14:05:09.119 INFO:teuthology.orchestra.run.smithi003.stderr:22700000 of 50000000 tuples (45%) done (elapsed 54.62 s, remaining 65.69 s) 2023-08-11T14:05:09.267 INFO:teuthology.orchestra.run.smithi003.stderr:22800000 of 50000000 tuples (45%) done (elapsed 54.77 s, remaining 65.34 s) 2023-08-11T14:05:09.425 INFO:teuthology.orchestra.run.smithi003.stderr:22900000 of 50000000 tuples (45%) done (elapsed 54.92 s, remaining 65.00 s) 2023-08-11T14:05:09.630 INFO:teuthology.orchestra.run.smithi003.stderr:23000000 of 50000000 tuples (46%) done (elapsed 55.13 s, remaining 64.72 s) 2023-08-11T14:05:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:09 smithi103 ceph-mon[130434]: pgmap v384: 129 pgs: 129 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 114 MiB/s wr, 68 op/s 2023-08-11T14:05:09.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:09 smithi003 ceph-mon[97195]: pgmap v384: 129 pgs: 129 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 114 MiB/s wr, 68 op/s 2023-08-11T14:05:09.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:09 smithi187 ceph-mon[130973]: pgmap v384: 129 pgs: 129 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 114 MiB/s wr, 68 op/s 2023-08-11T14:05:09.840 INFO:teuthology.orchestra.run.smithi003.stderr:23100000 of 50000000 tuples (46%) done (elapsed 55.34 s, remaining 64.44 s) 2023-08-11T14:05:10.023 INFO:teuthology.orchestra.run.smithi003.stderr:23200000 of 50000000 tuples (46%) done (elapsed 55.52 s, remaining 64.14 s) 2023-08-11T14:05:10.182 INFO:teuthology.orchestra.run.smithi003.stderr:23300000 of 50000000 tuples (46%) done (elapsed 55.68 s, remaining 63.81 s) 2023-08-11T14:05:10.359 INFO:teuthology.orchestra.run.smithi003.stderr:23400000 of 50000000 tuples (46%) done (elapsed 55.86 s, remaining 63.50 s) 2023-08-11T14:05:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:23500000 of 50000000 tuples (47%) done (elapsed 56.06 s, remaining 63.21 s) 2023-08-11T14:05:10.753 INFO:teuthology.orchestra.run.smithi003.stderr:23600000 of 50000000 tuples (47%) done (elapsed 56.25 s, remaining 62.93 s) 2023-08-11T14:05:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:23700000 of 50000000 tuples (47%) done (elapsed 56.43 s, remaining 62.62 s) 2023-08-11T14:05:11.063 INFO:teuthology.orchestra.run.smithi003.stderr:23800000 of 50000000 tuples (47%) done (elapsed 56.56 s, remaining 62.27 s) 2023-08-11T14:05:11.263 INFO:teuthology.orchestra.run.smithi003.stderr:23900000 of 50000000 tuples (47%) done (elapsed 56.76 s, remaining 61.99 s) 2023-08-11T14:05:11.439 INFO:teuthology.orchestra.run.smithi003.stderr:24000000 of 50000000 tuples (48%) done (elapsed 56.94 s, remaining 61.68 s) 2023-08-11T14:05:11.587 INFO:teuthology.orchestra.run.smithi003.stderr:24100000 of 50000000 tuples (48%) done (elapsed 57.09 s, remaining 61.35 s) 2023-08-11T14:05:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:11 smithi103 ceph-mon[130434]: pgmap v385: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 114 MiB/s wr, 62 op/s 2023-08-11T14:05:11.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:11 smithi003 ceph-mon[97195]: pgmap v385: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 114 MiB/s wr, 62 op/s 2023-08-11T14:05:11.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:11 smithi187 ceph-mon[130973]: pgmap v385: 129 pgs: 129 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 114 MiB/s wr, 62 op/s 2023-08-11T14:05:11.817 INFO:teuthology.orchestra.run.smithi003.stderr:24200000 of 50000000 tuples (48%) done (elapsed 57.32 s, remaining 61.11 s) 2023-08-11T14:05:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:24300000 of 50000000 tuples (48%) done (elapsed 57.47 s, remaining 60.79 s) 2023-08-11T14:05:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:24400000 of 50000000 tuples (48%) done (elapsed 57.65 s, remaining 60.48 s) 2023-08-11T14:05:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:24500000 of 50000000 tuples (49%) done (elapsed 57.85 s, remaining 60.22 s) 2023-08-11T14:05:12.544 INFO:teuthology.orchestra.run.smithi003.stderr:24600000 of 50000000 tuples (49%) done (elapsed 58.04 s, remaining 59.93 s) 2023-08-11T14:05:12.694 INFO:teuthology.orchestra.run.smithi003.stderr:24700000 of 50000000 tuples (49%) done (elapsed 58.19 s, remaining 59.61 s) 2023-08-11T14:05:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:24800000 of 50000000 tuples (49%) done (elapsed 58.38 s, remaining 59.32 s) 2023-08-11T14:05:13.052 INFO:teuthology.orchestra.run.smithi003.stderr:24900000 of 50000000 tuples (49%) done (elapsed 58.55 s, remaining 59.02 s) 2023-08-11T14:05:13.217 INFO:teuthology.orchestra.run.smithi003.stderr:25000000 of 50000000 tuples (50%) done (elapsed 58.72 s, remaining 58.72 s) 2023-08-11T14:05:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:25100000 of 50000000 tuples (50%) done (elapsed 58.86 s, remaining 58.39 s) 2023-08-11T14:05:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:25200000 of 50000000 tuples (50%) done (elapsed 59.06 s, remaining 58.13 s) 2023-08-11T14:05:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:13 smithi103 ceph-mon[130434]: pgmap v386: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 MiB/s wr, 69 op/s 2023-08-11T14:05:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:25300000 of 50000000 tuples (50%) done (elapsed 59.25 s, remaining 57.84 s) 2023-08-11T14:05:13.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:13 smithi003 ceph-mon[97195]: pgmap v386: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 MiB/s wr, 69 op/s 2023-08-11T14:05:13.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:13 smithi187 ceph-mon[130973]: pgmap v386: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 MiB/s wr, 69 op/s 2023-08-11T14:05:13.921 INFO:teuthology.orchestra.run.smithi003.stderr:25400000 of 50000000 tuples (50%) done (elapsed 59.42 s, remaining 57.55 s) 2023-08-11T14:05:14.415 INFO:teuthology.orchestra.run.smithi003.stderr:25500000 of 50000000 tuples (51%) done (elapsed 59.92 s, remaining 57.57 s) 2023-08-11T14:05:14.588 INFO:teuthology.orchestra.run.smithi003.stderr:25600000 of 50000000 tuples (51%) done (elapsed 60.09 s, remaining 57.27 s) 2023-08-11T14:05:15.228 INFO:teuthology.orchestra.run.smithi003.stderr:25700000 of 50000000 tuples (51%) done (elapsed 60.73 s, remaining 57.42 s) 2023-08-11T14:05:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:15 smithi103 ceph-mon[130434]: pgmap v387: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 97 MiB/s wr, 70 op/s 2023-08-11T14:05:15.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:15 smithi003 ceph-mon[97195]: pgmap v387: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 97 MiB/s wr, 70 op/s 2023-08-11T14:05:15.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:15 smithi187 ceph-mon[130973]: pgmap v387: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 97 MiB/s wr, 70 op/s 2023-08-11T14:05:15.909 INFO:teuthology.orchestra.run.smithi003.stderr:25800000 of 50000000 tuples (51%) done (elapsed 61.41 s, remaining 57.60 s) 2023-08-11T14:05:16.098 INFO:teuthology.orchestra.run.smithi003.stderr:25900000 of 50000000 tuples (51%) done (elapsed 61.60 s, remaining 57.32 s) 2023-08-11T14:05:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:26000000 of 50000000 tuples (52%) done (elapsed 61.72 s, remaining 56.97 s) 2023-08-11T14:05:16.429 INFO:teuthology.orchestra.run.smithi003.stderr:26100000 of 50000000 tuples (52%) done (elapsed 61.93 s, remaining 56.71 s) 2023-08-11T14:05:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:26200000 of 50000000 tuples (52%) done (elapsed 62.10 s, remaining 56.41 s) 2023-08-11T14:05:16.790 INFO:teuthology.orchestra.run.smithi003.stderr:26300000 of 50000000 tuples (52%) done (elapsed 62.29 s, remaining 56.13 s) 2023-08-11T14:05:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:26400000 of 50000000 tuples (52%) done (elapsed 62.45 s, remaining 55.83 s) 2023-08-11T14:05:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:26500000 of 50000000 tuples (53%) done (elapsed 62.59 s, remaining 55.51 s) 2023-08-11T14:05:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:26600000 of 50000000 tuples (53%) done (elapsed 62.81 s, remaining 55.25 s) 2023-08-11T14:05:17.534 INFO:teuthology.orchestra.run.smithi003.stderr:26700000 of 50000000 tuples (53%) done (elapsed 63.03 s, remaining 55.01 s) 2023-08-11T14:05:17.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:17 smithi103 ceph-mon[130434]: pgmap v388: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 98 MiB/s wr, 60 op/s 2023-08-11T14:05:17.682 INFO:teuthology.orchestra.run.smithi003.stderr:26800000 of 50000000 tuples (53%) done (elapsed 63.18 s, remaining 54.69 s) 2023-08-11T14:05:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:17 smithi003 ceph-mon[97195]: pgmap v388: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 98 MiB/s wr, 60 op/s 2023-08-11T14:05:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:17 smithi187 ceph-mon[130973]: pgmap v388: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 98 MiB/s wr, 60 op/s 2023-08-11T14:05:17.809 INFO:teuthology.orchestra.run.smithi003.stderr:26900000 of 50000000 tuples (53%) done (elapsed 63.31 s, remaining 54.37 s) 2023-08-11T14:05:17.984 INFO:teuthology.orchestra.run.smithi003.stderr:27000000 of 50000000 tuples (54%) done (elapsed 63.48 s, remaining 54.08 s) 2023-08-11T14:05:18.182 INFO:teuthology.orchestra.run.smithi003.stderr:27100000 of 50000000 tuples (54%) done (elapsed 63.68 s, remaining 53.81 s) 2023-08-11T14:05:18.329 INFO:teuthology.orchestra.run.smithi003.stderr:27200000 of 50000000 tuples (54%) done (elapsed 63.83 s, remaining 53.50 s) 2023-08-11T14:05:18.512 INFO:teuthology.orchestra.run.smithi003.stderr:27300000 of 50000000 tuples (54%) done (elapsed 64.01 s, remaining 53.23 s) 2023-08-11T14:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 -- 172.21.15.3:0/3583598479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4108810 msgr2=0x7f83b410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 --2- 172.21.15.3:0/3583598479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4108810 0x7f83b410aca0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009f90 tx=0x7f83ac00b330 comp rx=0 tx=0).stop 2023-08-11T14:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 -- 172.21.15.3:0/3583598479 shutdown_connections 2023-08-11T14:05:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 --2- 172.21.15.3:0/3583598479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4108810 0x7f83b410aca0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 --2- 172.21.15.3:0/3583598479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 --2- 172.21.15.3:0/3583598479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4073380 0x7f83b4073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 -- 172.21.15.3:0/3583598479 >> 172.21.15.3:0/3583598479 conn(0x7f83b40fbea0 msgr2=0x7f83b40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 -- 172.21.15.3:0/3583598479 shutdown_connections 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.700+0000 7f83b89ff700 1 -- 172.21.15.3:0/3583598479 wait complete. 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.701+0000 7f83b89ff700 1 Processor -- start 2023-08-11T14:05:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 -- start start 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 0x7f83b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 0x7f83b4199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b419a460 con 0x7f83b4074960 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b419a5d0 con 0x7f83b4108810 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b89ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b419a790 con 0x7f83b4073380 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38696/0 (socket says 172.21.15.3:38696) 2023-08-11T14:05:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b1d9b700 1 -- 172.21.15.3:0/113773270 learned_addr learned my addr 172.21.15.3:0/113773270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.702+0000 7f83b259c700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 0x7f83b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 0x7f83b4199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 -- 172.21.15.3:0/113773270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 msgr2=0x7f83b4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 0x7f83b4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 -- 172.21.15.3:0/113773270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 msgr2=0x7f83b4199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 0x7f83b4199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 -- 172.21.15.3:0/113773270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a8009520 con 0x7f83b4074960 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b259c700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 0x7f83b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 0x7f83b4199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ea50 tx=0x7f83a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83a37fe700 1 -- 172.21.15.3:0/113773270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800eee0 con 0x7f83b4074960 2023-08-11T14:05:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac009c40 con 0x7f83b4074960 2023-08-11T14:05:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.703+0000 7f83a37fe700 1 -- 172.21.15.3:0/113773270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a800cbd0 con 0x7f83b4074960 2023-08-11T14:05:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.704+0000 7f83a37fe700 1 -- 172.21.15.3:0/113773270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8018680 con 0x7f83b4074960 2023-08-11T14:05:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.704+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b419ac60 con 0x7f83b4074960 2023-08-11T14:05:18.710 INFO:teuthology.orchestra.run.smithi003.stderr:27400000 of 50000000 tuples (54%) done (elapsed 64.21 s, remaining 52.96 s) 2023-08-11T14:05:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.707+0000 7f83a37fe700 1 -- 172.21.15.3:0/113773270 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f83a8010430 con 0x7f83b4074960 2023-08-11T14:05:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.708+0000 7f83a37fe700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 0x7f839c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.708+0000 7f83cc33cb80 1 Processor -- start 2023-08-11T14:05:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.708+0000 7f83a37fe700 1 -- 172.21.15.3:0/113773270 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83a8014070 con 0x7f83b4074960 2023-08-11T14:05:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 -- start start 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b259c700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 0x7f839c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046ef40 0x55a87046f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a870470c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703c85d0 con 0x55a87046f8a0 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703c5ef0 con 0x55a870470790 2023-08-11T14:05:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8703c17e0 con 0x55a87046ef40 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b259c700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 0x7f839c0a2410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005fd0 tx=0x7f83a4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38712/0 (socket says 172.21.15.3:38712) 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3389929025 learned_addr learned my addr 172.21.15.3:0/3389929025 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.709+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a870470c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b259c700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046ef40 0x55a87046f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3389929025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046ef40 msgr2=0x55a87046f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046ef40 0x55a87046f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3389929025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 msgr2=0x55a870470c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a870470c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3389929025 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8702415d0 con 0x55a87046f8a0 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a870470c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83a8011fd0 tx=0x7f83a80999a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b5a659b0a5f12a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83a0ff9700 1 -- 172.21.15.3:0/3389929025 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80a0fe0 con 0x55a87046f8a0 2023-08-11T14:05:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83a0ff9700 1 -- 172.21.15.3:0/3389929025 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a800cd40 con 0x55a87046f8a0 2023-08-11T14:05:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.710+0000 7f83a0ff9700 1 -- 172.21.15.3:0/3389929025 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8098900 con 0x55a87046f8a0 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3389929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 msgr2=0x55a8704701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f83a8011fd0 tx=0x7f83a80999a0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3389929025 shutdown_connections 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a870470c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046ef40 0x55a87046f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3389929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a8704701b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3389929025 >> 172.21.15.3:0/3389929025 conn(0x55a87039c8f0 msgr2=0x55a87046b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3389929025 shutdown_connections 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.711+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3389929025 wait complete. 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.712+0000 7f83cc33cb80 1 -- start start 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.712+0000 7f83cc33cb80 1 Processor -- start 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a870478cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870480240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87035fa60 con 0x55a87046ef40 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87035f760 con 0x55a87046f8a0 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8702438c0 con 0x55a870470790 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83cc33cb80 10 client.0 _ll_register_callbacks cb 0x55a870362a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a870478cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:59250/0 (socket says 172.21.15.3:59250) 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.713+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3955152292 learned_addr learned my addr 172.21.15.3:0/3955152292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870480240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3955152292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 msgr2=0x55a870480240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870480240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3955152292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 msgr2=0x55a870478cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a870478cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 -- 172.21.15.3:0/3955152292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b410b8c0 con 0x55a87046f8a0 2023-08-11T14:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f83a8099830 tx=0x7f83a80b1440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b6924c55a376803 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f8399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80a0fe0 con 0x55a87046f8a0 2023-08-11T14:05:18.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8704627e0 con 0x55a87046f8a0 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83a80a0b50 con 0x55a87046f8a0 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a80a0d10 con 0x55a87046f8a0 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83cc33cb80 10 client.25964 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.714+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f83b419ac60 con 0x55a87046f8a0 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a807a030 con 0x55a87046f8a0 2023-08-11T14:05:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f83a8098310 con 0x55a87046f8a0 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 10 client.25964 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 10 client.25964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 20 client.25964 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a870484810 0x55a870486c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 4 client.25964 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a87043e480 con 0x55a870484810 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.715+0000 7f83b259c700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a870484810 0x55a870486c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.716+0000 7f83b259c700 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a870484810 0x55a870486c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f2fa92967831c6 server_cookie=11894f4455414af1 in_seq=0 out_seq=0 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.716+0000 7f8399ffb700 10 client.25964 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.716+0000 7f8399ffb700 1 -- 172.21.15.3:0/3955152292 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a87043e480 con 0x55a870484810 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.716+0000 7f8399ffb700 10 client.25964 handle_command_reply: tid=0 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.716+0000 7f83cc33cb80 1 client.25964 shutdown 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 msgr2=0x55a870479200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f83a8099830 tx=0x7f83a80b1440 comp rx=0 tx=0).stop 2023-08-11T14:05:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 shutdown_connections 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870480240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a870478cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a870484810 0x55a870486c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/3955152292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046f8a0 0x55a870479200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 >> 172.21.15.3:0/3955152292 conn(0x55a87039c8f0 msgr2=0x55a87046a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f8399ffb700 0 client.25964 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 shutdown_connections 2023-08-11T14:05:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 1 -- 172.21.15.3:0/3955152292 wait complete. 2023-08-11T14:05:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.717+0000 7f83cc33cb80 20 client.25964 trim_cache size 0 max 16384 2023-08-11T14:05:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.732+0000 7f83cc33cb80 1 Processor -- start 2023-08-11T14:05:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 -- start start 2023-08-11T14:05:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046ef40 0x55a87046f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a870480200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a870470640 con 0x55a87046f8a0 2023-08-11T14:05:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a870479bf0 con 0x55a87046ef40 2023-08-11T14:05:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87046add0 con 0x55a870470790 2023-08-11T14:05:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a870480200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b2d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a870480200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38724/0 (socket says 172.21.15.3:38724) 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b1d9b700 1 -- 172.21.15.3:0/2776287542 learned_addr learned my addr 172.21.15.3:0/2776287542 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b2d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35244/0 (socket says 172.21.15.3:35244) 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.733+0000 7f83b259c700 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046ef40 0x55a87046f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 -- 172.21.15.3:0/2776287542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046ef40 msgr2=0x55a87046f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046ef40 0x55a87046f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 -- 172.21.15.3:0/2776287542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 msgr2=0x55a870480200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a870480200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 -- 172.21.15.3:0/2776287542 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8701edd80 con 0x55a870470790 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f83ac007900 tx=0x7f83ac007420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de89178c7ab68b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83a17fa700 1 -- 172.21.15.3:0/2776287542 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac021070 con 0x55a870470790 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83a17fa700 1 -- 172.21.15.3:0/2776287542 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ac007510 con 0x55a870470790 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.734+0000 7f83a17fa700 1 -- 172.21.15.3:0/2776287542 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00e040 con 0x55a870470790 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 -- 172.21.15.3:0/2776287542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 msgr2=0x55a870476a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f83ac007900 tx=0x7f83ac007420 comp rx=0 tx=0).stop 2023-08-11T14:05:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 -- 172.21.15.3:0/2776287542 shutdown_connections 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a870470790 0x55a870476a10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a87046ef40 0x55a87046f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/2776287542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046f8a0 0x55a870480200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 -- 172.21.15.3:0/2776287542 >> 172.21.15.3:0/2776287542 conn(0x55a87039c8f0 msgr2=0x55a870478c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 -- 172.21.15.3:0/2776287542 shutdown_connections 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.735+0000 7f83cc33cb80 1 -- 172.21.15.3:0/2776287542 wait complete. 2023-08-11T14:05:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.736+0000 7f83cc33cb80 1 -- start start 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.736+0000 7f83cc33cb80 1 Processor -- start 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a8704d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 0x55a8704d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a8704d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8704d7990 con 0x55a87046ef40 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8704d7b00 con 0x55a870470790 2023-08-11T14:05:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8704d7cc0 con 0x55a87046f8a0 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83cc33cb80 10 client.0 _ll_register_callbacks cb 0x7f83a40091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a8704d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 0x55a8704d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a8704d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38732/0 (socket says 172.21.15.3:38732) 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b259c700 1 -- 172.21.15.3:0/885087281 learned_addr learned my addr 172.21.15.3:0/885087281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b2d9d700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a8704d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b1d9b700 1 -- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 msgr2=0x55a8704d7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a8704d7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b1d9b700 1 -- 172.21.15.3:0/885087281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 msgr2=0x55a8704d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.737+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a8704d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f83b1d9b700 1 -- 172.21.15.3:0/885087281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac008e90 con 0x55a87046f8a0 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f83b1d9b700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 0x55a8704d4ba0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83a80956d0 tx=0x7f83a80092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea5ccef13cde7f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f8399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80a0fe0 con 0x55a87046f8a0 2023-08-11T14:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f83a8082590 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83a800cd40 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a8082c10 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f83cc33cb80 10 client.25963 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a8701d6c00 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.738+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a800cd40 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f83a8094fc0 con 0x55a87046f8a0 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 10 client.25963 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 20 client.25963 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a8704d9340 0x55a8704db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 4 client.25963 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a8703ba7c0 con 0x55a8704d9340 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83b259c700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a8704d9340 0x55a8704db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f83b259c700 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a8704d9340 0x55a8704db760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bccfcb64e94898d server_cookie=18f155da34b1fbe6 in_seq=0 out_seq=0 2023-08-11T14:05:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.739+0000 7f8399ffb700 10 client.25963 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f8399ffb700 1 -- 172.21.15.3:0/885087281 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a8703ba7c0 con 0x55a8704d9340 2023-08-11T14:05:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f8399ffb700 10 client.25963 handle_command_reply: tid=0 2023-08-11T14:05:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 client.25963 shutdown 2023-08-11T14:05:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 msgr2=0x55a8704d4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 0x55a8704d4ba0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83a80956d0 tx=0x7f83a80092d0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 shutdown_connections 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a8704d9340 0x55a8704db760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a870470790 0x55a8704d7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a87046ef40 0x55a8704d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 --2- 172.21.15.3:0/885087281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a87046f8a0 0x55a8704d4ba0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f8399ffb700 0 client.25963 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 >> 172.21.15.3:0/885087281 conn(0x55a87039c8f0 msgr2=0x55a870478bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 shutdown_connections 2023-08-11T14:05:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 1 -- 172.21.15.3:0/885087281 wait complete. 2023-08-11T14:05:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.772+0000 7f83cc33cb80 20 client.25963 trim_cache size 0 max 16384 2023-08-11T14:05:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:05:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:05:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:05:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 msgr2=0x7f839c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 0x7f839c0a2410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005fd0 tx=0x7f83a4009310 comp rx=0 tx=0).stop 2023-08-11T14:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 msgr2=0x7f83b4198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ea50 tx=0x7f83a800ed60 comp rx=0 tx=0).stop 2023-08-11T14:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 shutdown_connections 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f839c09ff50 0x7f839c0a2410 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b4108810 0x7f83b4199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83b4073380 0x7f83b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 --2- 172.21.15.3:0/113773270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4074960 0x7f83b4198f80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 >> 172.21.15.3:0/113773270 conn(0x7f83b40fbea0 msgr2=0x7f83b40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 shutdown_connections 2023-08-11T14:05:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:18.773+0000 7f83b89ff700 1 -- 172.21.15.3:0/113773270 wait complete. 2023-08-11T14:05:18.908 INFO:teuthology.orchestra.run.smithi003.stderr:27500000 of 50000000 tuples (55%) done (elapsed 64.41 s, remaining 52.70 s) 2023-08-11T14:05:19.080 INFO:teuthology.orchestra.run.smithi003.stderr:27600000 of 50000000 tuples (55%) done (elapsed 64.58 s, remaining 52.41 s) 2023-08-11T14:05:19.224 INFO:teuthology.orchestra.run.smithi003.stderr:27700000 of 50000000 tuples (55%) done (elapsed 64.72 s, remaining 52.11 s) 2023-08-11T14:05:19.443 INFO:teuthology.orchestra.run.smithi003.stderr:27800000 of 50000000 tuples (55%) done (elapsed 64.94 s, remaining 51.86 s) 2023-08-11T14:05:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:27900000 of 50000000 tuples (55%) done (elapsed 65.11 s, remaining 51.57 s) 2023-08-11T14:05:19.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:19 smithi103 ceph-mon[130434]: pgmap v389: 129 pgs: 129 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 76 op/s 2023-08-11T14:05:19.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:19 smithi003 ceph-mon[97195]: pgmap v389: 129 pgs: 129 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 76 op/s 2023-08-11T14:05:19.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:19 smithi187 ceph-mon[130973]: pgmap v389: 129 pgs: 129 active+clean; 4.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 76 op/s 2023-08-11T14:05:19.829 INFO:teuthology.orchestra.run.smithi003.stderr:28000000 of 50000000 tuples (56%) done (elapsed 65.33 s, remaining 51.33 s) 2023-08-11T14:05:19.991 INFO:teuthology.orchestra.run.smithi003.stderr:28100000 of 50000000 tuples (56%) done (elapsed 65.49 s, remaining 51.04 s) 2023-08-11T14:05:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:28200000 of 50000000 tuples (56%) done (elapsed 65.64 s, remaining 50.74 s) 2023-08-11T14:05:20.320 INFO:teuthology.orchestra.run.smithi003.stderr:28300000 of 50000000 tuples (56%) done (elapsed 65.82 s, remaining 50.47 s) 2023-08-11T14:05:20.509 INFO:teuthology.orchestra.run.smithi003.stderr:28400000 of 50000000 tuples (56%) done (elapsed 66.01 s, remaining 50.20 s) 2023-08-11T14:05:20.718 INFO:teuthology.orchestra.run.smithi003.stderr:28500000 of 50000000 tuples (57%) done (elapsed 66.22 s, remaining 49.95 s) 2023-08-11T14:05:20.863 INFO:teuthology.orchestra.run.smithi003.stderr:28600000 of 50000000 tuples (57%) done (elapsed 66.36 s, remaining 49.66 s) 2023-08-11T14:05:21.025 INFO:teuthology.orchestra.run.smithi003.stderr:28700000 of 50000000 tuples (57%) done (elapsed 66.52 s, remaining 49.37 s) 2023-08-11T14:05:21.174 INFO:teuthology.orchestra.run.smithi003.stderr:28800000 of 50000000 tuples (57%) done (elapsed 66.67 s, remaining 49.08 s) 2023-08-11T14:05:21.377 INFO:teuthology.orchestra.run.smithi003.stderr:28900000 of 50000000 tuples (57%) done (elapsed 66.88 s, remaining 48.83 s) 2023-08-11T14:05:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:29000000 of 50000000 tuples (58%) done (elapsed 67.05 s, remaining 48.55 s) 2023-08-11T14:05:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:21 smithi103 ceph-mon[130434]: pgmap v390: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 112 MiB/s wr, 71 op/s 2023-08-11T14:05:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:29100000 of 50000000 tuples (58%) done (elapsed 67.25 s, remaining 48.30 s) 2023-08-11T14:05:21.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:21 smithi003 ceph-mon[97195]: pgmap v390: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 112 MiB/s wr, 71 op/s 2023-08-11T14:05:21.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:21 smithi187 ceph-mon[130973]: pgmap v390: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 112 MiB/s wr, 71 op/s 2023-08-11T14:05:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:29200000 of 50000000 tuples (58%) done (elapsed 67.43 s, remaining 48.03 s) 2023-08-11T14:05:22.072 INFO:teuthology.orchestra.run.smithi003.stderr:29300000 of 50000000 tuples (58%) done (elapsed 67.57 s, remaining 47.74 s) 2023-08-11T14:05:22.437 INFO:teuthology.orchestra.run.smithi003.stderr:29400000 of 50000000 tuples (58%) done (elapsed 67.94 s, remaining 47.60 s) 2023-08-11T14:05:22.740 INFO:teuthology.orchestra.run.smithi003.stderr:29500000 of 50000000 tuples (59%) done (elapsed 68.24 s, remaining 47.42 s) 2023-08-11T14:05:23.626 INFO:teuthology.orchestra.run.smithi003.stderr:29600000 of 50000000 tuples (59%) done (elapsed 69.13 s, remaining 47.64 s) 2023-08-11T14:05:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:23 smithi103 ceph-mon[130434]: pgmap v391: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 82 op/s 2023-08-11T14:05:23.787 INFO:teuthology.orchestra.run.smithi003.stderr:29700000 of 50000000 tuples (59%) done (elapsed 69.28 s, remaining 47.36 s) 2023-08-11T14:05:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:23 smithi003 ceph-mon[97195]: pgmap v391: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 82 op/s 2023-08-11T14:05:23.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:23 smithi187 ceph-mon[130973]: pgmap v391: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 121 MiB/s wr, 82 op/s 2023-08-11T14:05:24.026 INFO:teuthology.orchestra.run.smithi003.stderr:29800000 of 50000000 tuples (59%) done (elapsed 69.53 s, remaining 47.13 s) 2023-08-11T14:05:24.150 INFO:teuthology.orchestra.run.smithi003.stderr:29900000 of 50000000 tuples (59%) done (elapsed 69.65 s, remaining 46.82 s) 2023-08-11T14:05:24.341 INFO:teuthology.orchestra.run.smithi003.stderr:30000000 of 50000000 tuples (60%) done (elapsed 69.84 s, remaining 46.56 s) 2023-08-11T14:05:24.513 INFO:teuthology.orchestra.run.smithi003.stderr:30100000 of 50000000 tuples (60%) done (elapsed 70.01 s, remaining 46.29 s) 2023-08-11T14:05:24.643 INFO:teuthology.orchestra.run.smithi003.stderr:30200000 of 50000000 tuples (60%) done (elapsed 70.14 s, remaining 45.99 s) 2023-08-11T14:05:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:30300000 of 50000000 tuples (60%) done (elapsed 70.34 s, remaining 45.73 s) 2023-08-11T14:05:25.035 INFO:teuthology.orchestra.run.smithi003.stderr:30400000 of 50000000 tuples (60%) done (elapsed 70.53 s, remaining 45.48 s) 2023-08-11T14:05:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:30500000 of 50000000 tuples (61%) done (elapsed 70.73 s, remaining 45.22 s) 2023-08-11T14:05:25.424 INFO:teuthology.orchestra.run.smithi003.stderr:30600000 of 50000000 tuples (61%) done (elapsed 70.92 s, remaining 44.96 s) 2023-08-11T14:05:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:30700000 of 50000000 tuples (61%) done (elapsed 71.05 s, remaining 44.67 s) 2023-08-11T14:05:25.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:25 smithi103 ceph-mon[130434]: pgmap v392: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 106 MiB/s wr, 80 op/s 2023-08-11T14:05:25.749 INFO:teuthology.orchestra.run.smithi003.stderr:30800000 of 50000000 tuples (61%) done (elapsed 71.25 s, remaining 44.41 s) 2023-08-11T14:05:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:25 smithi003 ceph-mon[97195]: pgmap v392: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 106 MiB/s wr, 80 op/s 2023-08-11T14:05:25.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:25 smithi187 ceph-mon[130973]: pgmap v392: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 106 MiB/s wr, 80 op/s 2023-08-11T14:05:25.953 INFO:teuthology.orchestra.run.smithi003.stderr:30900000 of 50000000 tuples (61%) done (elapsed 71.45 s, remaining 44.17 s) 2023-08-11T14:05:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:31000000 of 50000000 tuples (62%) done (elapsed 71.69 s, remaining 43.94 s) 2023-08-11T14:05:26.323 INFO:teuthology.orchestra.run.smithi003.stderr:31100000 of 50000000 tuples (62%) done (elapsed 71.82 s, remaining 43.65 s) 2023-08-11T14:05:26.519 INFO:teuthology.orchestra.run.smithi003.stderr:31200000 of 50000000 tuples (62%) done (elapsed 72.02 s, remaining 43.40 s) 2023-08-11T14:05:26.719 INFO:teuthology.orchestra.run.smithi003.stderr:31300000 of 50000000 tuples (62%) done (elapsed 72.22 s, remaining 43.15 s) 2023-08-11T14:05:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:31400000 of 50000000 tuples (62%) done (elapsed 72.38 s, remaining 42.87 s) 2023-08-11T14:05:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:31500000 of 50000000 tuples (63%) done (elapsed 72.54 s, remaining 42.60 s) 2023-08-11T14:05:27.234 INFO:teuthology.orchestra.run.smithi003.stderr:31600000 of 50000000 tuples (63%) done (elapsed 72.73 s, remaining 42.35 s) 2023-08-11T14:05:27.456 INFO:teuthology.orchestra.run.smithi003.stderr:31700000 of 50000000 tuples (63%) done (elapsed 72.96 s, remaining 42.12 s) 2023-08-11T14:05:27.662 INFO:teuthology.orchestra.run.smithi003.stderr:31800000 of 50000000 tuples (63%) done (elapsed 73.16 s, remaining 41.87 s) 2023-08-11T14:05:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:27 smithi103 ceph-mon[130434]: pgmap v393: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 117 MiB/s wr, 72 op/s 2023-08-11T14:05:27.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:27 smithi003 ceph-mon[97195]: pgmap v393: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 117 MiB/s wr, 72 op/s 2023-08-11T14:05:27.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:27 smithi187 ceph-mon[130973]: pgmap v393: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 117 MiB/s wr, 72 op/s 2023-08-11T14:05:27.804 INFO:teuthology.orchestra.run.smithi003.stderr:31900000 of 50000000 tuples (63%) done (elapsed 73.30 s, remaining 41.59 s) 2023-08-11T14:05:27.985 INFO:teuthology.orchestra.run.smithi003.stderr:32000000 of 50000000 tuples (64%) done (elapsed 73.48 s, remaining 41.34 s) 2023-08-11T14:05:28.183 INFO:teuthology.orchestra.run.smithi003.stderr:32100000 of 50000000 tuples (64%) done (elapsed 73.68 s, remaining 41.09 s) 2023-08-11T14:05:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:32200000 of 50000000 tuples (64%) done (elapsed 73.85 s, remaining 40.82 s) 2023-08-11T14:05:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:32300000 of 50000000 tuples (64%) done (elapsed 74.03 s, remaining 40.57 s) 2023-08-11T14:05:28.729 INFO:teuthology.orchestra.run.smithi003.stderr:32400000 of 50000000 tuples (64%) done (elapsed 74.23 s, remaining 40.32 s) 2023-08-11T14:05:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:32500000 of 50000000 tuples (65%) done (elapsed 74.41 s, remaining 40.07 s) 2023-08-11T14:05:29.091 INFO:teuthology.orchestra.run.smithi003.stderr:32600000 of 50000000 tuples (65%) done (elapsed 74.59 s, remaining 39.81 s) 2023-08-11T14:05:29.215 INFO:teuthology.orchestra.run.smithi003.stderr:32700000 of 50000000 tuples (65%) done (elapsed 74.72 s, remaining 39.53 s) 2023-08-11T14:05:29.419 INFO:teuthology.orchestra.run.smithi003.stderr:32800000 of 50000000 tuples (65%) done (elapsed 74.92 s, remaining 39.29 s) 2023-08-11T14:05:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:32900000 of 50000000 tuples (65%) done (elapsed 75.13 s, remaining 39.05 s) 2023-08-11T14:05:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:29 smithi103 ceph-mon[130434]: pgmap v394: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 MiB/s wr, 84 op/s 2023-08-11T14:05:29.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:29 smithi003 ceph-mon[97195]: pgmap v394: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 MiB/s wr, 84 op/s 2023-08-11T14:05:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:29 smithi187 ceph-mon[130973]: pgmap v394: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 MiB/s wr, 84 op/s 2023-08-11T14:05:29.846 INFO:teuthology.orchestra.run.smithi003.stderr:33000000 of 50000000 tuples (66%) done (elapsed 75.35 s, remaining 38.81 s) 2023-08-11T14:05:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:33100000 of 50000000 tuples (66%) done (elapsed 75.51 s, remaining 38.55 s) 2023-08-11T14:05:30.185 INFO:teuthology.orchestra.run.smithi003.stderr:33200000 of 50000000 tuples (66%) done (elapsed 75.68 s, remaining 38.30 s) 2023-08-11T14:05:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:33300000 of 50000000 tuples (66%) done (elapsed 76.25 s, remaining 38.24 s) 2023-08-11T14:05:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:33400000 of 50000000 tuples (66%) done (elapsed 77.07 s, remaining 38.31 s) 2023-08-11T14:05:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:31 smithi103 ceph-mon[130434]: pgmap v395: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 107 MiB/s wr, 68 op/s 2023-08-11T14:05:31.717 INFO:teuthology.orchestra.run.smithi003.stderr:33500000 of 50000000 tuples (67%) done (elapsed 77.22 s, remaining 38.03 s) 2023-08-11T14:05:31.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:31 smithi003 ceph-mon[97195]: pgmap v395: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 107 MiB/s wr, 68 op/s 2023-08-11T14:05:31.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:31 smithi187 ceph-mon[130973]: pgmap v395: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 107 MiB/s wr, 68 op/s 2023-08-11T14:05:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/2378415795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0410cf00 msgr2=0x7f7c0410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/2378415795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0410cf00 0x7f7c0410f2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009f90 tx=0x7f7bfc00b330 comp rx=0 tx=0).stop 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/2378415795 shutdown_connections 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/2378415795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0410cf00 0x7f7c0410f2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/2378415795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c04074960 0x7f7c04072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/2378415795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 0x7f7c04073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/2378415795 >> 172.21.15.3:0/2378415795 conn(0x7f7c040fbe60 msgr2=0x7f7c040fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/2378415795 shutdown_connections 2023-08-11T14:05:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.031+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/2378415795 wait complete. 2023-08-11T14:05:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.032+0000 7f7c0b5ff700 1 Processor -- start 2023-08-11T14:05:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.032+0000 7f7c0b5ff700 1 -- start start 2023-08-11T14:05:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 0x7f7c0419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c0410cf00 0x7f7c0419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419e820 con 0x7f7c04073380 2023-08-11T14:05:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419e990 con 0x7f7c04074960 2023-08-11T14:05:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0b5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0419eb50 con 0x7f7c0410cf00 2023-08-11T14:05:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37308/0 (socket says 172.21.15.3:37308) 2023-08-11T14:05:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3122621030 learned_addr learned my addr 172.21.15.3:0/3122621030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 0x7f7c0419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c09b9c700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c0410cf00 0x7f7c0419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3122621030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c0410cf00 msgr2=0x7f7c0419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c0410cf00 0x7f7c0419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 msgr2=0x7f7c0419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 0x7f7c0419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.033+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3122621030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40095f0 con 0x7f7c04074960 2023-08-11T14:05:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400ebf0 tx=0x7f7bf400ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7bfa7fc700 1 -- 172.21.15.3:0/3122621030 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4009a10 con 0x7f7c04074960 2023-08-11T14:05:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7bfa7fc700 1 -- 172.21.15.3:0/3122621030 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf400cc90 con 0x7f7c04074960 2023-08-11T14:05:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc009c40 con 0x7f7c04074960 2023-08-11T14:05:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7bfa7fc700 1 -- 172.21.15.3:0/3122621030 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4018750 con 0x7f7c04074960 2023-08-11T14:05:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.034+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c041a5540 con 0x7f7c04074960 2023-08-11T14:05:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.037+0000 7f7bfa7fc700 1 -- 172.21.15.3:0/3122621030 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7bf40188b0 con 0x7f7c04074960 2023-08-11T14:05:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.039+0000 7f7bfa7fc700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 0x7f7bf00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.039+0000 7f7bfa7fc700 1 -- 172.21.15.3:0/3122621030 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7bf4014070 con 0x7f7c04074960 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.039+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 0x7f7bf00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.040+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 0x7f7bf00a2410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c0419cb80 tx=0x7f7c00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 Processor -- start 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 -- start start 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef051c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eefa95d0 con 0x5626ef051790 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eefa6ef0 con 0x5626ef04ff40 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.041+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eefa27e0 con 0x5626ef0508a0 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c0939b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c0939b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37320/0 (socket says 172.21.15.3:37320) 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c0939b700 1 -- 172.21.15.3:0/3825209975 learned_addr learned my addr 172.21.15.3:0/3825209975 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c09b9c700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef051c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3825209975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 msgr2=0x5626ef050360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3825209975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 msgr2=0x5626ef051c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef051c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3825209975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eee225d0 con 0x5626ef0508a0 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.042+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0511b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40991e0 tx=0x7f7bf400ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e40505827eaffa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.043+0000 7f7bef7fe700 1 -- 172.21.15.3:0/3825209975 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4099250 con 0x5626ef0508a0 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.043+0000 7f7bef7fe700 1 -- 172.21.15.3:0/3825209975 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf409d080 con 0x5626ef0508a0 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.043+0000 7f7bef7fe700 1 -- 172.21.15.3:0/3825209975 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40a0e30 con 0x5626ef0508a0 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.043+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3825209975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 msgr2=0x5626ef0511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.043+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0511b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40991e0 tx=0x7f7bf400ce80 comp rx=0 tx=0).stop 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3825209975 shutdown_connections 2023-08-11T14:05:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef051c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef050360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3825209975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0511b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3825209975 >> 172.21.15.3:0/3825209975 conn(0x5626eef7d8f0 msgr2=0x5626ef04c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3825209975 shutdown_connections 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3825209975 wait complete. 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 -- start start 2023-08-11T14:05:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.044+0000 7f7c1efc3b80 1 Processor -- start 2023-08-11T14:05:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 0x5626ef059c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 0x5626ef05a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eef40a60 con 0x5626ef0508a0 2023-08-11T14:05:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eef40760 con 0x5626ef051790 2023-08-11T14:05:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eee248c0 con 0x5626ef04ff40 2023-08-11T14:05:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c1efc3b80 10 client.0 _ll_register_callbacks cb 0x5626eef43a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 0x5626ef05a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c09b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c0939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 0x5626ef059c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c09b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37334/0 (socket says 172.21.15.3:37334) 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 0x5626ef05a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60766/0 (socket says 172.21.15.3:60766) 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.045+0000 7f7c09b9c700 1 -- 172.21.15.3:0/1955744133 learned_addr learned my addr 172.21.15.3:0/1955744133 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 -- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 msgr2=0x5626ef061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef061200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 -- 172.21.15.3:0/1955744133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 msgr2=0x5626ef05a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 0x5626ef05a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 -- 172.21.15.3:0/1955744133 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c00017070 con 0x5626ef04ff40 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c0939b700 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 0x5626ef059c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c00011fd0 tx=0x7f7c000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920d3a20fc9411e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7becff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00022070 con 0x5626ef04ff40 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ef0437e0 con 0x5626ef04ff40 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.046+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7c00004830 con 0x5626ef04ff40 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c000049f0 con 0x5626ef04ff40 2023-08-11T14:05:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 10 client.25975 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f7c041a5540 con 0x5626ef04ff40 2023-08-11T14:05:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626ef0437e0 con 0x5626ef04ff40 2023-08-11T14:05:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7c0001c6d0 con 0x5626ef04ff40 2023-08-11T14:05:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 20 client.25975 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef065bd0 0x5626ef067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 4 client.25975 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.047+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626ef01f480 con 0x5626ef065bd0 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.048+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef065bd0 0x5626ef067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.048+0000 7f7becff9700 10 client.25975 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.048+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef065bd0 0x5626ef067ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5702ccf015af6d39 server_cookie=9ca6b5b9973121ff in_seq=0 out_seq=0 2023-08-11T14:05:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7becff9700 1 -- 172.21.15.3:0/1955744133 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5626ef01f480 con 0x5626ef065bd0 2023-08-11T14:05:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7becff9700 10 client.25975 handle_command_reply: tid=0 2023-08-11T14:05:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 client.25975 shutdown 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 msgr2=0x5626ef059c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 0x5626ef059c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c00011fd0 tx=0x7f7c000047c0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 shutdown_connections 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef065bd0 0x5626ef067ff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef04ff40 0x5626ef059c80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/1955744133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef0508a0 0x5626ef05a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7becff9700 0 client.25975 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 >> 172.21.15.3:0/1955744133 conn(0x5626eef7d8f0 msgr2=0x5626ef04b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 shutdown_connections 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.049+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/1955744133 wait complete. 2023-08-11T14:05:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.050+0000 7f7c1efc3b80 20 client.25975 trim_cache size 0 max 16384 2023-08-11T14:05:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.065+0000 7f7c1efc3b80 1 Processor -- start 2023-08-11T14:05:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.065+0000 7f7c1efc3b80 1 -- start start 2023-08-11T14:05:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.065+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef04ff40 0x5626ef04bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.065+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.065+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef04c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef058fe0 con 0x5626ef04ff40 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef055290 con 0x5626ef051790 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef0658f0 con 0x5626ef0508a0 2023-08-11T14:05:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39160/0 (socket says 172.21.15.3:39160) 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3129471366 learned_addr learned my addr 172.21.15.3:0/3129471366 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c09b9c700 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef04c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef04ff40 0x5626ef04bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3129471366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 msgr2=0x5626ef04c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef04c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3129471366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef04ff40 msgr2=0x5626ef04bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef04ff40 0x5626ef04bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.066+0000 7f7c08b9a700 1 -- 172.21.15.3:0/3129471366 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eedced80 con 0x5626ef0508a0 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4095750 tx=0x7f7bf4010d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56f76b9fbfc4710 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7beffff700 1 -- 172.21.15.3:0/3129471366 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40a10e0 con 0x5626ef0508a0 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7beffff700 1 -- 172.21.15.3:0/3129471366 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf4099020 con 0x5626ef0508a0 2023-08-11T14:05:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7beffff700 1 -- 172.21.15.3:0/3129471366 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40a0bd0 con 0x5626ef0508a0 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3129471366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 msgr2=0x5626ef0611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.067+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4095750 tx=0x7f7bf4010d60 comp rx=0 tx=0).stop 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3129471366 shutdown_connections 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef051790 0x5626ef04c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef04ff40 0x5626ef04bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3129471366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef0611c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3129471366 >> 172.21.15.3:0/3129471366 conn(0x5626eef7d8f0 msgr2=0x5626ef057980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3129471366 shutdown_connections 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3129471366 wait complete. 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 -- start start 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.068+0000 7f7c1efc3b80 1 Processor -- start 2023-08-11T14:05:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef05f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef0604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef0609f0 con 0x5626ef051790 2023-08-11T14:05:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef060b60 con 0x5626ef04ff40 2023-08-11T14:05:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ef060d20 con 0x5626ef0508a0 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c1efc3b80 10 client.0 _ll_register_callbacks cb 0x7f7c0001d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c0939b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c0939b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37360/0 (socket says 172.21.15.3:37360) 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c0939b700 1 -- 172.21.15.3:0/3210278048 learned_addr learned my addr 172.21.15.3:0/3210278048 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.069+0000 7f7c09b9c700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef0604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef05f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 -- 172.21.15.3:0/3210278048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 msgr2=0x5626ef05f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef05f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 -- 172.21.15.3:0/3210278048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 msgr2=0x5626ef0604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef0604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 -- 172.21.15.3:0/3210278048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626eee225d0 con 0x5626ef04ff40 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c0939b700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009b20 tx=0x7f7c00005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368f6fbfaba648c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7becff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00022070 con 0x5626ef04ff40 2023-08-11T14:05:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bf409d080 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.070+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7c0001ce20 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c00004250 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 10 client.25988 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5626eedb7c00 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf409d080 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f7c000051c0 con 0x5626ef04ff40 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 20 client.25988 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef0ba890 0x5626ef0bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 4 client.25988 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626eef9b7c0 con 0x5626ef0ba890 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.071+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef0ba890 0x5626ef0bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.072+0000 7f7c08b9a700 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef0ba890 0x5626ef0bccb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2d4279ba1a01c5 server_cookie=7c3fb1ce8267d8e2 in_seq=0 out_seq=0 2023-08-11T14:05:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.072+0000 7f7becff9700 10 client.25988 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.100+0000 7f7becff9700 1 -- 172.21.15.3:0/3210278048 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626eef9b7c0 con 0x5626ef0ba890 2023-08-11T14:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.100+0000 7f7becff9700 10 client.25988 handle_command_reply: tid=0 2023-08-11T14:05:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.100+0000 7f7c1efc3b80 1 client.25988 shutdown 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 msgr2=0x5626ef05ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009b20 tx=0x7f7c00005c40 comp rx=0 tx=0).stop 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 shutdown_connections 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5626ef051790 0x5626ef0604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5626ef0ba890 0x5626ef0bccb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626ef04ff40 0x5626ef05ef60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 --2- 172.21.15.3:0/3210278048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5626ef0508a0 0x5626ef05f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7becff9700 0 client.25988 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.101+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 >> 172.21.15.3:0/3210278048 conn(0x5626eef7d8f0 msgr2=0x5626ef04bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 shutdown_connections 2023-08-11T14:05:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c1efc3b80 1 -- 172.21.15.3:0/3210278048 wait complete. 2023-08-11T14:05:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c1efc3b80 20 client.25988 trim_cache size 0 max 16384 2023-08-11T14:05:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:05:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:05:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:05:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 msgr2=0x7f7bf00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 0x7f7bf00a2410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7c0419cb80 tx=0x7f7c00009260 comp rx=0 tx=0).stop 2023-08-11T14:05:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 msgr2=0x7f7c0419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.102+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400ebf0 tx=0x7f7bf400ef00 comp rx=0 tx=0).stop 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 shutdown_connections 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c0410cf00 0x7f7c0419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c04073380 0x7f7c0419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7bf009ff50 0x7f7bf00a2410 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 --2- 172.21.15.3:0/3122621030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c04074960 0x7f7c0419d370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 >> 172.21.15.3:0/3122621030 conn(0x7f7c040fbe60 msgr2=0x7f7c040fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 shutdown_connections 2023-08-11T14:05:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:32.103+0000 7f7c0b5ff700 1 -- 172.21.15.3:0/3122621030 wait complete. 2023-08-11T14:05:32.226 INFO:teuthology.orchestra.run.smithi003.stderr:33600000 of 50000000 tuples (67%) done (elapsed 77.73 s, remaining 37.94 s) 2023-08-11T14:05:32.405 INFO:teuthology.orchestra.run.smithi003.stderr:33700000 of 50000000 tuples (67%) done (elapsed 77.90 s, remaining 37.68 s) 2023-08-11T14:05:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:33800000 of 50000000 tuples (67%) done (elapsed 78.03 s, remaining 37.40 s) 2023-08-11T14:05:32.726 INFO:teuthology.orchestra.run.smithi003.stderr:33900000 of 50000000 tuples (67%) done (elapsed 78.23 s, remaining 37.15 s) 2023-08-11T14:05:32.905 INFO:teuthology.orchestra.run.smithi003.stderr:34000000 of 50000000 tuples (68%) done (elapsed 78.41 s, remaining 36.90 s) 2023-08-11T14:05:33.045 INFO:teuthology.orchestra.run.smithi003.stderr:34100000 of 50000000 tuples (68%) done (elapsed 78.55 s, remaining 36.62 s) 2023-08-11T14:05:33.257 INFO:teuthology.orchestra.run.smithi003.stderr:34200000 of 50000000 tuples (68%) done (elapsed 78.76 s, remaining 36.38 s) 2023-08-11T14:05:33.425 INFO:teuthology.orchestra.run.smithi003.stderr:34300000 of 50000000 tuples (68%) done (elapsed 78.92 s, remaining 36.13 s) 2023-08-11T14:05:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:34400000 of 50000000 tuples (68%) done (elapsed 79.12 s, remaining 35.88 s) 2023-08-11T14:05:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:33 smithi103 ceph-mon[130434]: pgmap v396: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 112 MiB/s wr, 70 op/s 2023-08-11T14:05:33.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:33 smithi003 ceph-mon[97195]: pgmap v396: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 112 MiB/s wr, 70 op/s 2023-08-11T14:05:33.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:33 smithi187 ceph-mon[130973]: pgmap v396: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 112 MiB/s wr, 70 op/s 2023-08-11T14:05:33.794 INFO:teuthology.orchestra.run.smithi003.stderr:34500000 of 50000000 tuples (69%) done (elapsed 79.29 s, remaining 35.63 s) 2023-08-11T14:05:33.927 INFO:teuthology.orchestra.run.smithi003.stderr:34600000 of 50000000 tuples (69%) done (elapsed 79.43 s, remaining 35.35 s) 2023-08-11T14:05:34.108 INFO:teuthology.orchestra.run.smithi003.stderr:34700000 of 50000000 tuples (69%) done (elapsed 79.61 s, remaining 35.10 s) 2023-08-11T14:05:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:34800000 of 50000000 tuples (69%) done (elapsed 79.81 s, remaining 34.86 s) 2023-08-11T14:05:34.505 INFO:teuthology.orchestra.run.smithi003.stderr:34900000 of 50000000 tuples (69%) done (elapsed 80.00 s, remaining 34.62 s) 2023-08-11T14:05:34.664 INFO:teuthology.orchestra.run.smithi003.stderr:35000000 of 50000000 tuples (70%) done (elapsed 80.16 s, remaining 34.36 s) 2023-08-11T14:05:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:35100000 of 50000000 tuples (70%) done (elapsed 80.34 s, remaining 34.10 s) 2023-08-11T14:05:35.045 INFO:teuthology.orchestra.run.smithi003.stderr:35200000 of 50000000 tuples (70%) done (elapsed 80.55 s, remaining 33.87 s) 2023-08-11T14:05:35.236 INFO:teuthology.orchestra.run.smithi003.stderr:35300000 of 50000000 tuples (70%) done (elapsed 80.74 s, remaining 33.62 s) 2023-08-11T14:05:35.487 INFO:teuthology.orchestra.run.smithi003.stderr:35400000 of 50000000 tuples (70%) done (elapsed 80.99 s, remaining 33.40 s) 2023-08-11T14:05:35.641 INFO:teuthology.orchestra.run.smithi003.stderr:35500000 of 50000000 tuples (71%) done (elapsed 81.14 s, remaining 33.14 s) 2023-08-11T14:05:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:35 smithi103 ceph-mon[130434]: pgmap v397: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 MiB/s wr, 64 op/s 2023-08-11T14:05:35.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:35 smithi003 ceph-mon[97195]: pgmap v397: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 MiB/s wr, 64 op/s 2023-08-11T14:05:35.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:35 smithi187 ceph-mon[130973]: pgmap v397: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 MiB/s wr, 64 op/s 2023-08-11T14:05:35.817 INFO:teuthology.orchestra.run.smithi003.stderr:35600000 of 50000000 tuples (71%) done (elapsed 81.32 s, remaining 32.89 s) 2023-08-11T14:05:36.024 INFO:teuthology.orchestra.run.smithi003.stderr:35700000 of 50000000 tuples (71%) done (elapsed 81.52 s, remaining 32.66 s) 2023-08-11T14:05:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:35800000 of 50000000 tuples (71%) done (elapsed 81.75 s, remaining 32.43 s) 2023-08-11T14:05:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:35900000 of 50000000 tuples (71%) done (elapsed 81.93 s, remaining 32.18 s) 2023-08-11T14:05:36.552 INFO:teuthology.orchestra.run.smithi003.stderr:36000000 of 50000000 tuples (72%) done (elapsed 82.05 s, remaining 31.91 s) 2023-08-11T14:05:36.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:36 smithi103 ceph-mon[130434]: pgmap v398: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 MiB/s wr, 63 op/s 2023-08-11T14:05:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:36100000 of 50000000 tuples (72%) done (elapsed 82.26 s, remaining 31.67 s) 2023-08-11T14:05:36.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:36 smithi003 ceph-mon[97195]: pgmap v398: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 MiB/s wr, 63 op/s 2023-08-11T14:05:36.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:36 smithi187 ceph-mon[130973]: pgmap v398: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 MiB/s wr, 63 op/s 2023-08-11T14:05:36.923 INFO:teuthology.orchestra.run.smithi003.stderr:36200000 of 50000000 tuples (72%) done (elapsed 82.42 s, remaining 31.42 s) 2023-08-11T14:05:37.066 INFO:teuthology.orchestra.run.smithi003.stderr:36300000 of 50000000 tuples (72%) done (elapsed 82.57 s, remaining 31.16 s) 2023-08-11T14:05:37.290 INFO:teuthology.orchestra.run.smithi003.stderr:36400000 of 50000000 tuples (72%) done (elapsed 82.79 s, remaining 30.93 s) 2023-08-11T14:05:37.475 INFO:teuthology.orchestra.run.smithi003.stderr:36500000 of 50000000 tuples (73%) done (elapsed 82.97 s, remaining 30.69 s) 2023-08-11T14:05:37.612 INFO:teuthology.orchestra.run.smithi003.stderr:36600000 of 50000000 tuples (73%) done (elapsed 83.11 s, remaining 30.43 s) 2023-08-11T14:05:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:36700000 of 50000000 tuples (73%) done (elapsed 83.35 s, remaining 30.20 s) 2023-08-11T14:05:38.029 INFO:teuthology.orchestra.run.smithi003.stderr:36800000 of 50000000 tuples (73%) done (elapsed 83.53 s, remaining 29.96 s) 2023-08-11T14:05:38.243 INFO:teuthology.orchestra.run.smithi003.stderr:36900000 of 50000000 tuples (73%) done (elapsed 83.74 s, remaining 29.73 s) 2023-08-11T14:05:38.432 INFO:teuthology.orchestra.run.smithi003.stderr:37000000 of 50000000 tuples (74%) done (elapsed 83.93 s, remaining 29.49 s) 2023-08-11T14:05:38.720 INFO:teuthology.orchestra.run.smithi003.stderr:37100000 of 50000000 tuples (74%) done (elapsed 84.22 s, remaining 29.28 s) 2023-08-11T14:05:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:37200000 of 50000000 tuples (74%) done (elapsed 84.36 s, remaining 29.03 s) 2023-08-11T14:05:39.247 INFO:teuthology.orchestra.run.smithi003.stderr:37300000 of 50000000 tuples (74%) done (elapsed 84.75 s, remaining 28.85 s) 2023-08-11T14:05:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:39 smithi103 ceph-mon[130434]: pgmap v399: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 MiB/s wr, 72 op/s 2023-08-11T14:05:39.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:39 smithi187 ceph-mon[130973]: pgmap v399: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 MiB/s wr, 72 op/s 2023-08-11T14:05:39.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:39 smithi003 ceph-mon[97195]: pgmap v399: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 MiB/s wr, 72 op/s 2023-08-11T14:05:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:37400000 of 50000000 tuples (74%) done (elapsed 85.44 s, remaining 28.78 s) 2023-08-11T14:05:40.309 INFO:teuthology.orchestra.run.smithi003.stderr:37500000 of 50000000 tuples (75%) done (elapsed 85.81 s, remaining 28.60 s) 2023-08-11T14:05:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:37600000 of 50000000 tuples (75%) done (elapsed 85.98 s, remaining 28.35 s) 2023-08-11T14:05:40.637 INFO:teuthology.orchestra.run.smithi003.stderr:37700000 of 50000000 tuples (75%) done (elapsed 86.14 s, remaining 28.10 s) 2023-08-11T14:05:40.866 INFO:teuthology.orchestra.run.smithi003.stderr:37800000 of 50000000 tuples (75%) done (elapsed 86.37 s, remaining 27.87 s) 2023-08-11T14:05:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:37900000 of 50000000 tuples (75%) done (elapsed 86.52 s, remaining 27.62 s) 2023-08-11T14:05:41.222 INFO:teuthology.orchestra.run.smithi003.stderr:38000000 of 50000000 tuples (76%) done (elapsed 86.72 s, remaining 27.39 s) 2023-08-11T14:05:41.386 INFO:teuthology.orchestra.run.smithi003.stderr:38100000 of 50000000 tuples (76%) done (elapsed 86.89 s, remaining 27.14 s) 2023-08-11T14:05:41.525 INFO:teuthology.orchestra.run.smithi003.stderr:38200000 of 50000000 tuples (76%) done (elapsed 87.03 s, remaining 26.88 s) 2023-08-11T14:05:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:41 smithi103 ceph-mon[130434]: pgmap v400: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 91 MiB/s wr, 60 op/s 2023-08-11T14:05:41.688 INFO:teuthology.orchestra.run.smithi003.stderr:38300000 of 50000000 tuples (76%) done (elapsed 87.19 s, remaining 26.63 s) 2023-08-11T14:05:41.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:41 smithi003 ceph-mon[97195]: pgmap v400: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 91 MiB/s wr, 60 op/s 2023-08-11T14:05:41.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:41 smithi187 ceph-mon[130973]: pgmap v400: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 91 MiB/s wr, 60 op/s 2023-08-11T14:05:41.870 INFO:teuthology.orchestra.run.smithi003.stderr:38400000 of 50000000 tuples (76%) done (elapsed 87.37 s, remaining 26.39 s) 2023-08-11T14:05:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:38500000 of 50000000 tuples (77%) done (elapsed 87.56 s, remaining 26.15 s) 2023-08-11T14:05:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:38600000 of 50000000 tuples (77%) done (elapsed 87.70 s, remaining 25.90 s) 2023-08-11T14:05:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:38700000 of 50000000 tuples (77%) done (elapsed 87.88 s, remaining 25.66 s) 2023-08-11T14:05:42.586 INFO:teuthology.orchestra.run.smithi003.stderr:38800000 of 50000000 tuples (77%) done (elapsed 88.09 s, remaining 25.43 s) 2023-08-11T14:05:42.750 INFO:teuthology.orchestra.run.smithi003.stderr:38900000 of 50000000 tuples (77%) done (elapsed 88.25 s, remaining 25.18 s) 2023-08-11T14:05:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:39000000 of 50000000 tuples (78%) done (elapsed 88.42 s, remaining 24.94 s) 2023-08-11T14:05:43.107 INFO:teuthology.orchestra.run.smithi003.stderr:39100000 of 50000000 tuples (78%) done (elapsed 88.60 s, remaining 24.70 s) 2023-08-11T14:05:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:39200000 of 50000000 tuples (78%) done (elapsed 88.82 s, remaining 24.47 s) 2023-08-11T14:05:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:39300000 of 50000000 tuples (78%) done (elapsed 88.99 s, remaining 24.23 s) 2023-08-11T14:05:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:43 smithi103 ceph-mon[130434]: pgmap v401: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 124 MiB/s wr, 77 op/s 2023-08-11T14:05:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:39400000 of 50000000 tuples (78%) done (elapsed 89.24 s, remaining 24.01 s) 2023-08-11T14:05:43.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:43 smithi003 ceph-mon[97195]: pgmap v401: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 124 MiB/s wr, 77 op/s 2023-08-11T14:05:43.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:43 smithi187 ceph-mon[130973]: pgmap v401: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 124 MiB/s wr, 77 op/s 2023-08-11T14:05:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:39500000 of 50000000 tuples (79%) done (elapsed 89.36 s, remaining 23.75 s) 2023-08-11T14:05:44.092 INFO:teuthology.orchestra.run.smithi003.stderr:39600000 of 50000000 tuples (79%) done (elapsed 89.59 s, remaining 23.53 s) 2023-08-11T14:05:44.268 INFO:teuthology.orchestra.run.smithi003.stderr:39700000 of 50000000 tuples (79%) done (elapsed 89.77 s, remaining 23.29 s) 2023-08-11T14:05:44.446 INFO:teuthology.orchestra.run.smithi003.stderr:39800000 of 50000000 tuples (79%) done (elapsed 89.95 s, remaining 23.05 s) 2023-08-11T14:05:44.584 INFO:teuthology.orchestra.run.smithi003.stderr:39900000 of 50000000 tuples (79%) done (elapsed 90.08 s, remaining 22.80 s) 2023-08-11T14:05:44.807 INFO:teuthology.orchestra.run.smithi003.stderr:40000000 of 50000000 tuples (80%) done (elapsed 90.31 s, remaining 22.58 s) 2023-08-11T14:05:44.969 INFO:teuthology.orchestra.run.smithi003.stderr:40100000 of 50000000 tuples (80%) done (elapsed 90.47 s, remaining 22.34 s) 2023-08-11T14:05:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:40200000 of 50000000 tuples (80%) done (elapsed 90.69 s, remaining 22.11 s) 2023-08-11T14:05:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 -- 172.21.15.3:0/579899556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68073380 msgr2=0x7fba68073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 --2- 172.21.15.3:0/579899556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68073380 0x7fba68073800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009990 tx=0x7fba5c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 -- 172.21.15.3:0/579899556 shutdown_connections 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 --2- 172.21.15.3:0/579899556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba6810cf00 0x7fba6810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 --2- 172.21.15.3:0/579899556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba68074960 0x7fba68072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 --2- 172.21.15.3:0/579899556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68073380 0x7fba68073800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 -- 172.21.15.3:0/579899556 >> 172.21.15.3:0/579899556 conn(0x7fba680fbe60 msgr2=0x7fba680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 -- 172.21.15.3:0/579899556 shutdown_connections 2023-08-11T14:05:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.273+0000 7fba6cbff700 1 -- 172.21.15.3:0/579899556 wait complete. 2023-08-11T14:05:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 Processor -- start 2023-08-11T14:05:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 -- start start 2023-08-11T14:05:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 0x7fba6819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba6810cf00 0x7fba6819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6819e7e0 con 0x7fba68074960 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6819e950 con 0x7fba68073380 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.275+0000 7fba6cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6819eb10 con 0x7fba6810cf00 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47196/0 (socket says 172.21.15.3:47196) 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba6659c700 1 -- 172.21.15.3:0/1208817396 learned_addr learned my addr 172.21.15.3:0/1208817396 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 0x7fba6819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba66d9d700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba6810cf00 0x7fba6819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 -- 172.21.15.3:0/1208817396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba6810cf00 msgr2=0x7fba6819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba6810cf00 0x7fba6819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 -- 172.21.15.3:0/1208817396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 msgr2=0x7fba6819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 -- 172.21.15.3:0/1208817396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0095f0 con 0x7fba68074960 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba6659c700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.276+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 0x7fba6819d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0098e0 tx=0x7fba5c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.277+0000 7fba577fe700 1 -- 172.21.15.3:0/1208817396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c01d070 con 0x7fba68074960 2023-08-11T14:05:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.277+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6819ed90 con 0x7fba68074960 2023-08-11T14:05:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.277+0000 7fba577fe700 1 -- 172.21.15.3:0/1208817396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5c021410 con 0x7fba68074960 2023-08-11T14:05:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.277+0000 7fba577fe700 1 -- 172.21.15.3:0/1208817396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c005720 con 0x7fba68074960 2023-08-11T14:05:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.277+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba681a5530 con 0x7fba68074960 2023-08-11T14:05:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.280+0000 7fba577fe700 1 -- 172.21.15.3:0/1208817396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fba5c021580 con 0x7fba68074960 2023-08-11T14:05:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.281+0000 7fba80616b80 1 Processor -- start 2023-08-11T14:05:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.281+0000 7fba80616b80 1 -- start start 2023-08-11T14:05:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba577fe700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 0x7fba500a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e82360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba577fe700 1 -- 172.21.15.3:0/1208817396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba5c0cc7f0 con 0x7fba68074960 2023-08-11T14:05:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e83790 0x560ad3e83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3ddb5d0 con 0x560ad3e81f40 2023-08-11T14:05:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3dd8ef0 con 0x560ad3e83790 2023-08-11T14:05:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba80616b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3dd47e0 con 0x560ad3e828a0 2023-08-11T14:05:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba65d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba65d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40700/0 (socket says 172.21.15.3:40700) 2023-08-11T14:05:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba65d9b700 1 -- 172.21.15.3:0/165549762 learned_addr learned my addr 172.21.15.3:0/165549762 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba6659c700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e82360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.282+0000 7fba66d9d700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 0x7fba500a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 -- 172.21.15.3:0/165549762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e83790 msgr2=0x560ad3e83c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba66d9d700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e83790 0x560ad3e83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e83790 0x560ad3e83c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 -- 172.21.15.3:0/165549762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 msgr2=0x560ad3e82360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e82360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 -- 172.21.15.3:0/165549762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad3c545d0 con 0x560ad3e828a0 2023-08-11T14:05:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba6659c700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e82360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba65d9b700 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba5c096450 tx=0x7fba5c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8da5da36d0d2ab0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba54ff9700 1 -- 172.21.15.3:0/165549762 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c09dd80 con 0x560ad3e828a0 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba66d9d700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 0x7fba500a2410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x560ad3e82f70 tx=0x7fba60009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba54ff9700 1 -- 172.21.15.3:0/165549762 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5c0a1cd0 con 0x560ad3e828a0 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.283+0000 7fba54ff9700 1 -- 172.21.15.3:0/165549762 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0996a0 con 0x560ad3e828a0 2023-08-11T14:05:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 -- 172.21.15.3:0/165549762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 msgr2=0x560ad3e831b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba5c096450 tx=0x7fba5c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 -- 172.21.15.3:0/165549762 shutdown_connections 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e83790 0x560ad3e83c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e82360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 --2- 172.21.15.3:0/165549762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e831b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 -- 172.21.15.3:0/165549762 >> 172.21.15.3:0/165549762 conn(0x560ad3daf8f0 msgr2=0x560ad3e7e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 -- 172.21.15.3:0/165549762 shutdown_connections 2023-08-11T14:05:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.284+0000 7fba80616b80 1 -- 172.21.15.3:0/165549762 wait complete. 2023-08-11T14:05:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 -- start start 2023-08-11T14:05:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 Processor -- start 2023-08-11T14:05:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e828a0 0x560ad3e8bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e92dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3d72a60 con 0x560ad3e828a0 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3d72760 con 0x560ad3e81f40 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3c568c0 con 0x560ad3e83790 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.285+0000 7fba80616b80 10 client.0 _ll_register_callbacks cb 0x560ad3d75a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47220/0 (socket says 172.21.15.3:47220) 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 -- 172.21.15.3:0/3649149920 learned_addr learned my addr 172.21.15.3:0/3649149920 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba65d9b700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e828a0 0x560ad3e8bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba66d9d700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e92dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 -- 172.21.15.3:0/3649149920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 msgr2=0x560ad3e92dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e92dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 -- 172.21.15.3:0/3649149920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e828a0 msgr2=0x560ad3e8bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e828a0 0x560ad3e8bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 -- 172.21.15.3:0/3649149920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba681a5530 con 0x560ad3e81f40 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba6659c700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba58012cf0 tx=0x7fba58009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb36ca59476569a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.286+0000 7fba4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58017070 con 0x560ad3e81f40 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ad3e757e0 con 0x560ad3e81f40 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba58004020 con 0x560ad3e81f40 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba580122e0 con 0x560ad3e81f40 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba80616b80 10 client.25997 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560ad3c00d80 con 0x560ad3e81f40 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ad3e757e0 con 0x560ad3e81f40 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.287+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fba58003840 con 0x560ad3e81f40 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 20 client.25997 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3e97790 0x560ad3e99bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 4 client.25997 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ad3e51480 con 0x560ad3e97790 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba65d9b700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3e97790 0x560ad3e99bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba4dffb700 10 client.25997 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.288+0000 7fba65d9b700 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3e97790 0x560ad3e99bb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc909ff1c3d2029 server_cookie=f7bf365776184c0e in_seq=0 out_seq=0 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba4dffb700 1 -- 172.21.15.3:0/3649149920 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560ad3e51480 con 0x560ad3e97790 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba4dffb700 10 client.25997 handle_command_reply: tid=0 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 client.25997 shutdown 2023-08-11T14:05:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 msgr2=0x560ad3e8b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fba58012cf0 tx=0x7fba58009de0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 shutdown_connections 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e92dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3e97790 0x560ad3e99bb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e8b840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 --2- 172.21.15.3:0/3649149920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e828a0 0x560ad3e8bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 >> 172.21.15.3:0/3649149920 conn(0x560ad3daf8f0 msgr2=0x560ad3e7d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba4dffb700 0 client.25997 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 shutdown_connections 2023-08-11T14:05:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.289+0000 7fba80616b80 1 -- 172.21.15.3:0/3649149920 wait complete. 2023-08-11T14:05:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.290+0000 7fba80616b80 20 client.25997 trim_cache size 0 max 16384 2023-08-11T14:05:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 Processor -- start 2023-08-11T14:05:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 -- start start 2023-08-11T14:05:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e7e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e7dd50 con 0x560ad3e81f40 2023-08-11T14:05:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e7df50 con 0x560ad3e828a0 2023-08-11T14:05:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.311+0000 7fba80616b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e82d60 con 0x560ad3e83790 2023-08-11T14:05:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba6659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba6659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40124/0 (socket says 172.21.15.3:40124) 2023-08-11T14:05:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba6659c700 1 -- 172.21.15.3:0/4283665416 learned_addr learned my addr 172.21.15.3:0/4283665416 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47232/0 (socket says 172.21.15.3:47232) 2023-08-11T14:05:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba66d9d700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e7e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 -- 172.21.15.3:0/4283665416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 msgr2=0x560ad3e7e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e7e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 -- 172.21.15.3:0/4283665416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 msgr2=0x560ad3e9e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba65d9b700 1 -- 172.21.15.3:0/4283665416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad3be9c00 con 0x560ad3e828a0 2023-08-11T14:05:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba6659c700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.312+0000 7fba66d9d700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e7e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba65d9b700 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba5c005f50 tx=0x7fba5c0999e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db5a8fe2bfd859b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba557fa700 1 -- 172.21.15.3:0/4283665416 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c09a480 con 0x560ad3e828a0 2023-08-11T14:05:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba557fa700 1 -- 172.21.15.3:0/4283665416 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5c0982f0 con 0x560ad3e828a0 2023-08-11T14:05:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba557fa700 1 -- 172.21.15.3:0/4283665416 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0a1aa0 con 0x560ad3e828a0 2023-08-11T14:05:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba80616b80 1 -- 172.21.15.3:0/4283665416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 msgr2=0x560ad3e92ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.313+0000 7fba80616b80 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba5c005f50 tx=0x7fba5c0999e0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 -- 172.21.15.3:0/4283665416 shutdown_connections 2023-08-11T14:05:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e83790 0x560ad3e7e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e81f40 0x560ad3e9e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 --2- 172.21.15.3:0/4283665416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e828a0 0x560ad3e92ce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 -- 172.21.15.3:0/4283665416 >> 172.21.15.3:0/4283665416 conn(0x560ad3daf8f0 msgr2=0x560ad3e7e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 -- 172.21.15.3:0/4283665416 shutdown_connections 2023-08-11T14:05:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 -- 172.21.15.3:0/4283665416 wait complete. 2023-08-11T14:05:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 -- start start 2023-08-11T14:05:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.314+0000 7fba80616b80 1 Processor -- start 2023-08-11T14:05:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e83790 0x560ad3e90c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e91300 con 0x560ad3e83790 2023-08-11T14:05:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e91470 con 0x560ad3e81f40 2023-08-11T14:05:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad3e91630 con 0x560ad3e828a0 2023-08-11T14:05:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba80616b80 10 client.0 _ll_register_callbacks cb 0x7fba580127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba6659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47236/0 (socket says 172.21.15.3:47236) 2023-08-11T14:05:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.315+0000 7fba6659c700 1 -- 172.21.15.3:0/1118286002 learned_addr learned my addr 172.21.15.3:0/1118286002 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba66d9d700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e83790 0x560ad3e90c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 -- 172.21.15.3:0/1118286002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 msgr2=0x560ad3e92ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e92ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 -- 172.21.15.3:0/1118286002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e83790 msgr2=0x560ad3e90c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e83790 0x560ad3e90c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 -- 172.21.15.3:0/1118286002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ad3c00d80 con 0x560ad3e81f40 2023-08-11T14:05:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba6659c700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fba58012cf0 tx=0x7fba58003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b2d0f2f49e2907 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.316+0000 7fba4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58017030 con 0x560ad3e81f40 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba5c0a1cd0 con 0x560ad3e81f40 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fba5800fcc0 con 0x560ad3e81f40 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba5800fec0 con 0x560ad3e81f40 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 10 client.26003 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560ad3c545d0 con 0x560ad3e81f40 2023-08-11T14:05:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba5c0a1cd0 con 0x560ad3e81f40 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fba5801c770 con 0x560ad3e81f40 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 20 client.26003 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3eec8b0 0x560ad3eeecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 4 client.26003 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:05:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.317+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ad3dcd7c0 con 0x560ad3eec8b0 2023-08-11T14:05:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.318+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3eec8b0 0x560ad3eeecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.318+0000 7fba65d9b700 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3eec8b0 0x560ad3eeecd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5a6a50c08da1f5 server_cookie=f1497b815974c192 in_seq=0 out_seq=0 2023-08-11T14:05:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.318+0000 7fba4dffb700 10 client.26003 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba4dffb700 1 -- 172.21.15.3:0/1118286002 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ad3dcd7c0 con 0x560ad3eec8b0 2023-08-11T14:05:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba4dffb700 10 client.26003 handle_command_reply: tid=0 2023-08-11T14:05:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 client.26003 shutdown 2023-08-11T14:05:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 msgr2=0x560ad3e92660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fba58012cf0 tx=0x7fba58003960 comp rx=0 tx=0).stop 2023-08-11T14:05:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 shutdown_connections 2023-08-11T14:05:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ad3e83790 0x560ad3e90c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560ad3eec8b0 0x560ad3eeecd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560ad3e81f40 0x560ad3e92660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 --2- 172.21.15.3:0/1118286002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ad3e828a0 0x560ad3e92ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba4dffb700 0 client.26003 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 >> 172.21.15.3:0/1118286002 conn(0x560ad3daf8f0 msgr2=0x560ad3e8aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.355+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 shutdown_connections 2023-08-11T14:05:45.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba80616b80 1 -- 172.21.15.3:0/1118286002 wait complete. 2023-08-11T14:05:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba80616b80 20 client.26003 trim_cache size 0 max 16384 2023-08-11T14:05:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:05:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:05:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:05:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:05:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 msgr2=0x7fba500a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 0x7fba500a2410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x560ad3e82f70 tx=0x7fba60009920 comp rx=0 tx=0).stop 2023-08-11T14:05:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 msgr2=0x7fba6819d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.356+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 0x7fba6819d330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fba5c0098e0 tx=0x7fba5c009580 comp rx=0 tx=0).stop 2023-08-11T14:05:45.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 shutdown_connections 2023-08-11T14:05:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba6810cf00 0x7fba6819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fba5009ff50 0x7fba500a2410 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba68073380 0x7fba6819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 --2- 172.21.15.3:0/1208817396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba68074960 0x7fba6819d330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 >> 172.21.15.3:0/1208817396 conn(0x7fba680fbe60 msgr2=0x7fba680fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 shutdown_connections 2023-08-11T14:05:45.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:45.357+0000 7fba6cbff700 1 -- 172.21.15.3:0/1208817396 wait complete. 2023-08-11T14:05:45.434 INFO:teuthology.orchestra.run.smithi003.stderr:40300000 of 50000000 tuples (80%) done (elapsed 90.93 s, remaining 21.89 s) 2023-08-11T14:05:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:40400000 of 50000000 tuples (80%) done (elapsed 91.05 s, remaining 21.64 s) 2023-08-11T14:05:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:45 smithi103 ceph-mon[130434]: pgmap v402: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 123 MiB/s wr, 74 op/s 2023-08-11T14:05:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:40500000 of 50000000 tuples (81%) done (elapsed 91.24 s, remaining 21.40 s) 2023-08-11T14:05:45.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:45 smithi003 ceph-mon[97195]: pgmap v402: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 123 MiB/s wr, 74 op/s 2023-08-11T14:05:45.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:45 smithi187 ceph-mon[130973]: pgmap v402: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 123 MiB/s wr, 74 op/s 2023-08-11T14:05:45.930 INFO:teuthology.orchestra.run.smithi003.stderr:40600000 of 50000000 tuples (81%) done (elapsed 91.43 s, remaining 21.17 s) 2023-08-11T14:05:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:40700000 of 50000000 tuples (81%) done (elapsed 91.62 s, remaining 20.94 s) 2023-08-11T14:05:46.280 INFO:teuthology.orchestra.run.smithi003.stderr:40800000 of 50000000 tuples (81%) done (elapsed 91.78 s, remaining 20.70 s) 2023-08-11T14:05:46.450 INFO:teuthology.orchestra.run.smithi003.stderr:40900000 of 50000000 tuples (81%) done (elapsed 91.95 s, remaining 20.46 s) 2023-08-11T14:05:46.670 INFO:teuthology.orchestra.run.smithi003.stderr:41000000 of 50000000 tuples (82%) done (elapsed 92.17 s, remaining 20.23 s) 2023-08-11T14:05:47.102 INFO:teuthology.orchestra.run.smithi003.stderr:41100000 of 50000000 tuples (82%) done (elapsed 92.60 s, remaining 20.05 s) 2023-08-11T14:05:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:41200000 of 50000000 tuples (82%) done (elapsed 92.96 s, remaining 19.86 s) 2023-08-11T14:05:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:47 smithi103 ceph-mon[130434]: pgmap v403: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 MiB/s wr, 65 op/s 2023-08-11T14:05:47.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:47 smithi003 ceph-mon[97195]: pgmap v403: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 MiB/s wr, 65 op/s 2023-08-11T14:05:47.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:47 smithi187 ceph-mon[130973]: pgmap v403: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 MiB/s wr, 65 op/s 2023-08-11T14:05:48.084 INFO:teuthology.orchestra.run.smithi003.stderr:41300000 of 50000000 tuples (82%) done (elapsed 93.58 s, remaining 19.71 s) 2023-08-11T14:05:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:41400000 of 50000000 tuples (82%) done (elapsed 93.99 s, remaining 19.53 s) 2023-08-11T14:05:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:41500000 of 50000000 tuples (83%) done (elapsed 94.17 s, remaining 19.29 s) 2023-08-11T14:05:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:41600000 of 50000000 tuples (83%) done (elapsed 94.32 s, remaining 19.04 s) 2023-08-11T14:05:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:41700000 of 50000000 tuples (83%) done (elapsed 94.51 s, remaining 18.81 s) 2023-08-11T14:05:49.212 INFO:teuthology.orchestra.run.smithi003.stderr:41800000 of 50000000 tuples (83%) done (elapsed 94.71 s, remaining 18.58 s) 2023-08-11T14:05:49.378 INFO:teuthology.orchestra.run.smithi003.stderr:41900000 of 50000000 tuples (83%) done (elapsed 94.88 s, remaining 18.34 s) 2023-08-11T14:05:49.540 INFO:teuthology.orchestra.run.smithi003.stderr:42000000 of 50000000 tuples (84%) done (elapsed 95.04 s, remaining 18.10 s) 2023-08-11T14:05:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:49 smithi103 ceph-mon[130434]: pgmap v404: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 103 MiB/s wr, 70 op/s 2023-08-11T14:05:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:42100000 of 50000000 tuples (84%) done (elapsed 95.21 s, remaining 17.87 s) 2023-08-11T14:05:49.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:49 smithi003 ceph-mon[97195]: pgmap v404: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 103 MiB/s wr, 70 op/s 2023-08-11T14:05:49.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:49 smithi187 ceph-mon[130973]: pgmap v404: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 103 MiB/s wr, 70 op/s 2023-08-11T14:05:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:42200000 of 50000000 tuples (84%) done (elapsed 95.40 s, remaining 17.63 s) 2023-08-11T14:05:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:42300000 of 50000000 tuples (84%) done (elapsed 95.60 s, remaining 17.40 s) 2023-08-11T14:05:50.218 INFO:teuthology.orchestra.run.smithi003.stderr:42400000 of 50000000 tuples (84%) done (elapsed 95.72 s, remaining 17.16 s) 2023-08-11T14:05:50.410 INFO:teuthology.orchestra.run.smithi003.stderr:42500000 of 50000000 tuples (85%) done (elapsed 95.91 s, remaining 16.93 s) 2023-08-11T14:05:50.600 INFO:teuthology.orchestra.run.smithi003.stderr:42600000 of 50000000 tuples (85%) done (elapsed 96.10 s, remaining 16.69 s) 2023-08-11T14:05:50.746 INFO:teuthology.orchestra.run.smithi003.stderr:42700000 of 50000000 tuples (85%) done (elapsed 96.25 s, remaining 16.45 s) 2023-08-11T14:05:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:42800000 of 50000000 tuples (85%) done (elapsed 96.43 s, remaining 16.22 s) 2023-08-11T14:05:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:42900000 of 50000000 tuples (85%) done (elapsed 96.62 s, remaining 15.99 s) 2023-08-11T14:05:51.340 INFO:teuthology.orchestra.run.smithi003.stderr:43000000 of 50000000 tuples (86%) done (elapsed 96.84 s, remaining 15.76 s) 2023-08-11T14:05:51.538 INFO:teuthology.orchestra.run.smithi003.stderr:43100000 of 50000000 tuples (86%) done (elapsed 97.04 s, remaining 15.54 s) 2023-08-11T14:05:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:51 smithi103 ceph-mon[130434]: pgmap v405: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 105 MiB/s wr, 65 op/s 2023-08-11T14:05:51.726 INFO:teuthology.orchestra.run.smithi003.stderr:43200000 of 50000000 tuples (86%) done (elapsed 97.23 s, remaining 15.30 s) 2023-08-11T14:05:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:51 smithi003 ceph-mon[97195]: pgmap v405: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 105 MiB/s wr, 65 op/s 2023-08-11T14:05:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:51 smithi187 ceph-mon[130973]: pgmap v405: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 105 MiB/s wr, 65 op/s 2023-08-11T14:05:51.860 INFO:teuthology.orchestra.run.smithi003.stderr:43300000 of 50000000 tuples (86%) done (elapsed 97.36 s, remaining 15.06 s) 2023-08-11T14:05:52.011 INFO:teuthology.orchestra.run.smithi003.stderr:43400000 of 50000000 tuples (86%) done (elapsed 97.51 s, remaining 14.83 s) 2023-08-11T14:05:52.239 INFO:teuthology.orchestra.run.smithi003.stderr:43500000 of 50000000 tuples (87%) done (elapsed 97.74 s, remaining 14.60 s) 2023-08-11T14:05:52.479 INFO:teuthology.orchestra.run.smithi003.stderr:43600000 of 50000000 tuples (87%) done (elapsed 97.98 s, remaining 14.38 s) 2023-08-11T14:05:52.664 INFO:teuthology.orchestra.run.smithi003.stderr:43700000 of 50000000 tuples (87%) done (elapsed 98.16 s, remaining 14.15 s) 2023-08-11T14:05:52.779 INFO:teuthology.orchestra.run.smithi003.stderr:43800000 of 50000000 tuples (87%) done (elapsed 98.28 s, remaining 13.91 s) 2023-08-11T14:05:52.983 INFO:teuthology.orchestra.run.smithi003.stderr:43900000 of 50000000 tuples (87%) done (elapsed 98.48 s, remaining 13.68 s) 2023-08-11T14:05:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:44000000 of 50000000 tuples (88%) done (elapsed 98.64 s, remaining 13.45 s) 2023-08-11T14:05:53.289 INFO:teuthology.orchestra.run.smithi003.stderr:44100000 of 50000000 tuples (88%) done (elapsed 98.79 s, remaining 13.22 s) 2023-08-11T14:05:53.503 INFO:teuthology.orchestra.run.smithi003.stderr:44200000 of 50000000 tuples (88%) done (elapsed 99.00 s, remaining 12.99 s) 2023-08-11T14:05:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:53 smithi103 ceph-mon[130434]: pgmap v406: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 138 MiB/s wr, 84 op/s 2023-08-11T14:05:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:44300000 of 50000000 tuples (88%) done (elapsed 99.18 s, remaining 12.76 s) 2023-08-11T14:05:53.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:53 smithi003 ceph-mon[97195]: pgmap v406: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 138 MiB/s wr, 84 op/s 2023-08-11T14:05:53.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:53 smithi187 ceph-mon[130973]: pgmap v406: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 138 MiB/s wr, 84 op/s 2023-08-11T14:05:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:44400000 of 50000000 tuples (88%) done (elapsed 99.39 s, remaining 12.54 s) 2023-08-11T14:05:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:44500000 of 50000000 tuples (89%) done (elapsed 99.55 s, remaining 12.30 s) 2023-08-11T14:05:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:44600000 of 50000000 tuples (89%) done (elapsed 99.70 s, remaining 12.07 s) 2023-08-11T14:05:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:44700000 of 50000000 tuples (89%) done (elapsed 99.91 s, remaining 11.85 s) 2023-08-11T14:05:54.599 INFO:teuthology.orchestra.run.smithi003.stderr:44800000 of 50000000 tuples (89%) done (elapsed 100.10 s, remaining 11.62 s) 2023-08-11T14:05:54.739 INFO:teuthology.orchestra.run.smithi003.stderr:44900000 of 50000000 tuples (89%) done (elapsed 100.24 s, remaining 11.39 s) 2023-08-11T14:05:55.289 INFO:teuthology.orchestra.run.smithi003.stderr:45000000 of 50000000 tuples (90%) done (elapsed 100.79 s, remaining 11.20 s) 2023-08-11T14:05:55.624 INFO:teuthology.orchestra.run.smithi003.stderr:45100000 of 50000000 tuples (90%) done (elapsed 101.12 s, remaining 10.99 s) 2023-08-11T14:05:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:55 smithi103 ceph-mon[130434]: pgmap v407: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 113 MiB/s wr, 71 op/s 2023-08-11T14:05:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:55 smithi003 ceph-mon[97195]: pgmap v407: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 113 MiB/s wr, 71 op/s 2023-08-11T14:05:55.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:55 smithi187 ceph-mon[130973]: pgmap v407: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 113 MiB/s wr, 71 op/s 2023-08-11T14:05:56.152 INFO:teuthology.orchestra.run.smithi003.stderr:45200000 of 50000000 tuples (90%) done (elapsed 101.65 s, remaining 10.79 s) 2023-08-11T14:05:56.350 INFO:teuthology.orchestra.run.smithi003.stderr:45300000 of 50000000 tuples (90%) done (elapsed 101.85 s, remaining 10.57 s) 2023-08-11T14:05:56.537 INFO:teuthology.orchestra.run.smithi003.stderr:45400000 of 50000000 tuples (90%) done (elapsed 102.04 s, remaining 10.34 s) 2023-08-11T14:05:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:45500000 of 50000000 tuples (91%) done (elapsed 102.18 s, remaining 10.11 s) 2023-08-11T14:05:56.902 INFO:teuthology.orchestra.run.smithi003.stderr:45600000 of 50000000 tuples (91%) done (elapsed 102.40 s, remaining 9.88 s) 2023-08-11T14:05:57.085 INFO:teuthology.orchestra.run.smithi003.stderr:45700000 of 50000000 tuples (91%) done (elapsed 102.59 s, remaining 9.65 s) 2023-08-11T14:05:57.282 INFO:teuthology.orchestra.run.smithi003.stderr:45800000 of 50000000 tuples (91%) done (elapsed 102.78 s, remaining 9.43 s) 2023-08-11T14:05:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:45900000 of 50000000 tuples (91%) done (elapsed 102.91 s, remaining 9.19 s) 2023-08-11T14:05:57.584 INFO:teuthology.orchestra.run.smithi003.stderr:46000000 of 50000000 tuples (92%) done (elapsed 103.08 s, remaining 8.96 s) 2023-08-11T14:05:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:57 smithi103 ceph-mon[130434]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 MiB/s wr, 65 op/s 2023-08-11T14:05:57.788 INFO:teuthology.orchestra.run.smithi003.stderr:46100000 of 50000000 tuples (92%) done (elapsed 103.29 s, remaining 8.74 s) 2023-08-11T14:05:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:57 smithi003 ceph-mon[97195]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 MiB/s wr, 65 op/s 2023-08-11T14:05:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:57 smithi187 ceph-mon[130973]: pgmap v408: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 MiB/s wr, 65 op/s 2023-08-11T14:05:57.986 INFO:teuthology.orchestra.run.smithi003.stderr:46200000 of 50000000 tuples (92%) done (elapsed 103.49 s, remaining 8.51 s) 2023-08-11T14:05:58.115 INFO:teuthology.orchestra.run.smithi003.stderr:46300000 of 50000000 tuples (92%) done (elapsed 103.61 s, remaining 8.28 s) 2023-08-11T14:05:58.326 INFO:teuthology.orchestra.run.smithi003.stderr:46400000 of 50000000 tuples (92%) done (elapsed 103.83 s, remaining 8.06 s) 2023-08-11T14:05:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 -- 172.21.15.3:0/2652292987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c0074960 msgr2=0x7ff2c0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/2652292987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c0074960 0x7ff2c0072db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009990 tx=0x7ff2b0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 -- 172.21.15.3:0/2652292987 shutdown_connections 2023-08-11T14:05:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/2652292987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c010cf00 0x7ff2c010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/2652292987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c0074960 0x7ff2c0072db0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/2652292987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c0073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 -- 172.21.15.3:0/2652292987 >> 172.21.15.3:0/2652292987 conn(0x7ff2c00fbe60 msgr2=0x7ff2c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.524+0000 7ff2c5fff700 1 -- 172.21.15.3:0/2652292987 shutdown_connections 2023-08-11T14:05:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.525+0000 7ff2c5fff700 1 -- 172.21.15.3:0/2652292987 wait complete. 2023-08-11T14:05:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.526+0000 7ff2c5fff700 1 Processor -- start 2023-08-11T14:05:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.526+0000 7ff2c5fff700 1 -- start start 2023-08-11T14:05:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c0074960 0x7ff2c019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:46500000 of 50000000 tuples (93%) done (elapsed 104.03 s, remaining 7.83 s) 2023-08-11T14:05:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 0x7ff2c019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019e7a0 con 0x7ff2c010cf00 2023-08-11T14:05:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019e910 con 0x7ff2c0074960 2023-08-11T14:05:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2c5fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c019ead0 con 0x7ff2c0073380 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50460/0 (socket says 172.21.15.3:50460) 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/70693296 learned_addr learned my addr 172.21.15.3:0/70693296 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bffff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 0x7ff2c019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2beffd700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c0074960 0x7ff2c019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/70693296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c0074960 msgr2=0x7ff2c019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c0074960 0x7ff2c019d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 msgr2=0x7ff2c019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 0x7ff2c019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/70693296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00095f0 con 0x7ff2c0073380 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.527+0000 7ff2bffff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 0x7ff2c019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b680 tx=0x7ff2b0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2bcff9700 1 -- 172.21.15.3:0/70693296 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001d070 con 0x7ff2c0073380 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c019ed50 con 0x7ff2c0073380 2023-08-11T14:05:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2bcff9700 1 -- 172.21.15.3:0/70693296 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b0022470 con 0x7ff2c0073380 2023-08-11T14:05:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2bcff9700 1 -- 172.21.15.3:0/70693296 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0017650 con 0x7ff2c0073380 2023-08-11T14:05:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.528+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c01a54f0 con 0x7ff2c0073380 2023-08-11T14:05:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.532+0000 7ff2bcff9700 1 -- 172.21.15.3:0/70693296 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff2b00225e0 con 0x7ff2c0073380 2023-08-11T14:05:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.533+0000 7ff2d98d1b80 1 Processor -- start 2023-08-11T14:05:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.533+0000 7ff2d98d1b80 1 -- start start 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.533+0000 7ff2bcff9700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 0x7ff2a80a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2bcff9700 1 -- 172.21.15.3:0/70693296 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff2b00ccaa0 con 0x7ff2c0073380 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed689360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2bffff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 0x7ff2a80a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed68a790 0x55f2ed68ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed5e25d0 con 0x55f2ed6898a0 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed5dfef0 con 0x55f2ed68a790 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed5db7e0 con 0x55f2ed688f40 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38810/0 (socket says 172.21.15.3:38810) 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2beffd700 1 -- 172.21.15.3:0/2215587472 learned_addr learned my addr 172.21.15.3:0/2215587472 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed689360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.534+0000 7ff2bffff700 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed68a790 0x55f2ed68ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 -- 172.21.15.3:0/2215587472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 msgr2=0x55f2ed689360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed689360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 -- 172.21.15.3:0/2215587472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed68a790 msgr2=0x55f2ed68ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed68a790 0x55f2ed68ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 -- 172.21.15.3:0/2215587472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2ed45b5d0 con 0x55f2ed6898a0 2023-08-11T14:05:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2beffd700 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009cf0 tx=0x7ff2b4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ec2ca7e71f37ad server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2bffff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 0x7ff2a80a2410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009fd0 tx=0x7ff2b8007730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2a5ffb700 1 -- 172.21.15.3:0/2215587472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4011d80 con 0x55f2ed6898a0 2023-08-11T14:05:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2a5ffb700 1 -- 172.21.15.3:0/2215587472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b40044e0 con 0x55f2ed6898a0 2023-08-11T14:05:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.535+0000 7ff2a5ffb700 1 -- 172.21.15.3:0/2215587472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4005280 con 0x55f2ed6898a0 2023-08-11T14:05:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/2215587472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 msgr2=0x55f2ed68a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009cf0 tx=0x7ff2b4011440 comp rx=0 tx=0).stop 2023-08-11T14:05:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/2215587472 shutdown_connections 2023-08-11T14:05:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed68a790 0x55f2ed68ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed689360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/2215587472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed6898a0 0x55f2ed68a1b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/2215587472 >> 172.21.15.3:0/2215587472 conn(0x55f2ed5b68f0 msgr2=0x55f2ed685920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/2215587472 shutdown_connections 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.536+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/2215587472 wait complete. 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.537+0000 7ff2d98d1b80 1 -- start start 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.537+0000 7ff2d98d1b80 1 Processor -- start 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed693130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed68a790 0x55f2ed69a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed579a60 con 0x55f2ed688f40 2023-08-11T14:05:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed579760 con 0x55f2ed6898a0 2023-08-11T14:05:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed45d8c0 con 0x55f2ed68a790 2023-08-11T14:05:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2d98d1b80 10 client.0 _ll_register_callbacks cb 0x55f2ed57ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38826/0 (socket says 172.21.15.3:38826) 2023-08-11T14:05:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/114599722 learned_addr learned my addr 172.21.15.3:0/114599722 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2beffd700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed693130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.538+0000 7ff2bffff700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed68a790 0x55f2ed69a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/114599722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed68a790 msgr2=0x55f2ed69a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed68a790 0x55f2ed69a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 msgr2=0x55f2ed693130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed693130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/114599722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b4009920 con 0x55f2ed688f40 2023-08-11T14:05:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00966b0 tx=0x7ff2b0022ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99e22b4f37b627f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff29effd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00b2320 con 0x55f2ed688f40 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2ed67c7e0 con 0x55f2ed688f40 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.539+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff2b009a200 con 0x55f2ed688f40 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b009e030 con 0x55f2ed688f40 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 10 client.15969 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f2ed407d80 con 0x55f2ed688f40 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2ed67c7e0 con 0x55f2ed688f40 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff2b00a1c10 con 0x55f2ed688f40 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 20 client.15969 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed69e6e0 0x55f2ed6a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 4 client.15969 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:05:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.540+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2ed658480 con 0x55f2ed69e6e0 2023-08-11T14:05:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.541+0000 7ff2beffd700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed69e6e0 0x55f2ed6a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.541+0000 7ff29effd700 10 client.15969 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.541+0000 7ff2beffd700 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed69e6e0 0x55f2ed6a0b00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7512293aa25a9abd server_cookie=8a1ac2f39d814e17 in_seq=0 out_seq=0 2023-08-11T14:05:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff29effd700 1 -- 172.21.15.3:0/114599722 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f2ed658480 con 0x55f2ed69e6e0 2023-08-11T14:05:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff29effd700 10 client.15969 handle_command_reply: tid=0 2023-08-11T14:05:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 client.15969 shutdown 2023-08-11T14:05:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 msgr2=0x55f2ed692bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00966b0 tx=0x7ff2b0022ba0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 shutdown_connections 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed68a790 0x55f2ed69a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed688f40 0x55f2ed692bf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed69e6e0 0x55f2ed6a0b00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/114599722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed693130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff29effd700 0 client.15969 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 >> 172.21.15.3:0/114599722 conn(0x55f2ed5b68f0 msgr2=0x55f2ed684710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 shutdown_connections 2023-08-11T14:05:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.542+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/114599722 wait complete. 2023-08-11T14:05:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.543+0000 7ff2d98d1b80 20 client.15969 trim_cache size 0 max 16384 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.558+0000 7ff2d98d1b80 1 Processor -- start 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.558+0000 7ff2d98d1b80 1 -- start start 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed69a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed68d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed692140 con 0x55f2ed68a790 2023-08-11T14:05:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed6908b0 con 0x55f2ed6898a0 2023-08-11T14:05:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed68e1c0 con 0x55f2ed688f40 2023-08-11T14:05:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed68d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2beffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed69a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50500/0 (socket says 172.21.15.3:50500) 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/91057708 learned_addr learned my addr 172.21.15.3:0/91057708 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/91057708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 msgr2=0x55f2ed69a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed69a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/91057708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 msgr2=0x55f2ed68d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed68d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/91057708 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2ed3f0c00 con 0x55f2ed688f40 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.559+0000 7ff2beffd700 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed69a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:05:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.560+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0099f00 tx=0x7ff2b00987b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e965771c6360493 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.560+0000 7ff2a67fc700 1 -- 172.21.15.3:0/91057708 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00b2320 con 0x55f2ed688f40 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.560+0000 7ff2a67fc700 1 -- 172.21.15.3:0/91057708 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b009a200 con 0x55f2ed688f40 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.560+0000 7ff2a67fc700 1 -- 172.21.15.3:0/91057708 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00a1af0 con 0x55f2ed688f40 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.560+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/91057708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 msgr2=0x55f2ed6a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0099f00 tx=0x7ff2b00987b0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/91057708 shutdown_connections 2023-08-11T14:05:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed68d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed6a5460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/91057708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed69a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/91057708 >> 172.21.15.3:0/91057708 conn(0x55f2ed5b68f0 msgr2=0x55f2ed680740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/91057708 shutdown_connections 2023-08-11T14:05:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/91057708 wait complete. 2023-08-11T14:05:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 -- start start 2023-08-11T14:05:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.561+0000 7ff2d98d1b80 1 Processor -- start 2023-08-11T14:05:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed6988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed698df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed6994d0 con 0x55f2ed68a790 2023-08-11T14:05:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed699610 con 0x55f2ed6898a0 2023-08-11T14:05:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2ed6997d0 con 0x55f2ed688f40 2023-08-11T14:05:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2d98d1b80 10 client.0 _ll_register_callbacks cb 0x7ff2b0099d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2bf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50512/0 (socket says 172.21.15.3:50512) 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/77472484 learned_addr learned my addr 172.21.15.3:0/77472484 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2bffff700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed698df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.562+0000 7ff2beffd700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed6988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 msgr2=0x55f2ed6988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed6988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/77472484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 msgr2=0x55f2ed698df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed698df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 -- 172.21.15.3:0/77472484 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2ed45b5d0 con 0x55f2ed688f40 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2bf7fe700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005910 tx=0x7ff2b0022ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5b0e77d4ba58ec server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff29effd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:05:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00a1cf0 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2b40044e0 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff2b009e030 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b00ca0f0 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2d98d1b80 10 client.26014 fetch_fsmap learned FSMap version 31 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.563+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f2ed407d80 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b40044e0 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7ff2b00b91d0 con 0x55f2ed688f40 2023-08-11T14:05:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 20 client.26014 populate_metadata read hostname 'smithi003' 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed6f2870 0x55f2ed6f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 4 client.26014 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2ed5d47c0 con 0x55f2ed6f2870 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.564+0000 7ff2beffd700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed6f2870 0x55f2ed6f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:05:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.565+0000 7ff2beffd700 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed6f2870 0x55f2ed6f4c60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06852a9e078abe0 server_cookie=72987656a4a2364b in_seq=0 out_seq=0 2023-08-11T14:05:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.565+0000 7ff29effd700 10 client.26014 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff29effd700 1 -- 172.21.15.3:0/77472484 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2ed5d47c0 con 0x55f2ed6f2870 2023-08-11T14:05:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff29effd700 10 client.26014 handle_command_reply: tid=0 2023-08-11T14:05:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 client.26014 shutdown 2023-08-11T14:05:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 msgr2=0x55f2ed698370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005910 tx=0x7ff2b0022ba0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 shutdown_connections 2023-08-11T14:05:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2ed68a790 0x55f2ed698df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f2ed688f40 0x55f2ed698370 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55f2ed6f2870 0x55f2ed6f4c60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 --2- 172.21.15.3:0/77472484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2ed6898a0 0x55f2ed6988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff29effd700 0 client.26014 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:05:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.596+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 >> 172.21.15.3:0/77472484 conn(0x55f2ed5b68f0 msgr2=0x55f2ed68d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 shutdown_connections 2023-08-11T14:05:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2d98d1b80 1 -- 172.21.15.3:0/77472484 wait complete. 2023-08-11T14:05:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2d98d1b80 20 client.26014 trim_cache size 0 max 16384 2023-08-11T14:05:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:05:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:05:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:05:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:05:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 msgr2=0x7ff2a80a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 0x7ff2a80a2410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009fd0 tx=0x7ff2b8007730 comp rx=0 tx=0).stop 2023-08-11T14:05:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 msgr2=0x7ff2c019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:05:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b680 tx=0x7ff2b0005740 comp rx=0 tx=0).stop 2023-08-11T14:05:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 shutdown_connections 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c010cf00 0x7ff2c019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c0073380 0x7ff2c019cdb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff2a809ff50 0x7ff2a80a2410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 --2- 172.21.15.3:0/70693296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2c0074960 0x7ff2c019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.597+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 >> 172.21.15.3:0/70693296 conn(0x7ff2c00fbe60 msgr2=0x7ff2c00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.598+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 shutdown_connections 2023-08-11T14:05:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:05:58.598+0000 7ff2c5fff700 1 -- 172.21.15.3:0/70693296 wait complete. 2023-08-11T14:05:58.741 INFO:teuthology.orchestra.run.smithi003.stderr:46600000 of 50000000 tuples (93%) done (elapsed 104.24 s, remaining 7.61 s) 2023-08-11T14:05:58.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:05:58 smithi003 ceph-mon[97195]: pgmap v409: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 122 MiB/s wr, 80 op/s 2023-08-11T14:05:58.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:05:58 smithi187 ceph-mon[130973]: pgmap v409: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 122 MiB/s wr, 80 op/s 2023-08-11T14:05:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:46700000 of 50000000 tuples (93%) done (elapsed 104.41 s, remaining 7.38 s) 2023-08-11T14:05:58.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:05:58 smithi103 ceph-mon[130434]: pgmap v409: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 122 MiB/s wr, 80 op/s 2023-08-11T14:05:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:46800000 of 50000000 tuples (93%) done (elapsed 104.55 s, remaining 7.15 s) 2023-08-11T14:05:59.243 INFO:teuthology.orchestra.run.smithi003.stderr:46900000 of 50000000 tuples (93%) done (elapsed 104.74 s, remaining 6.92 s) 2023-08-11T14:05:59.430 INFO:teuthology.orchestra.run.smithi003.stderr:47000000 of 50000000 tuples (94%) done (elapsed 104.93 s, remaining 6.70 s) 2023-08-11T14:05:59.617 INFO:teuthology.orchestra.run.smithi003.stderr:47100000 of 50000000 tuples (94%) done (elapsed 105.12 s, remaining 6.47 s) 2023-08-11T14:05:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:47200000 of 50000000 tuples (94%) done (elapsed 105.26 s, remaining 6.24 s) 2023-08-11T14:05:59.919 INFO:teuthology.orchestra.run.smithi003.stderr:47300000 of 50000000 tuples (94%) done (elapsed 105.42 s, remaining 6.02 s) 2023-08-11T14:06:00.147 INFO:teuthology.orchestra.run.smithi003.stderr:47400000 of 50000000 tuples (94%) done (elapsed 105.65 s, remaining 5.79 s) 2023-08-11T14:06:00.298 INFO:teuthology.orchestra.run.smithi003.stderr:47500000 of 50000000 tuples (95%) done (elapsed 105.80 s, remaining 5.57 s) 2023-08-11T14:06:00.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:00 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:06:00.461 INFO:teuthology.orchestra.run.smithi003.stderr:47600000 of 50000000 tuples (95%) done (elapsed 105.96 s, remaining 5.34 s) 2023-08-11T14:06:00.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:00 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:06:00.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:00 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:06:00.694 INFO:teuthology.orchestra.run.smithi003.stderr:47700000 of 50000000 tuples (95%) done (elapsed 106.19 s, remaining 5.12 s) 2023-08-11T14:06:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:47800000 of 50000000 tuples (95%) done (elapsed 106.41 s, remaining 4.90 s) 2023-08-11T14:06:01.037 INFO:teuthology.orchestra.run.smithi003.stderr:47900000 of 50000000 tuples (95%) done (elapsed 106.54 s, remaining 4.67 s) 2023-08-11T14:06:01.236 INFO:teuthology.orchestra.run.smithi003.stderr:48000000 of 50000000 tuples (96%) done (elapsed 106.74 s, remaining 4.45 s) 2023-08-11T14:06:01.424 INFO:teuthology.orchestra.run.smithi003.stderr:48100000 of 50000000 tuples (96%) done (elapsed 106.92 s, remaining 4.22 s) 2023-08-11T14:06:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: pgmap v410: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 121 MiB/s wr, 74 op/s 2023-08-11T14:06:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:06:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:06:01.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: pgmap v410: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 121 MiB/s wr, 74 op/s 2023-08-11T14:06:01.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:06:01.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:06:01.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: pgmap v410: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 121 MiB/s wr, 74 op/s 2023-08-11T14:06:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:06:01.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:06:01.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:06:01.636 INFO:teuthology.orchestra.run.smithi003.stderr:48200000 of 50000000 tuples (96%) done (elapsed 107.14 s, remaining 4.00 s) 2023-08-11T14:06:01.793 INFO:teuthology.orchestra.run.smithi003.stderr:48300000 of 50000000 tuples (96%) done (elapsed 107.29 s, remaining 3.78 s) 2023-08-11T14:06:01.953 INFO:teuthology.orchestra.run.smithi003.stderr:48400000 of 50000000 tuples (96%) done (elapsed 107.45 s, remaining 3.55 s) 2023-08-11T14:06:02.151 INFO:teuthology.orchestra.run.smithi003.stderr:48500000 of 50000000 tuples (97%) done (elapsed 107.65 s, remaining 3.33 s) 2023-08-11T14:06:02.357 INFO:teuthology.orchestra.run.smithi003.stderr:48600000 of 50000000 tuples (97%) done (elapsed 107.86 s, remaining 3.11 s) 2023-08-11T14:06:02.531 INFO:teuthology.orchestra.run.smithi003.stderr:48700000 of 50000000 tuples (97%) done (elapsed 108.03 s, remaining 2.88 s) 2023-08-11T14:06:02.652 INFO:teuthology.orchestra.run.smithi003.stderr:48800000 of 50000000 tuples (97%) done (elapsed 108.15 s, remaining 2.66 s) 2023-08-11T14:06:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:48900000 of 50000000 tuples (97%) done (elapsed 108.47 s, remaining 2.44 s) 2023-08-11T14:06:03.253 INFO:teuthology.orchestra.run.smithi003.stderr:49000000 of 50000000 tuples (98%) done (elapsed 108.75 s, remaining 2.22 s) 2023-08-11T14:06:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:03 smithi103 ceph-mon[130434]: pgmap v411: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 MiB/s wr, 80 op/s 2023-08-11T14:06:03.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:03 smithi003 ceph-mon[97195]: pgmap v411: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 MiB/s wr, 80 op/s 2023-08-11T14:06:03.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:03 smithi187 ceph-mon[130973]: pgmap v411: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 MiB/s wr, 80 op/s 2023-08-11T14:06:03.981 INFO:teuthology.orchestra.run.smithi003.stderr:49100000 of 50000000 tuples (98%) done (elapsed 109.48 s, remaining 2.01 s) 2023-08-11T14:06:04.659 INFO:teuthology.orchestra.run.smithi003.stderr:49200000 of 50000000 tuples (98%) done (elapsed 110.16 s, remaining 1.79 s) 2023-08-11T14:06:04.794 INFO:teuthology.orchestra.run.smithi003.stderr:49300000 of 50000000 tuples (98%) done (elapsed 110.29 s, remaining 1.57 s) 2023-08-11T14:06:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:49400000 of 50000000 tuples (98%) done (elapsed 110.50 s, remaining 1.34 s) 2023-08-11T14:06:05.181 INFO:teuthology.orchestra.run.smithi003.stderr:49500000 of 50000000 tuples (99%) done (elapsed 110.68 s, remaining 1.12 s) 2023-08-11T14:06:05.395 INFO:teuthology.orchestra.run.smithi003.stderr:49600000 of 50000000 tuples (99%) done (elapsed 110.89 s, remaining 0.89 s) 2023-08-11T14:06:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:49700000 of 50000000 tuples (99%) done (elapsed 111.04 s, remaining 0.67 s) 2023-08-11T14:06:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:05 smithi103 ceph-mon[130434]: pgmap v412: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 101 MiB/s wr, 65 op/s 2023-08-11T14:06:05.738 INFO:teuthology.orchestra.run.smithi003.stderr:49800000 of 50000000 tuples (99%) done (elapsed 111.24 s, remaining 0.45 s) 2023-08-11T14:06:05.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:05 smithi003 ceph-mon[97195]: pgmap v412: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 101 MiB/s wr, 65 op/s 2023-08-11T14:06:05.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:05 smithi187 ceph-mon[130973]: pgmap v412: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 101 MiB/s wr, 65 op/s 2023-08-11T14:06:05.930 INFO:teuthology.orchestra.run.smithi003.stderr:49900000 of 50000000 tuples (99%) done (elapsed 111.43 s, remaining 0.22 s) 2023-08-11T14:06:06.086 INFO:teuthology.orchestra.run.smithi003.stderr:50000000 of 50000000 tuples (100%) done (elapsed 111.59 s, remaining 0.00 s) 2023-08-11T14:06:06.187 INFO:teuthology.orchestra.run.smithi003.stderr:vacuum... 2023-08-11T14:06:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:07 smithi103 ceph-mon[130434]: pgmap v413: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 MiB/s wr, 65 op/s 2023-08-11T14:06:07.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:07 smithi003 ceph-mon[97195]: pgmap v413: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 MiB/s wr, 65 op/s 2023-08-11T14:06:07.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:07 smithi187 ceph-mon[130973]: pgmap v413: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 MiB/s wr, 65 op/s 2023-08-11T14:06:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:09 smithi103 ceph-mon[130434]: pgmap v414: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 MiB/s wr, 82 op/s 2023-08-11T14:06:09.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:09 smithi003 ceph-mon[97195]: pgmap v414: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 MiB/s wr, 82 op/s 2023-08-11T14:06:09.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:09 smithi187 ceph-mon[130973]: pgmap v414: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 MiB/s wr, 82 op/s 2023-08-11T14:06:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:11 smithi103 ceph-mon[130434]: pgmap v415: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 106 MiB/s wr, 70 op/s 2023-08-11T14:06:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 -- 172.21.15.3:0/2197831213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb0073380 msgr2=0x7fcdb0073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/2197831213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb0073380 0x7fcdb0073800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcda00098f0 tx=0x7fcda0009c00 comp rx=0 tx=0).stop 2023-08-11T14:06:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 -- 172.21.15.3:0/2197831213 shutdown_connections 2023-08-11T14:06:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/2197831213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb010cf00 0x7fcdb010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/2197831213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb0072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/2197831213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb0073380 0x7fcdb0073800 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 -- 172.21.15.3:0/2197831213 >> 172.21.15.3:0/2197831213 conn(0x7fcdb00fbe60 msgr2=0x7fcdb00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 -- 172.21.15.3:0/2197831213 shutdown_connections 2023-08-11T14:06:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.753+0000 7fcdb81ff700 1 -- 172.21.15.3:0/2197831213 wait complete. 2023-08-11T14:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.755+0000 7fcdb81ff700 1 Processor -- start 2023-08-11T14:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.755+0000 7fcdb81ff700 1 -- start start 2023-08-11T14:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb010cf00 0x7fcdb019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb019e790 con 0x7fcdb0074960 2023-08-11T14:06:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb019e900 con 0x7fcdb0073380 2023-08-11T14:06:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb81ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb019eac0 con 0x7fcdb010cf00 2023-08-11T14:06:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55798/0 (socket says 172.21.15.3:55798) 2023-08-11T14:06:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/538355240 learned_addr learned my addr 172.21.15.3:0/538355240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb679c700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb010cf00 0x7fcdb019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb579a700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/538355240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb010cf00 msgr2=0x7fcdb019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb010cf00 0x7fcdb019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 msgr2=0x7fcdb019d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb019d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/538355240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0095f0 con 0x7fcdb0073380 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.756+0000 7fcdb579a700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00ec10 tx=0x7fcdac00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcda6ffd700 1 -- 172.21.15.3:0/538355240 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00cd40 con 0x7fcdb0073380 2023-08-11T14:06:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda0009520 con 0x7fcdb0073380 2023-08-11T14:06:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcda6ffd700 1 -- 172.21.15.3:0/538355240 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac0044e0 con 0x7fcdb0073380 2023-08-11T14:06:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcda6ffd700 1 -- 172.21.15.3:0/538355240 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac010690 con 0x7fcdb0073380 2023-08-11T14:06:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.757+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb01a54b0 con 0x7fcdb0073380 2023-08-11T14:06:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.762+0000 7fcda6ffd700 1 -- 172.21.15.3:0/538355240 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcdac003710 con 0x7fcdb0073380 2023-08-11T14:06:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 Processor -- start 2023-08-11T14:06:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 -- start start 2023-08-11T14:06:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 0x564db9e6e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 0x564db9e6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9dc65d0 con 0x564db9e6d8a0 2023-08-11T14:06:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9dc3ef0 con 0x564db9e6e790 2023-08-11T14:06:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.763+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9dbf7e0 con 0x564db9e6cf40 2023-08-11T14:06:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 0x564db9e6e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb679c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 0x564db9e6e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54994/0 (socket says 172.21.15.3:54994) 2023-08-11T14:06:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/413960294 learned_addr learned my addr 172.21.15.3:0/413960294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 0x564db9e6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 -- 172.21.15.3:0/413960294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 msgr2=0x564db9e6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 -- 172.21.15.3:0/413960294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 msgr2=0x564db9e6e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 0x564db9e6e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.764+0000 7fcdb579a700 1 -- 172.21.15.3:0/413960294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db9c3f5d0 con 0x564db9e6e790 2023-08-11T14:06:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcdb579a700 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 0x564db9e6ec10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695f92625d8a6c4e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcd9bfff700 1 -- 172.21.15.3:0/413960294 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda001c040 con 0x564db9e6e790 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcd9bfff700 1 -- 172.21.15.3:0/413960294 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcda000f760 con 0x564db9e6e790 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcd9bfff700 1 -- 172.21.15.3:0/413960294 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00046d0 con 0x564db9e6e790 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcda6ffd700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 0x7fcd9c0a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcda6ffd700 1 -- 172.21.15.3:0/538355240 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcdac014070 con 0x7fcdb0073380 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.765+0000 7fcdb679c700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 0x7fcd9c0a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/413960294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 msgr2=0x564db9e6ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 0x564db9e6ec10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda000bef0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/413960294 shutdown_connections 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6e790 0x564db9e6ec10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/413960294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6d8a0 0x564db9e6e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/413960294 >> 172.21.15.3:0/413960294 conn(0x564db9d9a8f0 msgr2=0x564db9e69920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdb679c700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 0x7fcd9c0a2410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/413960294 shutdown_connections 2023-08-11T14:06:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.766+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/413960294 wait complete. 2023-08-11T14:06:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 -- start start 2023-08-11T14:06:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 Processor -- start 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e76af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e77030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6e790 0x564db9e7e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9d5da60 con 0x564db9e6cf40 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9d5d760 con 0x564db9e6d8a0 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9c418c0 con 0x564db9e6e790 2023-08-11T14:06:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.767+0000 7fcdcbb20b80 10 client.0 _ll_register_callbacks cb 0x564db9d60a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e76af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e77030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e77030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55806/0 (socket says 172.21.15.3:55806) 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb579a700 1 -- 172.21.15.3:0/2401300013 learned_addr learned my addr 172.21.15.3:0/2401300013 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb679c700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6e790 0x564db9e7e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/2401300013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6e790 msgr2=0x564db9e7e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6e790 0x564db9e7e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 msgr2=0x564db9e77030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e77030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/2401300013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb010fca0 con 0x564db9e6cf40 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e76af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a0b20 tx=0x7fcdac009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da26f9fc0133a71a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.768+0000 7fcd997fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac099010 con 0x564db9e6cf40 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db9e607e0 con 0x564db9e6cf40 2023-08-11T14:06:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcdac099940 con 0x564db9e6cf40 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdac098d80 con 0x564db9e6cf40 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 10 client.15987 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564db9bebd80 con 0x564db9e6cf40 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564db9e607e0 con 0x564db9e6cf40 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fcdac098730 con 0x564db9e6cf40 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 20 client.15987 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e829d0 0x564db9e84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 4 client.15987 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.769+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db9e3c480 con 0x564db9e829d0 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.770+0000 7fcdb579a700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e829d0 0x564db9e84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.770+0000 7fcdb579a700 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e829d0 0x564db9e84df0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c4ed803a777cc8 server_cookie=85f4778bd1d945c5 in_seq=0 out_seq=0 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.770+0000 7fcd997fa700 10 client.15987 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcd997fa700 1 -- 172.21.15.3:0/2401300013 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564db9e3c480 con 0x564db9e829d0 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcd997fa700 10 client.15987 handle_command_reply: tid=0 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 client.15987 shutdown 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 msgr2=0x564db9e76af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e76af0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a0b20 tx=0x7fcdac009a10 comp rx=0 tx=0).stop 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 shutdown_connections 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6e790 0x564db9e7e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.771+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e76af0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e829d0 0x564db9e84df0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/2401300013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e77030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 >> 172.21.15.3:0/2401300013 conn(0x564db9d9a8f0 msgr2=0x564db9e68710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcd997fa700 0 client.15987 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 shutdown_connections 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/2401300013 wait complete. 2023-08-11T14:06:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.772+0000 7fcdcbb20b80 20 client.15987 trim_cache size 0 max 16384 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 Processor -- start 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 -- start start 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e7e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e6dee0 con 0x564db9d9a8f0 2023-08-11T14:06:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e74910 con 0x564db9e6d8a0 2023-08-11T14:06:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.787+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e72220 con 0x564db9e6cf40 2023-08-11T14:06:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55000/0 (socket says 172.21.15.3:55000) 2023-08-11T14:06:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/3121902767 learned_addr learned my addr 172.21.15.3:0/3121902767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb679c700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e7e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 -- 172.21.15.3:0/3121902767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 msgr2=0x564db9e75270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e75270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 -- 172.21.15.3:0/3121902767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 msgr2=0x564db9e6d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb579a700 1 -- 172.21.15.3:0/3121902767 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db9bd4c00 con 0x564db9e6cf40 2023-08-11T14:06:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.788+0000 7fcdb679c700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.789+0000 7fcdb579a700 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e7e030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564db9e856c0 tx=0x7fcda000f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb2d541aed4854c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.789+0000 7fcda4ff9700 1 -- 172.21.15.3:0/3121902767 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda001c040 con 0x564db9e6cf40 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.789+0000 7fcda4ff9700 1 -- 172.21.15.3:0/3121902767 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcda00053d0 con 0x564db9e6cf40 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.789+0000 7fcda4ff9700 1 -- 172.21.15.3:0/3121902767 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000b560 con 0x564db9e6cf40 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/3121902767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 msgr2=0x564db9e7e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e7e030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564db9e856c0 tx=0x7fcda000f8d0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/3121902767 shutdown_connections 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9e6d8a0 0x564db9e75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9d9a8f0 0x564db9e6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/3121902767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6cf40 0x564db9e7e030 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/3121902767 >> 172.21.15.3:0/3121902767 conn(0x564db9e75c30 msgr2=0x564db9e74b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/3121902767 shutdown_connections 2023-08-11T14:06:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/3121902767 wait complete. 2023-08-11T14:06:11.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:11 smithi187 ceph-mon[130973]: pgmap v415: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 106 MiB/s wr, 70 op/s 2023-08-11T14:06:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.790+0000 7fcdcbb20b80 1 -- start start 2023-08-11T14:06:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 Processor -- start 2023-08-11T14:06:11.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:11 smithi003 ceph-mon[97195]: pgmap v415: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 106 MiB/s wr, 70 op/s 2023-08-11T14:06:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e7a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6d8a0 0x564db9e7d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e7d600 con 0x564db9e6cf40 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e7d770 con 0x564db9d9a8f0 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e7d930 con 0x564db9e6d8a0 2023-08-11T14:06:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdcbb20b80 10 client.0 _ll_register_callbacks cb 0x7fcdac003a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdb579a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e7a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdb5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55822/0 (socket says 172.21.15.3:55822) 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdb679c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6d8a0 0x564db9e7d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.791+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/1689040812 learned_addr learned my addr 172.21.15.3:0/1689040812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/1689040812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6d8a0 msgr2=0x564db9e7d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6d8a0 0x564db9e7d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/1689040812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 msgr2=0x564db9e7a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e7a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 -- 172.21.15.3:0/1689040812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564db9bebd80 con 0x564db9d9a8f0 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdb5f9b700 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcdac095c60 tx=0x7fcdac0977f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac54770c09584b6d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcd997fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac098930 con 0x564db9d9a8f0 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcdac099940 con 0x564db9d9a8f0 2023-08-11T14:06:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcda000f760 con 0x564db9d9a8f0 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdac098d80 con 0x564db9d9a8f0 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdcbb20b80 10 client.26039 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.792+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fcda8003a00 con 0x564db9d9a8f0 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcda000f760 con 0x564db9d9a8f0 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fcdac0b1140 con 0x564db9d9a8f0 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 20 client.26039 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e6e790 0x564db9ed9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 4 client.26039 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564db9db87c0 con 0x564db9e6e790 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.793+0000 7fcdb579a700 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e6e790 0x564db9ed9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.794+0000 7fcdb579a700 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e6e790 0x564db9ed9150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d5c5df200eb2a5 server_cookie=c8fdfe43128b2706 in_seq=0 out_seq=0 2023-08-11T14:06:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.794+0000 7fcd997fa700 10 client.26039 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcd997fa700 1 -- 172.21.15.3:0/1689040812 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564db9db87c0 con 0x564db9e6e790 2023-08-11T14:06:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcd997fa700 10 client.26039 handle_command_reply: tid=0 2023-08-11T14:06:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 client.26039 shutdown 2023-08-11T14:06:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 msgr2=0x564db9e79b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcdac095c60 tx=0x7fcdac0977f0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 shutdown_connections 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564db9e6d8a0 0x564db9e7d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564db9e6e790 0x564db9ed9150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564db9d9a8f0 0x564db9e79b60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcd997fa700 0 client.26039 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 --2- 172.21.15.3:0/1689040812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564db9e6cf40 0x564db9e7a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 >> 172.21.15.3:0/1689040812 conn(0x564db9e75c30 msgr2=0x564db9e68fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 shutdown_connections 2023-08-11T14:06:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.823+0000 7fcdcbb20b80 1 -- 172.21.15.3:0/1689040812 wait complete. 2023-08-11T14:06:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdcbb20b80 20 client.26039 trim_cache size 0 max 16384 2023-08-11T14:06:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:06:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:06:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:06:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 msgr2=0x7fcd9c0a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 0x7fcd9c0a2410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009fd0 tx=0x7fcda800b040 comp rx=0 tx=0).stop 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 msgr2=0x7fcdb019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00ec10 tx=0x7fcdac00ef20 comp rx=0 tx=0).stop 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 shutdown_connections 2023-08-11T14:06:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdb010cf00 0x7fcdb019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcd9c09ff50 0x7fcd9c0a2410 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdb0073380 0x7fcdb019cda0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 --2- 172.21.15.3:0/538355240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb0074960 0x7fcdb019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.824+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 >> 172.21.15.3:0/538355240 conn(0x7fcdb00fbe60 msgr2=0x7fcdb00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.825+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 shutdown_connections 2023-08-11T14:06:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:11.825+0000 7fcdb81ff700 1 -- 172.21.15.3:0/538355240 wait complete. 2023-08-11T14:06:13.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:13 smithi103 ceph-mon[130434]: pgmap v416: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 91 MiB/s wr, 69 op/s 2023-08-11T14:06:13.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:13 smithi187 ceph-mon[130973]: pgmap v416: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 91 MiB/s wr, 69 op/s 2023-08-11T14:06:13.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:13 smithi003 ceph-mon[97195]: pgmap v416: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 91 MiB/s wr, 69 op/s 2023-08-11T14:06:15.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:15 smithi103 ceph-mon[130434]: pgmap v417: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 77 MiB/s wr, 61 op/s 2023-08-11T14:06:15.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:15 smithi003 ceph-mon[97195]: pgmap v417: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 77 MiB/s wr, 61 op/s 2023-08-11T14:06:15.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:15 smithi187 ceph-mon[130973]: pgmap v417: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 77 MiB/s wr, 61 op/s 2023-08-11T14:06:16.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:16 smithi003 ceph-mon[97195]: pgmap v418: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 MiB/s wr, 63 op/s 2023-08-11T14:06:16.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:16 smithi187 ceph-mon[130973]: pgmap v418: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 MiB/s wr, 63 op/s 2023-08-11T14:06:16.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:16 smithi103 ceph-mon[130434]: pgmap v418: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 MiB/s wr, 63 op/s 2023-08-11T14:06:19.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:19 smithi103 ceph-mon[130434]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 127 MiB/s wr, 75 op/s 2023-08-11T14:06:19.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:19 smithi003 ceph-mon[97195]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 127 MiB/s wr, 75 op/s 2023-08-11T14:06:19.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:19 smithi187 ceph-mon[130973]: pgmap v419: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 127 MiB/s wr, 75 op/s 2023-08-11T14:06:21.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:21 smithi103 ceph-mon[130434]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 132 MiB/s wr, 66 op/s 2023-08-11T14:06:21.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:21 smithi003 ceph-mon[97195]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 132 MiB/s wr, 66 op/s 2023-08-11T14:06:21.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:21 smithi187 ceph-mon[130973]: pgmap v420: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 132 MiB/s wr, 66 op/s 2023-08-11T14:06:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:23 smithi003 ceph-mon[97195]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 198 MiB/s wr, 83 op/s 2023-08-11T14:06:23.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:23 smithi187 ceph-mon[130973]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 198 MiB/s wr, 83 op/s 2023-08-11T14:06:23.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:23 smithi103 ceph-mon[130434]: pgmap v421: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 198 MiB/s wr, 83 op/s 2023-08-11T14:06:24.280 INFO:teuthology.orchestra.run.smithi003.stderr:set primary keys... 2023-08-11T14:06:24.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:24 smithi003 ceph-mon[97195]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 245 MiB/s wr, 91 op/s 2023-08-11T14:06:24.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:24 smithi187 ceph-mon[130973]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 245 MiB/s wr, 91 op/s 2023-08-11T14:06:24.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:24 smithi103 ceph-mon[130434]: pgmap v422: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 245 MiB/s wr, 91 op/s 2023-08-11T14:06:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.096+0000 7f914dfff700 1 -- 172.21.15.3:0/919143501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9148107b90 msgr2=0x7f9148109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.096+0000 7f914dfff700 1 --2- 172.21.15.3:0/919143501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9148107b90 0x7f9148109f80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f914000c310 tx=0x7f914000c620 comp rx=0 tx=0).stop 2023-08-11T14:06:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.101+0000 7f914dfff700 1 -- 172.21.15.3:0/919143501 shutdown_connections 2023-08-11T14:06:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.102+0000 7f914dfff700 1 --2- 172.21.15.3:0/919143501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f914810d010 0x7f914810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.102+0000 7f914dfff700 1 --2- 172.21.15.3:0/919143501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9148107b90 0x7f9148109f80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.102+0000 7f914dfff700 1 --2- 172.21.15.3:0/919143501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.102+0000 7f914dfff700 1 -- 172.21.15.3:0/919143501 >> 172.21.15.3:0/919143501 conn(0x7f914806daf0 msgr2=0x7f914806ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.105+0000 7f914dfff700 1 -- 172.21.15.3:0/919143501 shutdown_connections 2023-08-11T14:06:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.107+0000 7f914dfff700 1 -- 172.21.15.3:0/919143501 wait complete. 2023-08-11T14:06:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.109+0000 7f914dfff700 1 Processor -- start 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 -- start start 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9148107b90 0x7f914819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 0x7f914819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914819e880 con 0x7f914810a550 2023-08-11T14:06:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914819e9f0 con 0x7f9148107b90 2023-08-11T14:06:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.110+0000 7f914dfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914819ebb0 con 0x7f914810d010 2023-08-11T14:06:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.111+0000 7f9147fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 0x7f914819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.111+0000 7f9147fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 0x7f914819e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:54224/0 (socket says 172.21.15.3:54224) 2023-08-11T14:06:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.111+0000 7f9147fff700 1 -- 172.21.15.3:0/3137410314 learned_addr learned my addr 172.21.15.3:0/3137410314 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.111+0000 7f91477fe700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9148107b90 0x7f914819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.112+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 -- 172.21.15.3:0/3137410314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 msgr2=0x7f914819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 0x7f914819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 -- 172.21.15.3:0/3137410314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9148107b90 msgr2=0x7f914819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9148107b90 0x7f914819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 -- 172.21.15.3:0/3137410314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914000c050 con 0x7f914810a550 2023-08-11T14:06:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.113+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914819d350 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f913800c940 tx=0x7f913800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.114+0000 7f9144ff9700 1 -- 172.21.15.3:0/3137410314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380078d0 con 0x7f914810a550 2023-08-11T14:06:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.115+0000 7f9144ff9700 1 -- 172.21.15.3:0/3137410314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913800f450 con 0x7f914810a550 2023-08-11T14:06:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.115+0000 7f9144ff9700 1 -- 172.21.15.3:0/3137410314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e610 con 0x7f914810a550 2023-08-11T14:06:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.115+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914819ee90 con 0x7f914810a550 2023-08-11T14:06:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.115+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914819f640 con 0x7f914810a550 2023-08-11T14:06:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.122+0000 7f91619c3b80 1 Processor -- start 2023-08-11T14:06:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 -- start start 2023-08-11T14:06:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 0x5593775a7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 0x5593775a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593775005d0 con 0x5593775a6f40 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593774fdef0 con 0x5593775a78a0 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.123+0000 7f91619c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593774f97e0 con 0x5593775a8790 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.124+0000 7f9146ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 0x5593775a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.124+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 0x5593775a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.124+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 0x5593775a7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54308/0 (socket says 172.21.15.3:54308) 2023-08-11T14:06:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.124+0000 7f9147fff700 1 -- 172.21.15.3:0/1396865092 learned_addr learned my addr 172.21.15.3:0/1396865092 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 -- 172.21.15.3:0/1396865092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 msgr2=0x5593775a81b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775a81b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 -- 172.21.15.3:0/1396865092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 msgr2=0x5593775a7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 0x5593775a7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 -- 172.21.15.3:0/1396865092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593773795d0 con 0x5593775a8790 2023-08-11T14:06:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9144ff9700 1 -- 172.21.15.3:0/3137410314 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9138007a30 con 0x7f914810a550 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.125+0000 7f9146ffd700 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 0x5593775a8c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91380c8e80 tx=0x7f91380d1a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1885760962994e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.126+0000 7f9135ffb700 1 -- 172.21.15.3:0/1396865092 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380cd0d0 con 0x5593775a8790 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.126+0000 7f9135ffb700 1 -- 172.21.15.3:0/1396865092 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91380d16d0 con 0x5593775a8790 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.126+0000 7f9135ffb700 1 -- 172.21.15.3:0/1396865092 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380d5a80 con 0x5593775a8790 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 -- 172.21.15.3:0/1396865092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 msgr2=0x5593775a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 0x5593775a8c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91380c8e80 tx=0x7f91380d1a60 comp rx=0 tx=0).stop 2023-08-11T14:06:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 -- 172.21.15.3:0/1396865092 shutdown_connections 2023-08-11T14:06:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a8790 0x5593775a8c10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a6f40 0x5593775a7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 --2- 172.21.15.3:0/1396865092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 -- 172.21.15.3:0/1396865092 >> 172.21.15.3:0/1396865092 conn(0x5593774d48f0 msgr2=0x5593775a3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 -- 172.21.15.3:0/1396865092 shutdown_connections 2023-08-11T14:06:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.127+0000 7f91619c3b80 1 -- 172.21.15.3:0/1396865092 wait complete. 2023-08-11T14:06:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.128+0000 7f9144ff9700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 0x7f91300a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.128+0000 7f91619c3b80 1 -- start start 2023-08-11T14:06:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.128+0000 7f9144ff9700 1 -- 172.21.15.3:0/3137410314 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f91380cb760 con 0x7f914810a550 2023-08-11T14:06:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.128+0000 7f9147fff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 0x7f91300a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.129+0000 7f91619c3b80 1 Processor -- start 2023-08-11T14:06:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f9147fff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 0x7f91300a2520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f913c005920 tx=0x7f913c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a78a0 0x5593775b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775b8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559377497a60 con 0x5593775a8790 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559377497760 con 0x5593775a6f40 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937737b8c0 con 0x5593775a78a0 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91619c3b80 10 client.0 _ll_register_callbacks cb 0x55937749aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91477fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91477fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37744/0 (socket says 172.21.15.3:37744) 2023-08-11T14:06:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.130+0000 7f91477fe700 1 -- 172.21.15.3:0/2417267994 learned_addr learned my addr 172.21.15.3:0/2417267994 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f9146ffd700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a78a0 0x5593775b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f9147fff700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 -- 172.21.15.3:0/2417267994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a78a0 msgr2=0x5593775b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a78a0 0x5593775b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 -- 172.21.15.3:0/2417267994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 msgr2=0x5593775b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 -- 172.21.15.3:0/2417267994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914819f640 con 0x5593775a6f40 2023-08-11T14:06:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f9147fff700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f91477fe700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f914000e010 tx=0x7f9140003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2335f6dccb2aa5f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.131+0000 7f912affd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.132+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914000f040 con 0x5593775a6f40 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.132+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9140012d90 con 0x5593775a6f40 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.132+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559377576480 con 0x5593775a6f40 2023-08-11T14:06:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.132+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9140005070 con 0x5593775a6f40 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 10 client.26045 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559377325d80 con 0x5593775a6f40 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559377576480 con 0x5593775a6f40 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f914000c340 con 0x5593775a6f40 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 20 client.26045 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5593775bca00 0x5593775bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 4 client.26045 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:06:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.133+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593774f27c0 con 0x5593775bca00 2023-08-11T14:06:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.134+0000 7f9146ffd700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5593775bca00 0x5593775bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.134+0000 7f9146ffd700 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5593775bca00 0x5593775bee20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50518ff6f46bf3a server_cookie=e37bd4439f6da17d in_seq=0 out_seq=0 2023-08-11T14:06:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.134+0000 7f912affd700 10 client.26045 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.135+0000 7f912affd700 1 -- 172.21.15.3:0/2417267994 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5593774f27c0 con 0x5593775bca00 2023-08-11T14:06:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.135+0000 7f912affd700 10 client.26045 handle_command_reply: tid=0 2023-08-11T14:06:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.135+0000 7f91619c3b80 1 client.26045 shutdown 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 msgr2=0x5593775b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f914000e010 tx=0x7f9140003f90 comp rx=0 tx=0).stop 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 shutdown_connections 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5593775bca00 0x5593775bee20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a6f40 0x5593775b0ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2417267994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a78a0 0x5593775b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 >> 172.21.15.3:0/2417267994 conn(0x5593774d48f0 msgr2=0x5593775a2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f912affd700 0 client.26045 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 shutdown_connections 2023-08-11T14:06:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.139+0000 7f91619c3b80 1 -- 172.21.15.3:0/2417267994 wait complete. 2023-08-11T14:06:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.140+0000 7f91619c3b80 20 client.26045 trim_cache size 0 max 16384 2023-08-11T14:06:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.182+0000 7f91619c3b80 1 Processor -- start 2023-08-11T14:06:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.182+0000 7f91619c3b80 1 -- start start 2023-08-11T14:06:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593775b1ca0 con 0x5593775a8790 2023-08-11T14:06:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593775a8610 con 0x5593775a78a0 2023-08-11T14:06:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.183+0000 7f91619c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593775b0000 con 0x5593775a6f40 2023-08-11T14:06:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.184+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.184+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775a3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54326/0 (socket says 172.21.15.3:54326) 2023-08-11T14:06:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.184+0000 7f9147fff700 1 -- 172.21.15.3:0/3993540353 learned_addr learned my addr 172.21.15.3:0/3993540353 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.184+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.185+0000 7f9146ffd700 1 -- 172.21.15.3:0/3993540353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 msgr2=0x5593775a2f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:06:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.185+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775a2f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.185+0000 7f9146ffd700 1 -- 172.21.15.3:0/3993540353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 msgr2=0x5593775a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.185+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.185+0000 7f9146ffd700 1 -- 172.21.15.3:0/3993540353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55937730ec00 con 0x5593775a78a0 2023-08-11T14:06:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.186+0000 7f9146ffd700 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f913800afd0 tx=0x7f913800fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e30054d2c983f754 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.186+0000 7f91367fc700 1 -- 172.21.15.3:0/3993540353 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380d16d0 con 0x5593775a78a0 2023-08-11T14:06:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.186+0000 7f91367fc700 1 -- 172.21.15.3:0/3993540353 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91380cb970 con 0x5593775a78a0 2023-08-11T14:06:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.186+0000 7f91367fc700 1 -- 172.21.15.3:0/3993540353 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380995a0 con 0x5593775a78a0 2023-08-11T14:06:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.187+0000 7f91619c3b80 1 -- 172.21.15.3:0/3993540353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 msgr2=0x5593775b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.187+0000 7f91619c3b80 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f913800afd0 tx=0x7f913800fc50 comp rx=0 tx=0).stop 2023-08-11T14:06:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.209+0000 7f91619c3b80 1 -- 172.21.15.3:0/3993540353 shutdown_connections 2023-08-11T14:06:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.209+0000 7f91619c3b80 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x5593775a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.209+0000 7f91619c3b80 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.209+0000 7f91619c3b80 1 --2- 172.21.15.3:0/3993540353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7ff0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.209+0000 7f91619c3b80 1 -- 172.21.15.3:0/3993540353 >> 172.21.15.3:0/3993540353 conn(0x5593774d48f0 msgr2=0x5593775ab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.224+0000 7f91619c3b80 1 -- 172.21.15.3:0/3993540353 shutdown_connections 2023-08-11T14:06:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.232+0000 7f91619c3b80 1 -- 172.21.15.3:0/3993540353 wait complete. 2023-08-11T14:06:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.232+0000 7f91619c3b80 1 -- start start 2023-08-11T14:06:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 Processor -- start 2023-08-11T14:06:25.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937760f2a0 con 0x5593775a8790 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937760f3e0 con 0x5593775a78a0 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.233+0000 7f91619c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937760f570 con 0x5593775a6f40 2023-08-11T14:06:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.234+0000 7f91619c3b80 10 client.0 _ll_register_callbacks cb 0x7f9140003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.235+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.235+0000 7f9147fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54332/0 (socket says 172.21.15.3:54332) 2023-08-11T14:06:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.235+0000 7f9147fff700 1 -- 172.21.15.3:0/2228543800 learned_addr learned my addr 172.21.15.3:0/2228543800 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.236+0000 7f9147fff700 1 -- 172.21.15.3:0/2228543800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 msgr2=0x5593775b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.236+0000 7f9147fff700 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.236+0000 7f9147fff700 1 -- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 msgr2=0x5593775b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.236+0000 7f9147fff700 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.236+0000 7f9147fff700 1 -- 172.21.15.3:0/2228543800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593773795d0 con 0x5593775a8790 2023-08-11T14:06:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.238+0000 7f9147fff700 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f913c01bbd0 tx=0x7f913c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d46277d5ca3fd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.239+0000 7f912affd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:06:25.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.240+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c015070 con 0x5593775a8790 2023-08-11T14:06:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.240+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f913c00f9c0 con 0x5593775a8790 2023-08-11T14:06:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.240+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55937759a7e0 con 0x5593775a8790 2023-08-11T14:06:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.241+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f913c00fb80 con 0x5593775a8790 2023-08-11T14:06:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.241+0000 7f91619c3b80 10 client.16005 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:25.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.241+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55937734c1f0 con 0x5593775a8790 2023-08-11T14:06:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.242+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55937759a7e0 con 0x5593775a8790 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.242+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f913c0263d0 con 0x5593775a8790 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 20 client.16005 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559377633070 0x559377635490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:25.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 4 client.16005 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:06:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.249+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593774b2ff0 con 0x559377633070 2023-08-11T14:06:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.258+0000 7f91477fe700 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559377633070 0x559377635490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.261+0000 7f91477fe700 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559377633070 0x559377635490 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14401701264c7fb6 server_cookie=52aa4b5009c51084 in_seq=0 out_seq=0 2023-08-11T14:06:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.263+0000 7f912affd700 10 client.16005 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.284+0000 7f912affd700 1 -- 172.21.15.3:0/2228543800 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593774b2ff0 con 0x559377633070 2023-08-11T14:06:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.284+0000 7f912affd700 10 client.16005 handle_command_reply: tid=0 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.284+0000 7f91619c3b80 1 client.16005 shutdown 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 msgr2=0x55937760ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f913c01bbd0 tx=0x7f913c0047c0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 shutdown_connections 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5593775a8790 0x55937760ed60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593775a6f40 0x5593775b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559377633070 0x559377635490 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 --2- 172.21.15.3:0/2228543800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5593775a78a0 0x5593775b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 >> 172.21.15.3:0/2228543800 conn(0x5593774d48f0 msgr2=0x5593775a2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f912affd700 0 client.16005 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.285+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 shutdown_connections 2023-08-11T14:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.286+0000 7f91619c3b80 1 -- 172.21.15.3:0/2228543800 wait complete. 2023-08-11T14:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.286+0000 7f91619c3b80 20 client.16005 trim_cache size 0 max 16384 2023-08-11T14:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:06:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:06:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:06:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 msgr2=0x7f91300a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 0x7f91300a2520 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f913c005920 tx=0x7f913c00a300 comp rx=0 tx=0).stop 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 msgr2=0x7f914819d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914819d350 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f913800c940 tx=0x7f913800cc50 comp rx=0 tx=0).stop 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 shutdown_connections 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f914810d010 0x7f914819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f91300a0060 0x7f91300a2520 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9148107b90 0x7f914819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 --2- 172.21.15.3:0/3137410314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f914810a550 0x7f914819d350 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 >> 172.21.15.3:0/3137410314 conn(0x7f914806daf0 msgr2=0x7f9148108800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 shutdown_connections 2023-08-11T14:06:25.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:25.287+0000 7f914dfff700 1 -- 172.21.15.3:0/3137410314 wait complete. 2023-08-11T14:06:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:27 smithi103 ceph-mon[130434]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 281 MiB/s wr, 96 op/s 2023-08-11T14:06:27.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:27 smithi003 ceph-mon[97195]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 281 MiB/s wr, 96 op/s 2023-08-11T14:06:27.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:27 smithi187 ceph-mon[130973]: pgmap v423: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 281 MiB/s wr, 96 op/s 2023-08-11T14:06:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:29 smithi103 ceph-mon[130434]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 285 MiB/s wr, 101 op/s 2023-08-11T14:06:29.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:29 smithi003 ceph-mon[97195]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 285 MiB/s wr, 101 op/s 2023-08-11T14:06:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:29 smithi187 ceph-mon[130973]: pgmap v424: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 285 MiB/s wr, 101 op/s 2023-08-11T14:06:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:31 smithi103 ceph-mon[130434]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 241 MiB/s wr, 87 op/s 2023-08-11T14:06:31.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:31 smithi187 ceph-mon[130973]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 241 MiB/s wr, 87 op/s 2023-08-11T14:06:31.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:31 smithi003 ceph-mon[97195]: pgmap v425: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 241 MiB/s wr, 87 op/s 2023-08-11T14:06:33.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:33 smithi103 ceph-mon[130434]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 224 MiB/s wr, 82 op/s 2023-08-11T14:06:33.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:33 smithi003 ceph-mon[97195]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 224 MiB/s wr, 82 op/s 2023-08-11T14:06:33.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:33 smithi187 ceph-mon[130973]: pgmap v426: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 224 MiB/s wr, 82 op/s 2023-08-11T14:06:35.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:35 smithi103 ceph-mon[130434]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 177 MiB/s wr, 68 op/s 2023-08-11T14:06:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:35 smithi187 ceph-mon[130973]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 177 MiB/s wr, 68 op/s 2023-08-11T14:06:35.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:35 smithi003 ceph-mon[97195]: pgmap v427: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 177 MiB/s wr, 68 op/s 2023-08-11T14:06:36.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:36 smithi003 ceph-mon[97195]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 168 MiB/s wr, 64 op/s 2023-08-11T14:06:36.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:36 smithi187 ceph-mon[130973]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 168 MiB/s wr, 64 op/s 2023-08-11T14:06:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:36 smithi103 ceph-mon[130434]: pgmap v428: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 168 MiB/s wr, 64 op/s 2023-08-11T14:06:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/422743528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98100930 msgr2=0x7f0b98100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/422743528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98100930 0x7f0b98100d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b88009990 tx=0x7f0b88009ca0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/422743528 shutdown_connections 2023-08-11T14:06:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/422743528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b981024c0 0x7f0b98108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/422743528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b98100930 0x7f0b98100d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/422743528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b98101b20 0x7f0b98101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/422743528 >> 172.21.15.3:0/422743528 conn(0x7f0b980fbe60 msgr2=0x7f0b980fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/422743528 shutdown_connections 2023-08-11T14:06:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.522+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/422743528 wait complete. 2023-08-11T14:06:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.524+0000 7f0b9d7ff700 1 Processor -- start 2023-08-11T14:06:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.524+0000 7f0b9d7ff700 1 -- start start 2023-08-11T14:06:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b98101b20 0x7f0b98198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b981024c0 0x7f0b98199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9819a3a0 con 0x7f0b98101b20 2023-08-11T14:06:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9819a510 con 0x7f0b98100930 2023-08-11T14:06:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b9d7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9819a6d0 con 0x7f0b981024c0 2023-08-11T14:06:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b96ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b96ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50900/0 (socket says 172.21.15.3:50900) 2023-08-11T14:06:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.525+0000 7f0b96ffd700 1 -- 172.21.15.3:0/1985777555 learned_addr learned my addr 172.21.15.3:0/1985777555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 -- 172.21.15.3:0/1985777555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b981024c0 msgr2=0x7f0b98199d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:06:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b981024c0 0x7f0b98199d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 -- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b98101b20 msgr2=0x7f0b98198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b98101b20 0x7f0b98198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 -- 172.21.15.3:0/1985777555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90009c40 con 0x7f0b98100930 2023-08-11T14:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.526+0000 7f0b96ffd700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b88005720 tx=0x7f0b88009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.527+0000 7f0b7ffff700 1 -- 172.21.15.3:0/1985777555 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8801d070 con 0x7f0b98100930 2023-08-11T14:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.527+0000 7f0b7ffff700 1 -- 172.21.15.3:0/1985777555 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b88021410 con 0x7f0b98100930 2023-08-11T14:06:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.527+0000 7f0b7ffff700 1 -- 172.21.15.3:0/1985777555 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88017670 con 0x7f0b98100930 2023-08-11T14:06:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.527+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b880095f0 con 0x7f0b98100930 2023-08-11T14:06:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.527+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9819ab90 con 0x7f0b98100930 2023-08-11T14:06:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.534+0000 7f0bb1056b80 1 Processor -- start 2023-08-11T14:06:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.535+0000 7f0bb1056b80 1 -- start start 2023-08-11T14:06:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.535+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.536+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbbd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.536+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbdfa0 0x556becbbe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.536+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becb0d5d0 con 0x556becbbd0b0 2023-08-11T14:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.536+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becb0aef0 con 0x556becbbdfa0 2023-08-11T14:06:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.536+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becb067e0 con 0x556becbbc750 2023-08-11T14:06:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.537+0000 7f0b7ffff700 1 -- 172.21.15.3:0/1985777555 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0b88021580 con 0x7f0b98100930 2023-08-11T14:06:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.538+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.538+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:53388/0 (socket says 172.21.15.3:53388) 2023-08-11T14:06:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.538+0000 7f0b977fe700 1 -- 172.21.15.3:0/1279856823 learned_addr learned my addr 172.21.15.3:0/1279856823 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.539+0000 7f0b977fe700 1 -- 172.21.15.3:0/1279856823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbdfa0 msgr2=0x556becbbe420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:06:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.539+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbdfa0 0x556becbbe420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.539+0000 7f0b977fe700 1 -- 172.21.15.3:0/1279856823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 msgr2=0x556becbbd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.539+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbbd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.539+0000 7f0b977fe700 1 -- 172.21.15.3:0/1279856823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bec9865d0 con 0x556becbbc750 2023-08-11T14:06:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.541+0000 7f0b7ffff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 0x7f0b800a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.542+0000 7f0b7ffff700 1 -- 172.21.15.3:0/1985777555 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b880cc860 con 0x7f0b98100930 2023-08-11T14:06:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.548+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e590 tx=0x7f0b90009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551ab09642156ce6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.548+0000 7f0b7d7fa700 1 -- 172.21.15.3:0/1279856823 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90017070 con 0x556becbbc750 2023-08-11T14:06:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.548+0000 7f0b7d7fa700 1 -- 172.21.15.3:0/1279856823 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9000f040 con 0x556becbbc750 2023-08-11T14:06:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.548+0000 7f0b7d7fa700 1 -- 172.21.15.3:0/1279856823 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e050 con 0x556becbbc750 2023-08-11T14:06:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.549+0000 7f0b96ffd700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 0x7f0b800a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.549+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1279856823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 msgr2=0x556becbbcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.549+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e590 tx=0x7f0b90009920 comp rx=0 tx=0).stop 2023-08-11T14:06:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.551+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1279856823 shutdown_connections 2023-08-11T14:06:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.551+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbdfa0 0x556becbbe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.551+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbc750 0x556becbbcb70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.551+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1279856823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbbd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.551+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1279856823 >> 172.21.15.3:0/1279856823 conn(0x556becae18f0 msgr2=0x556becbb0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.553+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1279856823 shutdown_connections 2023-08-11T14:06:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.553+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1279856823 wait complete. 2023-08-11T14:06:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.554+0000 7f0bb1056b80 1 -- start start 2023-08-11T14:06:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.555+0000 7f0bb1056b80 1 Processor -- start 2023-08-11T14:06:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.558+0000 7f0b96ffd700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 0x7f0b800a25e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0b88000c00 tx=0x7f0b88009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.560+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbd0b0 0x556becbc6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbdfa0 0x556becbcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becaa4a60 con 0x556becbbdfa0 2023-08-11T14:06:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becaa4760 con 0x556becbbc750 2023-08-11T14:06:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bec9888c0 con 0x556becbbd0b0 2023-08-11T14:06:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0bb1056b80 10 client.0 _ll_register_callbacks cb 0x556becaa7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50916/0 (socket says 172.21.15.3:50916) 2023-08-11T14:06:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.561+0000 7f0b977fe700 1 -- 172.21.15.3:0/1278216573 learned_addr learned my addr 172.21.15.3:0/1278216573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 -- 172.21.15.3:0/1278216573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbd0b0 msgr2=0x556becbc6950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbd0b0 0x556becbc6950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 -- 172.21.15.3:0/1278216573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbdfa0 msgr2=0x556becbcd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbdfa0 0x556becbcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 -- 172.21.15.3:0/1278216573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9000e240 con 0x556becbbc750 2023-08-11T14:06:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.562+0000 7f0b977fe700 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009430 tx=0x7f0b9000e100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b71397b5f252e00 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.564+0000 7f0b7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:06:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.564+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90017070 con 0x556becbbc750 2023-08-11T14:06:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.564+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b90003a00 con 0x556becbbc750 2023-08-11T14:06:38.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.565+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556becba77e0 con 0x556becbbc750 2023-08-11T14:06:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.566+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b90003bc0 con 0x556becbbc750 2023-08-11T14:06:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.566+0000 7f0bb1056b80 10 client.26057 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:38.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.566+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556bec932d80 con 0x556becbbc750 2023-08-11T14:06:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.568+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556becba77e0 con 0x556becbbc750 2023-08-11T14:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.568+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0b90026060 con 0x556becbbc750 2023-08-11T14:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 20 client.26057 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becbb58d0 0x556becbb7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 4 client.26057 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:06:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.569+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556becb83480 con 0x556becbb58d0 2023-08-11T14:06:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.570+0000 7f0b967fc700 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becbb58d0 0x556becbb7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.571+0000 7f0b967fc700 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becbb58d0 0x556becbb7cf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241eb9c529f22a4b server_cookie=13094817c093741e in_seq=0 out_seq=0 2023-08-11T14:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.571+0000 7f0b7aff5700 10 client.26057 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0b7aff5700 1 -- 172.21.15.3:0/1278216573 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556becb83480 con 0x556becbb58d0 2023-08-11T14:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0b7aff5700 10 client.26057 handle_command_reply: tid=0 2023-08-11T14:06:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 client.26057 shutdown 2023-08-11T14:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 msgr2=0x556becbc6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009430 tx=0x7f0b9000e100 comp rx=0 tx=0).stop 2023-08-11T14:06:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 shutdown_connections 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbdfa0 0x556becbcd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becbb58d0 0x556becbb7cf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbc6410 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/1278216573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbd0b0 0x556becbc6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 >> 172.21.15.3:0/1278216573 conn(0x556becae18f0 msgr2=0x556becbaf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.572+0000 7f0b7aff5700 0 client.26057 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.573+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 shutdown_connections 2023-08-11T14:06:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.573+0000 7f0bb1056b80 1 -- 172.21.15.3:0/1278216573 wait complete. 2023-08-11T14:06:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.573+0000 7f0bb1056b80 20 client.26057 trim_cache size 0 max 16384 2023-08-11T14:06:38.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.593+0000 7f0bb1056b80 1 Processor -- start 2023-08-11T14:06:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.593+0000 7f0bb1056b80 1 -- start start 2023-08-11T14:06:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.593+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbcd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becbc59f0 con 0x556becbbd0b0 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becba77f0 con 0x556becbbc750 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becbe33a0 con 0x556becbbdfa0 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50918/0 (socket says 172.21.15.3:50918) 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0b977fe700 1 -- 172.21.15.3:0/3049752706 learned_addr learned my addr 172.21.15.3:0/3049752706 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0b967fc700 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.594+0000 7f0b977fe700 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 -- 172.21.15.3:0/3049752706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 msgr2=0x556becbb0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbb0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 -- 172.21.15.3:0/3049752706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 msgr2=0x556becbcd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbcd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 -- 172.21.15.3:0/3049752706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bec91bc00 con 0x556becbbc750 2023-08-11T14:06:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.595+0000 7f0b977fe700 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009490 tx=0x7f0b900021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a52bda552f89a1f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.596+0000 7f0b7dffb700 1 -- 172.21.15.3:0/3049752706 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90017030 con 0x556becbbc750 2023-08-11T14:06:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.596+0000 7f0b7dffb700 1 -- 172.21.15.3:0/3049752706 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b90018040 con 0x556becbbc750 2023-08-11T14:06:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.596+0000 7f0b7dffb700 1 -- 172.21.15.3:0/3049752706 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9001f8f0 con 0x556becbbc750 2023-08-11T14:06:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.597+0000 7f0bb1056b80 1 -- 172.21.15.3:0/3049752706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 msgr2=0x556becbb0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.597+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009490 tx=0x7f0b900021f0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.598+0000 7f0bb1056b80 1 -- 172.21.15.3:0/3049752706 shutdown_connections 2023-08-11T14:06:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.598+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbb0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.598+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbc750 0x556becbb0320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.598+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/3049752706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbd0b0 0x556becbcd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.598+0000 7f0bb1056b80 1 -- 172.21.15.3:0/3049752706 >> 172.21.15.3:0/3049752706 conn(0x556becae18f0 msgr2=0x556becbc0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.599+0000 7f0bb1056b80 1 -- 172.21.15.3:0/3049752706 shutdown_connections 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.599+0000 7f0bb1056b80 1 -- 172.21.15.3:0/3049752706 wait complete. 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.600+0000 7f0bb1056b80 1 -- start start 2023-08-11T14:06:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.600+0000 7f0bb1056b80 1 Processor -- start 2023-08-11T14:06:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 0x556becbcd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becbca360 con 0x556becbbc750 2023-08-11T14:06:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becbca4d0 con 0x556becbbd0b0 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556becbca690 con 0x556becbbdfa0 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.601+0000 7f0bb1056b80 10 client.0 _ll_register_callbacks cb 0x7f0b9000e460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.602+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.602+0000 7f0b977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43266/0 (socket says 172.21.15.3:43266) 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.602+0000 7f0b977fe700 1 -- 172.21.15.3:0/507959040 learned_addr learned my addr 172.21.15.3:0/507959040 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.602+0000 7f0b967fc700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 0x556becbcd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.602+0000 7f0b977fe700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 -- 172.21.15.3:0/507959040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 msgr2=0x556becbc9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbc9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 -- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 msgr2=0x556becbcd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 0x556becbcd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 -- 172.21.15.3:0/507959040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bec932d80 con 0x556becbbc750 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b967fc700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 0x556becbcd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b977fe700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009760 tx=0x7f0b90004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bd2a584b5cf1ca server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.604+0000 7f0b7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:06:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9001f850 con 0x556becbbc750 2023-08-11T14:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b9001f9b0 con 0x556becbbc750 2023-08-11T14:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b9000f040 con 0x556becbbc750 2023-08-11T14:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9001fb70 con 0x556becbbc750 2023-08-11T14:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0bb1056b80 10 client.16014 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.605+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556bec9591f0 con 0x556becbbc750 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9000f040 con 0x556becbbc750 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0b90061070 con 0x556becbbc750 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 20 client.16014 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becc2f070 0x556becc31490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 4 client.16014 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:06:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.606+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556becaff7c0 con 0x556becc2f070 2023-08-11T14:06:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.607+0000 7f0b967fc700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becc2f070 0x556becc31490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.608+0000 7f0b967fc700 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becc2f070 0x556becc31490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc0667f6411908f server_cookie=673459cd9f17d002 in_seq=0 out_seq=0 2023-08-11T14:06:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.608+0000 7f0b7aff5700 10 client.16014 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0b7aff5700 1 -- 172.21.15.3:0/507959040 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556becaff7c0 con 0x556becc2f070 2023-08-11T14:06:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0b7aff5700 10 client.16014 handle_command_reply: tid=0 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 client.16014 shutdown 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 msgr2=0x556becbcd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009760 tx=0x7f0b90004460 comp rx=0 tx=0).stop 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 shutdown_connections 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556becbbdfa0 0x556becbc9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556becbbc750 0x556becbcd270 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556becc2f070 0x556becc31490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 --2- 172.21.15.3:0/507959040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556becbbd0b0 0x556becbcd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 >> 172.21.15.3:0/507959040 conn(0x556becae18f0 msgr2=0x556becbb0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0b7aff5700 0 client.16014 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 shutdown_connections 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.628+0000 7f0bb1056b80 1 -- 172.21.15.3:0/507959040 wait complete. 2023-08-11T14:06:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0bb1056b80 20 client.16014 trim_cache size 0 max 16384 2023-08-11T14:06:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:06:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:06:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:06:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:06:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 msgr2=0x7f0b800a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 0x7f0b800a25e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0b88000c00 tx=0x7f0b88009290 comp rx=0 tx=0).stop 2023-08-11T14:06:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 msgr2=0x7f0b98198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b88005720 tx=0x7f0b88009580 comp rx=0 tx=0).stop 2023-08-11T14:06:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 shutdown_connections 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b981024c0 0x7f0b98199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b800a0120 0x7f0b800a25e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b98100930 0x7f0b98198980 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 --2- 172.21.15.3:0/1985777555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b98101b20 0x7f0b98198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.629+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 >> 172.21.15.3:0/1985777555 conn(0x7f0b980fbe60 msgr2=0x7f0b980fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.630+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 shutdown_connections 2023-08-11T14:06:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:38.630+0000 7f0b9d7ff700 1 -- 172.21.15.3:0/1985777555 wait complete. 2023-08-11T14:06:39.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:39 smithi103 ceph-mon[130434]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 185 MiB/s wr, 67 op/s 2023-08-11T14:06:39.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:39 smithi003 ceph-mon[97195]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 185 MiB/s wr, 67 op/s 2023-08-11T14:06:39.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:39 smithi187 ceph-mon[130973]: pgmap v429: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 185 MiB/s wr, 67 op/s 2023-08-11T14:06:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:41 smithi103 ceph-mon[130434]: pgmap v430: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 182 MiB/s wr, 62 op/s 2023-08-11T14:06:41.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:41 smithi003 ceph-mon[97195]: pgmap v430: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 182 MiB/s wr, 62 op/s 2023-08-11T14:06:41.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:41 smithi187 ceph-mon[130973]: pgmap v430: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 182 MiB/s wr, 62 op/s 2023-08-11T14:06:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:43 smithi103 ceph-mon[130434]: pgmap v431: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 207 MiB/s wr, 69 op/s 2023-08-11T14:06:43.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:43 smithi003 ceph-mon[97195]: pgmap v431: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 207 MiB/s wr, 69 op/s 2023-08-11T14:06:43.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:43 smithi187 ceph-mon[130973]: pgmap v431: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 207 MiB/s wr, 69 op/s 2023-08-11T14:06:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:45 smithi103 ceph-mon[130434]: pgmap v432: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 200 MiB/s wr, 70 op/s 2023-08-11T14:06:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:45 smithi003 ceph-mon[97195]: pgmap v432: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 200 MiB/s wr, 70 op/s 2023-08-11T14:06:45.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:45 smithi187 ceph-mon[130973]: pgmap v432: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 200 MiB/s wr, 70 op/s 2023-08-11T14:06:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:47 smithi103 ceph-mon[130434]: pgmap v433: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 182 MiB/s wr, 67 op/s 2023-08-11T14:06:47.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:47 smithi003 ceph-mon[97195]: pgmap v433: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 182 MiB/s wr, 67 op/s 2023-08-11T14:06:47.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:47 smithi187 ceph-mon[130973]: pgmap v433: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 182 MiB/s wr, 67 op/s 2023-08-11T14:06:49.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:49 smithi103 ceph-mon[130434]: pgmap v434: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 157 MiB/s wr, 78 op/s 2023-08-11T14:06:49.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:49 smithi003 ceph-mon[97195]: pgmap v434: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 157 MiB/s wr, 78 op/s 2023-08-11T14:06:49.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:49 smithi187 ceph-mon[130973]: pgmap v434: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 157 MiB/s wr, 78 op/s 2023-08-11T14:06:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:51 smithi103 ceph-mon[130434]: pgmap v435: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 106 MiB/s wr, 66 op/s 2023-08-11T14:06:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:51 smithi003 ceph-mon[97195]: pgmap v435: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 106 MiB/s wr, 66 op/s 2023-08-11T14:06:51.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:51 smithi187 ceph-mon[130973]: pgmap v435: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 106 MiB/s wr, 66 op/s 2023-08-11T14:06:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 -- 172.21.15.3:0/1815528707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc073380 msgr2=0x7efdfc073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 --2- 172.21.15.3:0/1815528707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc073380 0x7efdfc073800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098f0 tx=0x7efdf0009c00 comp rx=0 tx=0).stop 2023-08-11T14:06:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 -- 172.21.15.3:0/1815528707 shutdown_connections 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 --2- 172.21.15.3:0/1815528707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 0x7efdfc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 --2- 172.21.15.3:0/1815528707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc074960 0x7efdfc072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 --2- 172.21.15.3:0/1815528707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc073380 0x7efdfc073800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 -- 172.21.15.3:0/1815528707 >> 172.21.15.3:0/1815528707 conn(0x7efdfc0fbe60 msgr2=0x7efdfc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.802+0000 7efe01dff700 1 -- 172.21.15.3:0/1815528707 shutdown_connections 2023-08-11T14:06:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.803+0000 7efe01dff700 1 -- 172.21.15.3:0/1815528707 wait complete. 2023-08-11T14:06:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.803+0000 7efe01dff700 1 Processor -- start 2023-08-11T14:06:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 -- start start 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 0x7efdfc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 0x7efdfc19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19e790 con 0x7efdfc10cf00 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19e900 con 0x7efdfc074960 2023-08-11T14:06:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efe01dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19eac0 con 0x7efdfc073380 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efdfaffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46698/0 (socket says 172.21.15.3:46698) 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efdfaffd700 1 -- 172.21.15.3:0/2878167357 learned_addr learned my addr 172.21.15.3:0/2878167357 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 0x7efdfc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.804+0000 7efdfbfff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 0x7efdfc19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 -- 172.21.15.3:0/2878167357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 msgr2=0x7efdfc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 0x7efdfc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 -- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 msgr2=0x7efdfc19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 0x7efdfc19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 -- 172.21.15.3:0/2878167357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0095f0 con 0x7efdfc074960 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 0x7efdfc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c00 tx=0x7efdf000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdf8ff9700 1 -- 172.21.15.3:0/2878167357 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf001c070 con 0x7efdfc074960 2023-08-11T14:06:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdf8ff9700 1 -- 172.21.15.3:0/2878167357 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf000f7d0 con 0x7efdfc074960 2023-08-11T14:06:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf0009520 con 0x7efdfc074960 2023-08-11T14:06:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efdf8ff9700 1 -- 172.21.15.3:0/2878167357 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003a80 con 0x7efdfc074960 2023-08-11T14:06:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.805+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1a5300 con 0x7efdfc074960 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.809+0000 7efdf8ff9700 1 -- 172.21.15.3:0/2878167357 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7efdf0020030 con 0x7efdfc074960 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.810+0000 7efe1578bb80 1 Processor -- start 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.810+0000 7efe1578bb80 1 -- start start 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.810+0000 7efdf8ff9700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 0x7efde40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.810+0000 7efdf8ff9700 1 -- 172.21.15.3:0/2878167357 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efdf00cc720 con 0x7efdfc074960 2023-08-11T14:06:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfbfff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 0x7efde40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c742b5d0 con 0x55c7c74d3790 2023-08-11T14:06:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c7428ef0 con 0x55c7c74d1f40 2023-08-11T14:06:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efe1578bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74247e0 con 0x55c7c74d28a0 2023-08-11T14:06:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46708/0 (socket says 172.21.15.3:46708) 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3816784927 learned_addr learned my addr 172.21.15.3:0/3816784927 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfbfff700 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfaffd700 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3816784927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 msgr2=0x55c7c74d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3816784927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 msgr2=0x55c7c74d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.811+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3816784927 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7c72a45d0 con 0x55c7c74d1f40 2023-08-11T14:06:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efdfbfff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 0x7efde40a2410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efdf400f870 tx=0x7efdf400fc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efdec0117c0 tx=0x7efdec011ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ffc061e3622bd6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efde1ffb700 1 -- 172.21.15.3:0/3816784927 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00f600 con 0x55c7c74d1f40 2023-08-11T14:06:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efde1ffb700 1 -- 172.21.15.3:0/3816784927 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec00fc20 con 0x55c7c74d1f40 2023-08-11T14:06:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efde1ffb700 1 -- 172.21.15.3:0/3816784927 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec019b90 con 0x55c7c74d1f40 2023-08-11T14:06:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efe1578bb80 1 -- 172.21.15.3:0/3816784927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 msgr2=0x55c7c74d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.812+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efdec0117c0 tx=0x7efdec011ad0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 -- 172.21.15.3:0/3816784927 shutdown_connections 2023-08-11T14:06:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74d2360 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3816784927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 -- 172.21.15.3:0/3816784927 >> 172.21.15.3:0/3816784927 conn(0x55c7c73ff8f0 msgr2=0x55c7c74ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 -- 172.21.15.3:0/3816784927 shutdown_connections 2023-08-11T14:06:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 -- 172.21.15.3:0/3816784927 wait complete. 2023-08-11T14:06:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 -- start start 2023-08-11T14:06:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.813+0000 7efe1578bb80 1 Processor -- start 2023-08-11T14:06:51.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c73c2a60 con 0x55c7c74d3790 2023-08-11T14:06:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c73c2760 con 0x55c7c74d1f40 2023-08-11T14:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c72a68c0 con 0x55c7c74d28a0 2023-08-11T14:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efe1578bb80 10 client.0 _ll_register_callbacks cb 0x55c7c73c5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:51.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46714/0 (socket says 172.21.15.3:46714) 2023-08-11T14:06:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 -- 172.21.15.3:0/2680414812 learned_addr learned my addr 172.21.15.3:0/2680414812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfbfff700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 -- 172.21.15.3:0/2680414812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 msgr2=0x55c7c74dc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74dc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 -- 172.21.15.3:0/2680414812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 msgr2=0x55c7c74e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.814+0000 7efdfb7fe700 1 -- 172.21.15.3:0/2680414812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec011440 con 0x55c7c74d1f40 2023-08-11T14:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efdec019450 tx=0x7efdec005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bdef930e81e0af server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efddaffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec019ae0 con 0x55c7c74d1f40 2023-08-11T14:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7c74c57e0 con 0x55c7c74d1f40 2023-08-11T14:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efdec019c40 con 0x55c7c74d1f40 2023-08-11T14:06:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdec019e00 con 0x55c7c74d1f40 2023-08-11T14:06:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efe1578bb80 10 client.26075 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.815+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c7c7250d80 con 0x55c7c74d1f40 2023-08-11T14:06:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7c74c57e0 con 0x55c7c74d1f40 2023-08-11T14:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7efdec005400 con 0x55c7c74d1f40 2023-08-11T14:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:51.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 20 client.26075 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c74e79c0 0x55c7c74e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 4 client.26075 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:06:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7c74a1480 con 0x55c7c74e79c0 2023-08-11T14:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.816+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c74e79c0 0x55c7c74e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efdfaffd700 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c74e79c0 0x55c7c74e9de0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b855ef4d426885b server_cookie=a9633cdd3ed769d2 in_seq=0 out_seq=0 2023-08-11T14:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efddaffd700 10 client.26075 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efddaffd700 1 -- 172.21.15.3:0/2680414812 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c7c74a1480 con 0x55c7c74e79c0 2023-08-11T14:06:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efddaffd700 10 client.26075 handle_command_reply: tid=0 2023-08-11T14:06:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 client.26075 shutdown 2023-08-11T14:06:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 msgr2=0x55c7c74dbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efdec019450 tx=0x7efdec005af0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 shutdown_connections 2023-08-11T14:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c74e79c0 0x55c7c74e9de0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d1f40 0x55c7c74dbc20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 --2- 172.21.15.3:0/2680414812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d28a0 0x55c7c74dc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 >> 172.21.15.3:0/2680414812 conn(0x55c7c73ff8f0 msgr2=0x55c7c74cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.817+0000 7efddaffd700 0 client.26075 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.818+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 shutdown_connections 2023-08-11T14:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.818+0000 7efe1578bb80 1 -- 172.21.15.3:0/2680414812 wait complete. 2023-08-11T14:06:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.818+0000 7efe1578bb80 20 client.26075 trim_cache size 0 max 16384 2023-08-11T14:06:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 Processor -- start 2023-08-11T14:06:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 -- start start 2023-08-11T14:06:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 0x55c7c74ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74c5810 con 0x55c7c74d1f40 2023-08-11T14:06:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74d2fd0 con 0x55c7c74d28a0 2023-08-11T14:06:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.833+0000 7efe1578bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74d2d60 con 0x55c7c74d3790 2023-08-11T14:06:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 0x55c7c74ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43246/0 (socket says 172.21.15.3:43246) 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 -- 172.21.15.3:0/748102047 learned_addr learned my addr 172.21.15.3:0/748102047 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfaffd700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 -- 172.21.15.3:0/748102047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 msgr2=0x55c7c74ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 0x55c7c74ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 -- 172.21.15.3:0/748102047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 msgr2=0x55c7c74e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfb7fe700 1 -- 172.21.15.3:0/748102047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7c7239c00 con 0x55c7c74d1f40 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfaffd700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.834+0000 7efdfbfff700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 0x55c7c74ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.835+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efdec005d10 tx=0x7efdec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8383a46fe134e92c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.835+0000 7efde27fc700 1 -- 172.21.15.3:0/748102047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec009310 con 0x55c7c74d1f40 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.835+0000 7efde27fc700 1 -- 172.21.15.3:0/748102047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec022550 con 0x55c7c74d1f40 2023-08-11T14:06:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.835+0000 7efde27fc700 1 -- 172.21.15.3:0/748102047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec004290 con 0x55c7c74d1f40 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- 172.21.15.3:0/748102047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 msgr2=0x55c7c74d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efdec005d10 tx=0x7efdec011ad0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- 172.21.15.3:0/748102047 shutdown_connections 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d3790 0x55c7c74ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d1f40 0x55c7c74d3b70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 --2- 172.21.15.3:0/748102047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- 172.21.15.3:0/748102047 >> 172.21.15.3:0/748102047 conn(0x55c7c73ff8f0 msgr2=0x55c7c74d66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- 172.21.15.3:0/748102047 shutdown_connections 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- 172.21.15.3:0/748102047 wait complete. 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 -- start start 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.836+0000 7efe1578bb80 1 Processor -- start 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74e28f0 con 0x55c7c74d3790 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74e2a60 con 0x55c7c74d28a0 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7c74e2c20 con 0x55c7c74d1f40 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efe1578bb80 10 client.0 _ll_register_callbacks cb 0x7efdf0095aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58994/0 (socket says 172.21.15.3:58994) 2023-08-11T14:06:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3703479346 learned_addr learned my addr 172.21.15.3:0/3703479346 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfaffd700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 msgr2=0x55c7c74e13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3703479346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 msgr2=0x55c7c74e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.837+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efdfb7fe700 1 -- 172.21.15.3:0/3703479346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc1a5300 con 0x55c7c74d1f40 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efdfb7fe700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdec037040 tx=0x7efdec00f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4603ebfc3f1de3f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:06:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efddaffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec019ce0 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdec00fc20 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7efdec019e40 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdec005a00 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efe1578bb80 10 client.26074 fetch_fsmap learned FSMap version 31 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.838+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7efdf400fdb0 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec00fc20 con 0x55c7c74d1f40 2023-08-11T14:06:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7efdec04a070 con 0x55c7c74d1f40 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 10 client.26074 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 20 client.26074 populate_metadata read hostname 'smithi003' 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c753c0d0 0x55c7c753e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 4 client.26074 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c7c741d7c0 con 0x55c7c753c0d0 2023-08-11T14:06:51.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.839+0000 7efdfaffd700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c753c0d0 0x55c7c753e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.840+0000 7efdfaffd700 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c753c0d0 0x55c7c753e4f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334fadf99b0fae8d server_cookie=3a39446da292d0d2 in_seq=0 out_seq=0 2023-08-11T14:06:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.840+0000 7efddaffd700 10 client.26074 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efddaffd700 1 -- 172.21.15.3:0/3703479346 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c7c741d7c0 con 0x55c7c753c0d0 2023-08-11T14:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efddaffd700 10 client.26074 handle_command_reply: tid=0 2023-08-11T14:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 client.26074 shutdown 2023-08-11T14:06:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 msgr2=0x55c7c74e0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efdec037040 tx=0x7efdec00f9c0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 shutdown_connections 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c7c74d3790 0x55c7c74e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7c74d1f40 0x55c7c74e0e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c7c753c0d0 0x55c7c753e4f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 --2- 172.21.15.3:0/3703479346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7c74d28a0 0x55c7c74e13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efddaffd700 0 client.26074 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.871+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 >> 172.21.15.3:0/3703479346 conn(0x55c7c73ff8f0 msgr2=0x55c7c74ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 shutdown_connections 2023-08-11T14:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe1578bb80 1 -- 172.21.15.3:0/3703479346 wait complete. 2023-08-11T14:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe1578bb80 20 client.26074 trim_cache size 0 max 16384 2023-08-11T14:06:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:06:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 msgr2=0x7efde40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 0x7efde40a2410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efdf400f870 tx=0x7efdf400fc30 comp rx=0 tx=0).stop 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 msgr2=0x7efdfc19d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.872+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c00 tx=0x7efdf000b790 comp rx=0 tx=0).stop 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 shutdown_connections 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdfc10cf00 0x7efdfc19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc073380 0x7efdfc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7efde409ff50 0x7efde40a2410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 --2- 172.21.15.3:0/2878167357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdfc074960 0x7efdfc19d2e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:06:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 >> 172.21.15.3:0/2878167357 conn(0x7efdfc0fbe60 msgr2=0x7efdfc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:06:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 shutdown_connections 2023-08-11T14:06:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:06:51.873+0000 7efe01dff700 1 -- 172.21.15.3:0/2878167357 wait complete. 2023-08-11T14:06:53.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:53 smithi103 ceph-mon[130434]: pgmap v436: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 MiB/s wr, 73 op/s 2023-08-11T14:06:53.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:53 smithi003 ceph-mon[97195]: pgmap v436: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 MiB/s wr, 73 op/s 2023-08-11T14:06:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:53 smithi187 ceph-mon[130973]: pgmap v436: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 99 MiB/s wr, 73 op/s 2023-08-11T14:06:55.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:55 smithi103 ceph-mon[130434]: pgmap v437: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 MiB/s wr, 71 op/s 2023-08-11T14:06:55.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:55 smithi003 ceph-mon[97195]: pgmap v437: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 MiB/s wr, 71 op/s 2023-08-11T14:06:55.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:55 smithi187 ceph-mon[130973]: pgmap v437: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 MiB/s wr, 71 op/s 2023-08-11T14:06:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:57 smithi103 ceph-mon[130434]: pgmap v438: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 76 MiB/s wr, 71 op/s 2023-08-11T14:06:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:57 smithi003 ceph-mon[97195]: pgmap v438: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 76 MiB/s wr, 71 op/s 2023-08-11T14:06:57.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:57 smithi187 ceph-mon[130973]: pgmap v438: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 76 MiB/s wr, 71 op/s 2023-08-11T14:06:57.856 INFO:teuthology.orchestra.run.smithi003.stderr:done. 2023-08-11T14:06:57.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-08-11T14:06:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:starting vacuum...end. 2023-08-11T14:06:59.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:06:59 smithi103 ceph-mon[130434]: pgmap v439: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-08-11T14:06:59.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:06:59 smithi003 ceph-mon[97195]: pgmap v439: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-08-11T14:06:59.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:06:59 smithi187 ceph-mon[130973]: pgmap v439: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 109 MiB/s wr, 86 op/s 2023-08-11T14:07:00.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:00 smithi003 ceph-mon[97195]: pgmap v440: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 119 MiB/s wr, 77 op/s 2023-08-11T14:07:00.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:00 smithi187 ceph-mon[130973]: pgmap v440: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 119 MiB/s wr, 77 op/s 2023-08-11T14:07:00.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:00 smithi103 ceph-mon[130434]: pgmap v440: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 119 MiB/s wr, 77 op/s 2023-08-11T14:07:01.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:07:01.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:07:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:07:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 5.0 s, 948.0 tps, lat 100.322 ms stddev 127.531 2023-08-11T14:07:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:03 smithi103 ceph-mon[130434]: pgmap v441: 129 pgs: 129 active+clean; 8.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 162 MiB/s wr, 110 op/s 2023-08-11T14:07:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:07:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:07:03.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:07:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:03 smithi187 ceph-mon[130973]: pgmap v441: 129 pgs: 129 active+clean; 8.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 162 MiB/s wr, 110 op/s 2023-08-11T14:07:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:07:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:07:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:07:03.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:03 smithi003 ceph-mon[97195]: pgmap v441: 129 pgs: 129 active+clean; 8.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 162 MiB/s wr, 110 op/s 2023-08-11T14:07:03.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:07:03.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:07:03.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:07:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.128+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/905181832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 msgr2=0x7fbf24100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.128+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/905181832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf24100e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009920 tx=0x7fbf10009c30 comp rx=0 tx=0).stop 2023-08-11T14:07:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.130+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/905181832 shutdown_connections 2023-08-11T14:07:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.130+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/905181832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf241013b0 0x7fbf24107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.130+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/905181832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf240ff820 0x7fbf240ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.130+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/905181832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf24100e70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.130+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/905181832 >> 172.21.15.3:0/905181832 conn(0x7fbf240fad70 msgr2=0x7fbf240fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.137+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/905181832 shutdown_connections 2023-08-11T14:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.146+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/905181832 wait complete. 2023-08-11T14:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.147+0000 7fbf2b1ff700 1 Processor -- start 2023-08-11T14:07:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.148+0000 7fbf2b1ff700 1 -- start start 2023-08-11T14:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.148+0000 7fbf2b1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf240ff820 0x7fbf2406f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.148+0000 7fbf2b1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.148+0000 7fbf2b1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf241013b0 0x7fbf24070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf2b1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24070c70 con 0x7fbf241013b0 2023-08-11T14:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf2b1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24070de0 con 0x7fbf24100a10 2023-08-11T14:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf2b1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24070fa0 con 0x7fbf240ff820 2023-08-11T14:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42736/0 (socket says 172.21.15.3:42736) 2023-08-11T14:07:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 -- 172.21.15.3:0/552756317 learned_addr learned my addr 172.21.15.3:0/552756317 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 -- 172.21.15.3:0/552756317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf240ff820 msgr2=0x7fbf2406f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf240ff820 0x7fbf2406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 -- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf241013b0 msgr2=0x7fbf24070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf241013b0 0x7fbf24070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.149+0000 7fbf23fff700 1 -- 172.21.15.3:0/552756317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009520 con 0x7fbf24100a10 2023-08-11T14:07:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.150+0000 7fbf23fff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000fbb0 tx=0x7fbf10005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.151+0000 7fbf21ffb700 1 -- 172.21.15.3:0/552756317 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1001c070 con 0x7fbf24100a10 2023-08-11T14:07:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.151+0000 7fbf21ffb700 1 -- 172.21.15.3:0/552756317 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1000bc50 con 0x7fbf24100a10 2023-08-11T14:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.151+0000 7fbf21ffb700 1 -- 172.21.15.3:0/552756317 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10020410 con 0x7fbf24100a10 2023-08-11T14:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.151+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf24071220 con 0x7fbf24100a10 2023-08-11T14:07:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.151+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24071ab0 con 0x7fbf24100a10 2023-08-11T14:07:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.162+0000 7fbf3eaf5b80 1 Processor -- start 2023-08-11T14:07:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.164+0000 7fbf21ffb700 1 -- 172.21.15.3:0/552756317 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbf10003680 con 0x7fbf24100a10 2023-08-11T14:07:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.164+0000 7fbf3eaf5b80 1 -- start start 2023-08-11T14:07:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4df40 0x5609bbf4e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4f790 0x5609bbf4fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbea75d0 con 0x5609bbf4f790 2023-08-11T14:07:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbea4ef0 con 0x5609bbf4df40 2023-08-11T14:07:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.165+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbea07e0 con 0x5609bbf4e8a0 2023-08-11T14:07:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.168+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.168+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34694/0 (socket says 172.21.15.3:34694) 2023-08-11T14:07:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.168+0000 7fbf28f9b700 1 -- 172.21.15.3:0/2934921812 learned_addr learned my addr 172.21.15.3:0/2934921812 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf2979c700 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4df40 0x5609bbf4e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf28f9b700 1 -- 172.21.15.3:0/2934921812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4df40 msgr2=0x5609bbf4e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4df40 0x5609bbf4e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf28f9b700 1 -- 172.21.15.3:0/2934921812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4f790 msgr2=0x5609bbf4fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4f790 0x5609bbf4fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.170+0000 7fbf28f9b700 1 -- 172.21.15.3:0/2934921812 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609bbd205d0 con 0x5609bbf4e8a0 2023-08-11T14:07:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.171+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800bb60 tx=0x7fbf1800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eba831733000100 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.171+0000 7fbf21ffb700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 0x7fbf140a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.171+0000 7fbf21ffb700 1 -- 172.21.15.3:0/552756317 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbf100cc580 con 0x7fbf24100a10 2023-08-11T14:07:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.172+0000 7fbf0effd700 1 -- 172.21.15.3:0/2934921812 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800d890 con 0x5609bbf4e8a0 2023-08-11T14:07:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.172+0000 7fbf0effd700 1 -- 172.21.15.3:0/2934921812 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf18010460 con 0x5609bbf4e8a0 2023-08-11T14:07:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.172+0000 7fbf0effd700 1 -- 172.21.15.3:0/2934921812 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18018600 con 0x5609bbf4e8a0 2023-08-11T14:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.173+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2934921812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 msgr2=0x5609bbf4f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.173+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800bb60 tx=0x7fbf1800be70 comp rx=0 tx=0).stop 2023-08-11T14:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.174+0000 7fbf2979c700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 0x7fbf140a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.175+0000 7fbf2979c700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 0x7fbf140a6a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009e40 tx=0x7fbf1c009430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2934921812 shutdown_connections 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4f790 0x5609bbf4fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4df40 0x5609bbf4e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2934921812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf4f1b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2934921812 >> 172.21.15.3:0/2934921812 conn(0x5609bbe7b8f0 msgr2=0x5609bbf4a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2934921812 shutdown_connections 2023-08-11T14:07:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.182+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2934921812 wait complete. 2023-08-11T14:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.183+0000 7fbf3eaf5b80 1 -- start start 2023-08-11T14:07:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.184+0000 7fbf3eaf5b80 1 Processor -- start 2023-08-11T14:07:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbe3ea60 con 0x5609bbf4df40 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbe3e760 con 0x5609bbf4f790 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbd228c0 con 0x5609bbf4e8a0 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.194+0000 7fbf3eaf5b80 10 client.0 _ll_register_callbacks cb 0x5609bbe41a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.197+0000 7fbf2979c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.197+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.197+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52874/0 (socket says 172.21.15.3:52874) 2023-08-11T14:07:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.197+0000 7fbf28f9b700 1 -- 172.21.15.3:0/2280267276 learned_addr learned my addr 172.21.15.3:0/2280267276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf2979c700 1 -- 172.21.15.3:0/2280267276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 msgr2=0x5609bbf58200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf2979c700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf58200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf2979c700 1 -- 172.21.15.3:0/2280267276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 msgr2=0x5609bbf57cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf2979c700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf2979c700 1 -- 172.21.15.3:0/2280267276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1800b7c0 con 0x5609bbf4f790 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.199+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.200+0000 7fbf2979c700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5f240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009fd0 tx=0x7fbf1c004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ca4d42f403f34b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.200+0000 7fbf03fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:07:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.200+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01c070 con 0x5609bbf4f790 2023-08-11T14:07:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.200+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbf1c017e30 con 0x5609bbf4f790 2023-08-11T14:07:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.201+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609bbf417e0 con 0x5609bbf4f790 2023-08-11T14:07:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.201+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf1c011070 con 0x5609bbf4f790 2023-08-11T14:07:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.201+0000 7fbf3eaf5b80 10 client.26093 fetch_fsmap learned FSMap version 31 2023-08-11T14:07:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.201+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5609bbcccd80 con 0x5609bbf4f790 2023-08-11T14:07:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.202+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609bbf417e0 con 0x5609bbf4f790 2023-08-11T14:07:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.202+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fbf1c02a070 con 0x5609bbf4f790 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 10 client.26093 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 20 client.26093 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbf63bb0 0x5609bbf65fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 4 client.26093 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609bbf1d480 con 0x5609bbf63bb0 2023-08-11T14:07:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.203+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbf63bb0 0x5609bbf65fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.204+0000 7fbf03fff700 10 client.26093 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.204+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbf63bb0 0x5609bbf65fd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7923b6984b04b3 server_cookie=b5442318c3aab75a in_seq=0 out_seq=0 2023-08-11T14:07:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.204+0000 7fbf03fff700 1 -- 172.21.15.3:0/2280267276 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5609bbf1d480 con 0x5609bbf63bb0 2023-08-11T14:07:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.204+0000 7fbf03fff700 10 client.26093 handle_command_reply: tid=0 2023-08-11T14:07:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 client.26093 shutdown 2023-08-11T14:07:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 msgr2=0x5609bbf5f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5f240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009fd0 tx=0x7fbf1c004c50 comp rx=0 tx=0).stop 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 shutdown_connections 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbf63bb0 0x5609bbf65fd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5f240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/2280267276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf58200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.205+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 >> 172.21.15.3:0/2280267276 conn(0x5609bbe7b8f0 msgr2=0x5609bbf49710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.206+0000 7fbf03fff700 0 client.26093 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.206+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 shutdown_connections 2023-08-11T14:07:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.206+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/2280267276 wait complete. 2023-08-11T14:07:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.206+0000 7fbf3eaf5b80 20 client.26093 trim_cache size 0 max 16384 2023-08-11T14:07:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.233+0000 7fbf3eaf5b80 1 Processor -- start 2023-08-11T14:07:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 -- start start 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4df40 0x5609bbf4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf4db40 con 0x5609bbf4e8a0 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf55a10 con 0x5609bbf4f790 2023-08-11T14:07:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.234+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf53300 con 0x5609bbf4df40 2023-08-11T14:07:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42766/0 (socket says 172.21.15.3:42766) 2023-08-11T14:07:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 -- 172.21.15.3:0/1038380302 learned_addr learned my addr 172.21.15.3:0/1038380302 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 -- 172.21.15.3:0/1038380302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4df40 msgr2=0x5609bbf4e320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:07:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4df40 0x5609bbf4e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 -- 172.21.15.3:0/1038380302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4e8a0 msgr2=0x5609bbf5f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.235+0000 7fbf2979c700 1 -- 172.21.15.3:0/1038380302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609bbcb5c00 con 0x5609bbf4f790 2023-08-11T14:07:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.236+0000 7fbf2979c700 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009640 tx=0x7fbf1c02b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98a1a352d587402 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.236+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/1038380302 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01c030 con 0x5609bbf4f790 2023-08-11T14:07:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.236+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/1038380302 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1c017db0 con 0x5609bbf4f790 2023-08-11T14:07:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.236+0000 7fbf0f7fe700 1 -- 172.21.15.3:0/1038380302 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c022c10 con 0x5609bbf4f790 2023-08-11T14:07:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.237+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1038380302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 msgr2=0x5609bbf52790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.237+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009640 tx=0x7fbf1c02b740 comp rx=0 tx=0).stop 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1038380302 shutdown_connections 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf52790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4df40 0x5609bbf4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1038380302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1038380302 >> 172.21.15.3:0/1038380302 conn(0x5609bbe7b8f0 msgr2=0x5609bbf56990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.239+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1038380302 shutdown_connections 2023-08-11T14:07:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.240+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1038380302 wait complete. 2023-08-11T14:07:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.240+0000 7fbf3eaf5b80 1 -- start start 2023-08-11T14:07:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 Processor -- start 2023-08-11T14:07:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf5e3b0 con 0x5609bbf4df40 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf5e520 con 0x5609bbf4f790 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.241+0000 7fbf3eaf5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609bbf5e690 con 0x5609bbf4e8a0 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf3eaf5b80 10 client.0 _ll_register_callbacks cb 0x7fbf1c00cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52890/0 (socket says 172.21.15.3:52890) 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 -- 172.21.15.3:0/1784165323 learned_addr learned my addr 172.21.15.3:0/1784165323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf2979c700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 -- 172.21.15.3:0/1784165323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 msgr2=0x5609bbf5ce60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5ce60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 -- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 msgr2=0x5609bbf5de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.242+0000 7fbf28f9b700 1 -- 172.21.15.3:0/1784165323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609bbcccd80 con 0x5609bbf4df40 2023-08-11T14:07:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.243+0000 7fbf28f9b700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800d5c0 tx=0x7fbf18010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e334d959bc0c97 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.243+0000 7fbf03fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1801f040 con 0x5609bbf4df40 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf18010460 con 0x5609bbf4df40 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbf18010750 con 0x5609bbf4df40 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf18010910 con 0x5609bbf4df40 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf3eaf5b80 10 client.16038 fetch_fsmap learned FSMap version 31 2023-08-11T14:07:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5609bbcf31f0 con 0x5609bbf4df40 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.244+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf18010460 con 0x5609bbf4df40 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fbf1800dbc0 con 0x5609bbf4df40 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 31 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 20 client.16038 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbfb2320 0x5609bbfb4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 4 client.16038 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:07:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.245+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609bbe997c0 con 0x5609bbfb2320 2023-08-11T14:07:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.246+0000 7fbf23fff700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbfb2320 0x5609bbfb4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.247+0000 7fbf23fff700 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbfb2320 0x5609bbfb4740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb87d90eb95dd00 server_cookie=710b76198ac0ec12 in_seq=0 out_seq=0 2023-08-11T14:07:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.247+0000 7fbf03fff700 10 client.16038 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.277+0000 7fbf03fff700 1 -- 172.21.15.3:0/1784165323 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609bbe997c0 con 0x5609bbfb2320 2023-08-11T14:07:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.277+0000 7fbf03fff700 10 client.16038 handle_command_reply: tid=0 2023-08-11T14:07:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.278+0000 7fbf3eaf5b80 1 client.16038 shutdown 2023-08-11T14:07:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 msgr2=0x5609bbf5c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800d5c0 tx=0x7fbf18010e40 comp rx=0 tx=0).stop 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 shutdown_connections 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609bbfb2320 0x5609bbfb4740 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609bbf4f790 0x5609bbf5de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609bbf4df40 0x5609bbf5c920 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 --2- 172.21.15.3:0/1784165323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609bbf4e8a0 0x5609bbf5ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 >> 172.21.15.3:0/1784165323 conn(0x5609bbe7b8f0 msgr2=0x5609bbf52790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf03fff700 0 client.16038 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 shutdown_connections 2023-08-11T14:07:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 1 -- 172.21.15.3:0/1784165323 wait complete. 2023-08-11T14:07:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.279+0000 7fbf3eaf5b80 20 client.16038 trim_cache size 0 max 16384 2023-08-11T14:07:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:07:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:07:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:07:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 msgr2=0x7fbf140a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 0x7fbf140a6a20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009e40 tx=0x7fbf1c009430 comp rx=0 tx=0).stop 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 msgr2=0x7fbf2406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000fbb0 tx=0x7fbf10005320 comp rx=0 tx=0).stop 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 shutdown_connections 2023-08-11T14:07:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf241013b0 0x7fbf24070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf240ff820 0x7fbf2406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf140a4560 0x7fbf140a6a20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 --2- 172.21.15.3:0/552756317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf24100a10 0x7fbf2406f740 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 >> 172.21.15.3:0/552756317 conn(0x7fbf240fad70 msgr2=0x7fbf240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 shutdown_connections 2023-08-11T14:07:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:05.280+0000 7fbf2b1ff700 1 -- 172.21.15.3:0/552756317 wait complete. 2023-08-11T14:07:05.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:05 smithi003 ceph-mon[97195]: pgmap v442: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 174 MiB/s wr, 116 op/s 2023-08-11T14:07:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:05 smithi187 ceph-mon[130973]: pgmap v442: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 174 MiB/s wr, 116 op/s 2023-08-11T14:07:05.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:05 smithi103 ceph-mon[130434]: pgmap v442: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 174 MiB/s wr, 116 op/s 2023-08-11T14:07:07.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:07 smithi003 ceph-mon[97195]: pgmap v443: 129 pgs: 129 active+clean; 8.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 187 MiB/s wr, 144 op/s 2023-08-11T14:07:07.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:07 smithi187 ceph-mon[130973]: pgmap v443: 129 pgs: 129 active+clean; 8.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 187 MiB/s wr, 144 op/s 2023-08-11T14:07:07.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:07 smithi103 ceph-mon[130434]: pgmap v443: 129 pgs: 129 active+clean; 8.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 187 MiB/s wr, 144 op/s 2023-08-11T14:07:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 10.0 s, 2885.0 tps, lat 34.887 ms stddev 25.035 2023-08-11T14:07:08.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:08 smithi003 ceph-mon[97195]: pgmap v444: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 188 MiB/s wr, 216 op/s 2023-08-11T14:07:08.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:08 smithi187 ceph-mon[130973]: pgmap v444: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 188 MiB/s wr, 216 op/s 2023-08-11T14:07:08.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:08 smithi103 ceph-mon[130434]: pgmap v444: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 188 MiB/s wr, 216 op/s 2023-08-11T14:07:10.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:10 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:10 smithi003 ceph-mon[97195]: pgmap v445: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 157 MiB/s wr, 348 op/s 2023-08-11T14:07:10.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:10 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:10.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:10 smithi187 ceph-mon[130973]: pgmap v445: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 157 MiB/s wr, 348 op/s 2023-08-11T14:07:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:10 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:10 smithi103 ceph-mon[130434]: pgmap v445: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 MiB/s rd, 157 MiB/s wr, 348 op/s 2023-08-11T14:07:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 15.0 s, 1932.8 tps, lat 51.805 ms stddev 74.862 2023-08-11T14:07:13.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:13 smithi003 ceph-mon[97195]: pgmap v446: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 139 MiB/s wr, 575 op/s 2023-08-11T14:07:13.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:13 smithi187 ceph-mon[130973]: pgmap v446: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 139 MiB/s wr, 575 op/s 2023-08-11T14:07:13.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:13 smithi103 ceph-mon[130434]: pgmap v446: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 MiB/s rd, 139 MiB/s wr, 575 op/s 2023-08-11T14:07:14.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:14 smithi187 ceph-mon[130973]: pgmap v447: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 91 MiB/s wr, 671 op/s 2023-08-11T14:07:14.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:14 smithi103 ceph-mon[130434]: pgmap v447: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 91 MiB/s wr, 671 op/s 2023-08-11T14:07:15.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:14 smithi003 ceph-mon[97195]: pgmap v447: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 91 MiB/s wr, 671 op/s 2023-08-11T14:07:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:17 smithi103 ceph-mon[130434]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 67 MiB/s wr, 1.18k op/s 2023-08-11T14:07:17.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:17 smithi003 ceph-mon[97195]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 67 MiB/s wr, 1.18k op/s 2023-08-11T14:07:17.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:17 smithi187 ceph-mon[130973]: pgmap v448: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 67 MiB/s wr, 1.18k op/s 2023-08-11T14:07:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 20.0 s, 806.8 tps, lat 119.327 ms stddev 291.127 2023-08-11T14:07:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.533+0000 7fb82ffff700 1 -- 172.21.15.3:0/4276126693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8281044d0 msgr2=0x7fb828104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.533+0000 7fb82ffff700 1 --2- 172.21.15.3:0/4276126693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8281044d0 0x7fb828104950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0a0 tx=0x7fb82000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 -- 172.21.15.3:0/4276126693 shutdown_connections 2023-08-11T14:07:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 --2- 172.21.15.3:0/4276126693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8281044d0 0x7fb828104950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 --2- 172.21.15.3:0/4276126693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8280ff190 0x7fb8281015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 --2- 172.21.15.3:0/4276126693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb828101b00 0x7fb828103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 -- 172.21.15.3:0/4276126693 >> 172.21.15.3:0/4276126693 conn(0x7fb8280fad70 msgr2=0x7fb8280fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 -- 172.21.15.3:0/4276126693 shutdown_connections 2023-08-11T14:07:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.535+0000 7fb82ffff700 1 -- 172.21.15.3:0/4276126693 wait complete. 2023-08-11T14:07:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 Processor -- start 2023-08-11T14:07:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 -- start start 2023-08-11T14:07:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb828101b00 0x7fb82819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8281044d0 0x7fb82819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.537+0000 7fb82ffff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819e850 con 0x7fb8281044d0 2023-08-11T14:07:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.538+0000 7fb82ffff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819e9c0 con 0x7fb828101b00 2023-08-11T14:07:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.538+0000 7fb82ffff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82819eb80 con 0x7fb8280ff190 2023-08-11T14:07:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.539+0000 7fb82dd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.540+0000 7fb82dd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:43576/0 (socket says 172.21.15.3:43576) 2023-08-11T14:07:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.540+0000 7fb82dd9b700 1 -- 172.21.15.3:0/739230380 learned_addr learned my addr 172.21.15.3:0/739230380 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.541+0000 7fb82dd9b700 1 -- 172.21.15.3:0/739230380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb828101b00 msgr2=0x7fb82819d3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.541+0000 7fb82dd9b700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb828101b00 0x7fb82819d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.541+0000 7fb82dd9b700 1 -- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8281044d0 msgr2=0x7fb82819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.541+0000 7fb82dd9b700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8281044d0 0x7fb82819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.541+0000 7fb82dd9b700 1 -- 172.21.15.3:0/739230380 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009d00 con 0x7fb8280ff190 2023-08-11T14:07:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.546+0000 7fb82dd9b700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d950 tx=0x7fb82400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.552+0000 7fb81affd700 1 -- 172.21.15.3:0/739230380 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824009b40 con 0x7fb8280ff190 2023-08-11T14:07:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.552+0000 7fb81affd700 1 -- 172.21.15.3:0/739230380 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb824009ca0 con 0x7fb8280ff190 2023-08-11T14:07:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.552+0000 7fb81affd700 1 -- 172.21.15.3:0/739230380 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400f620 con 0x7fb8280ff190 2023-08-11T14:07:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.553+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82819ee60 con 0x7fb8280ff190 2023-08-11T14:07:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.554+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82806a520 con 0x7fb8280ff190 2023-08-11T14:07:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.561+0000 7fb84382bb80 1 Processor -- start 2023-08-11T14:07:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.561+0000 7fb84382bb80 1 -- start start 2023-08-11T14:07:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.561+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78bac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.561+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.561+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78badc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb84382bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78b055d0 con 0x564f78babf40 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb84382bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78b02ef0 con 0x564f78bad790 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb84382bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78afe7e0 con 0x564f78bac8a0 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb82d59a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78badc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78bac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78bac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42738/0 (socket says 172.21.15.3:42738) 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.562+0000 7fb82e59c700 1 -- 172.21.15.3:0/3575266943 learned_addr learned my addr 172.21.15.3:0/3575266943 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 -- 172.21.15.3:0/3575266943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 msgr2=0x564f78bad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 -- 172.21.15.3:0/3575266943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 msgr2=0x564f78bac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78bac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 -- 172.21.15.3:0/3575266943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f7897e5d0 con 0x564f78bad790 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.564+0000 7fb82d59a700 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78badc10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b60 tx=0x7fb81c009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0e820cd2bfa4cf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.565+0000 7fb813fff700 1 -- 172.21.15.3:0/3575266943 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c004230 con 0x564f78bad790 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.565+0000 7fb813fff700 1 -- 172.21.15.3:0/3575266943 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81c00edc0 con 0x564f78bad790 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.565+0000 7fb813fff700 1 -- 172.21.15.3:0/3575266943 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c005020 con 0x564f78bad790 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 -- 172.21.15.3:0/3575266943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 msgr2=0x564f78badc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78badc10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b60 tx=0x7fb81c009e70 comp rx=0 tx=0).stop 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 -- 172.21.15.3:0/3575266943 shutdown_connections 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78badc10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78bac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 --2- 172.21.15.3:0/3575266943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.566+0000 7fb84382bb80 1 -- 172.21.15.3:0/3575266943 >> 172.21.15.3:0/3575266943 conn(0x564f78ad98f0 msgr2=0x564f78ba8920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.567+0000 7fb84382bb80 1 -- 172.21.15.3:0/3575266943 shutdown_connections 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.567+0000 7fb84382bb80 1 -- 172.21.15.3:0/3575266943 wait complete. 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.568+0000 7fb84382bb80 1 -- start start 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.568+0000 7fb84382bb80 1 Processor -- start 2023-08-11T14:07:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 0x564f78bf9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bac8a0 0x564f78bfa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78c01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78a9ca60 con 0x564f78bac8a0 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78a9c760 con 0x564f78bad790 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f789808c0 con 0x564f78babf40 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.569+0000 7fb84382bb80 10 client.0 _ll_register_callbacks cb 0x564f78a9fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.571+0000 7fb82d59a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78c01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.571+0000 7fb82d59a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78c01270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37596/0 (socket says 172.21.15.3:37596) 2023-08-11T14:07:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.571+0000 7fb82d59a700 1 -- 172.21.15.3:0/760045833 learned_addr learned my addr 172.21.15.3:0/760045833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.571+0000 7fb82e59c700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 0x564f78bf9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.572+0000 7fb82e59c700 1 -- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 msgr2=0x564f78c01270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.572+0000 7fb82e59c700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78c01270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.572+0000 7fb82e59c700 1 -- 172.21.15.3:0/760045833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bac8a0 msgr2=0x564f78bfa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.572+0000 7fb82e59c700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bac8a0 0x564f78bfa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.572+0000 7fb82e59c700 1 -- 172.21.15.3:0/760045833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c009810 con 0x564f78babf40 2023-08-11T14:07:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.573+0000 7fb82e59c700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 0x564f78bf9cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0a0 tx=0x7fb82000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed161a80d7f7719e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.576+0000 7fb8117fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:07:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.576+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e070 con 0x564f78babf40 2023-08-11T14:07:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.576+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb820003c00 con 0x564f78babf40 2023-08-11T14:07:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.576+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f78b9f7e0 con 0x564f78babf40 2023-08-11T14:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.578+0000 7fb81affd700 1 -- 172.21.15.3:0/739230380 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb824010460 con 0x7fb8280ff190 2023-08-11T14:07:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.580+0000 7fb81affd700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 0x7fb8140a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.580+0000 7fb81affd700 1 -- 172.21.15.3:0/739230380 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb8240cd2c0 con 0x7fb8280ff190 2023-08-11T14:07:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.584+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb820003dc0 con 0x564f78babf40 2023-08-11T14:07:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.588+0000 7fb84382bb80 10 client.26089 fetch_fsmap learned FSMap version 32 2023-08-11T14:07:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.588+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564f7892ad80 con 0x564f78babf40 2023-08-11T14:07:18.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.594+0000 7fb82dd9b700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 0x7fb8140a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.602+0000 7fb82dd9b700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 0x7fb8140a24a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb824097580 tx=0x7fb824000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f78b9f7e0 con 0x564f78babf40 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb82000b0d0 con 0x564f78babf40 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 10 client.26089 fetch_fsmap finished waiting for FSMap version 32 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 20 client.26089 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c05c20 0x564f78c08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 4 client.26089 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:07:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.607+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f78b7b480 con 0x564f78c05c20 2023-08-11T14:07:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.609+0000 7fb82d59a700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c05c20 0x564f78c08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.610+0000 7fb82d59a700 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c05c20 0x564f78c08040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb7bca670a786e1 server_cookie=4980bd59201c98b8 in_seq=0 out_seq=0 2023-08-11T14:07:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.610+0000 7fb8117fa700 10 client.26089 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.617+0000 7fb8117fa700 1 -- 172.21.15.3:0/760045833 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564f78b7b480 con 0x564f78c05c20 2023-08-11T14:07:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.617+0000 7fb8117fa700 10 client.26089 handle_command_reply: tid=0 2023-08-11T14:07:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.618+0000 7fb84382bb80 1 client.26089 shutdown 2023-08-11T14:07:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.620+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 msgr2=0x564f78bf9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.620+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 0x564f78bf9cf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b0a0 tx=0x7fb82000bd90 comp rx=0 tx=0).stop 2023-08-11T14:07:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 shutdown_connections 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c05c20 0x564f78c08040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bad790 0x564f78c01270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78babf40 0x564f78bf9cf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 --2- 172.21.15.3:0/760045833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bac8a0 0x564f78bfa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 >> 172.21.15.3:0/760045833 conn(0x564f78ad98f0 msgr2=0x564f78ba7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.621+0000 7fb8117fa700 0 client.26089 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.627+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 shutdown_connections 2023-08-11T14:07:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.630+0000 7fb84382bb80 1 -- 172.21.15.3:0/760045833 wait complete. 2023-08-11T14:07:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.630+0000 7fb84382bb80 20 client.26089 trim_cache size 0 max 16384 2023-08-11T14:07:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 Processor -- start 2023-08-11T14:07:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 -- start start 2023-08-11T14:07:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78ba82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bad790 0x564f78ba8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78b9f7b0 con 0x564f78babf40 2023-08-11T14:07:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.669+0000 7fb84382bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78c0ce00 con 0x564f78bac8a0 2023-08-11T14:07:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.670+0000 7fb84382bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78bacc80 con 0x564f78bad790 2023-08-11T14:07:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.685+0000 7fb82d59a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.685+0000 7fb82d59a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37600/0 (socket says 172.21.15.3:37600) 2023-08-11T14:07:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.685+0000 7fb82d59a700 1 -- 172.21.15.3:0/4280633896 learned_addr learned my addr 172.21.15.3:0/4280633896 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.685+0000 7fb82e59c700 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bad790 0x564f78ba8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.685+0000 7fb82e59c700 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78ba82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 -- 172.21.15.3:0/4280633896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bad790 msgr2=0x564f78ba8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bad790 0x564f78ba8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 -- 172.21.15.3:0/4280633896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 msgr2=0x564f78ba82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78ba82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 -- 172.21.15.3:0/4280633896 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f78ba3a10 con 0x564f78bac8a0 2023-08-11T14:07:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.686+0000 7fb82d59a700 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b00 tx=0x7fb81c009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd01c1d7c8c8f71 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.692+0000 7fb818ff9700 1 -- 172.21.15.3:0/4280633896 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c02d070 con 0x564f78bac8a0 2023-08-11T14:07:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.692+0000 7fb818ff9700 1 -- 172.21.15.3:0/4280633896 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81c01d040 con 0x564f78bac8a0 2023-08-11T14:07:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.692+0000 7fb818ff9700 1 -- 172.21.15.3:0/4280633896 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c028b20 con 0x564f78bac8a0 2023-08-11T14:07:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.693+0000 7fb84382bb80 1 -- 172.21.15.3:0/4280633896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 msgr2=0x564f78c01230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.693+0000 7fb84382bb80 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b00 tx=0x7fb81c009440 comp rx=0 tx=0).stop 2023-08-11T14:07:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.697+0000 7fb84382bb80 1 -- 172.21.15.3:0/4280633896 shutdown_connections 2023-08-11T14:07:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.697+0000 7fb84382bb80 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bad790 0x564f78ba8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.697+0000 7fb84382bb80 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78babf40 0x564f78ba82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.697+0000 7fb84382bb80 1 --2- 172.21.15.3:0/4280633896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78bac8a0 0x564f78c01230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.697+0000 7fb84382bb80 1 -- 172.21.15.3:0/4280633896 >> 172.21.15.3:0/4280633896 conn(0x564f78ad98f0 msgr2=0x564f78bb0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.700+0000 7fb84382bb80 1 -- 172.21.15.3:0/4280633896 shutdown_connections 2023-08-11T14:07:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.706+0000 7fb84382bb80 1 -- 172.21.15.3:0/4280633896 wait complete. 2023-08-11T14:07:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.707+0000 7fb84382bb80 1 -- start start 2023-08-11T14:07:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.707+0000 7fb84382bb80 1 Processor -- start 2023-08-11T14:07:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bfd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bad790 0x564f78c000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78c005f0 con 0x564f78bad790 2023-08-11T14:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78c00760 con 0x564f78babf40 2023-08-11T14:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f78c00920 con 0x564f78bac8a0 2023-08-11T14:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb84382bb80 10 client.0 _ll_register_callbacks cb 0x7fb820009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb82e59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:37616/0 (socket says 172.21.15.3:37616) 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb82e59c700 1 -- 172.21.15.3:0/1829614242 learned_addr learned my addr 172.21.15.3:0/1829614242 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.708+0000 7fb82e59c700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bad790 0x564f78c000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 -- 172.21.15.3:0/1829614242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 msgr2=0x564f78bfd090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bfd090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 -- 172.21.15.3:0/1829614242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bad790 msgr2=0x564f78c000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bad790 0x564f78c000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 -- 172.21.15.3:0/1829614242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f7897e5d0 con 0x564f78babf40 2023-08-11T14:07:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.709+0000 7fb82e59c700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b780 tx=0x7fb8200092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5daec743847fd30 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.710+0000 7fb8117fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:07:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.710+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e020 con 0x564f78babf40 2023-08-11T14:07:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.710+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb820003a00 con 0x564f78babf40 2023-08-11T14:07:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.711+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81c00edc0 con 0x564f78babf40 2023-08-11T14:07:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.711+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb820003bc0 con 0x564f78babf40 2023-08-11T14:07:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.718+0000 7fb84382bb80 10 client.26114 fetch_fsmap learned FSMap version 32 2023-08-11T14:07:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.718+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564f78913c00 con 0x564f78babf40 2023-08-11T14:07:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.729+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81c00edc0 con 0x564f78babf40 2023-08-11T14:07:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.729+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fb820024c20 con 0x564f78babf40 2023-08-11T14:07:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.731+0000 7fb84382bb80 10 client.26114 fetch_fsmap finished waiting for FSMap version 32 2023-08-11T14:07:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.731+0000 7fb84382bb80 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.731+0000 7fb84382bb80 20 client.26114 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.732+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c6af90 0x564f78c6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.732+0000 7fb84382bb80 4 client.26114 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:07:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.732+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f78af77c0 con 0x564f78c6af90 2023-08-11T14:07:18.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.732+0000 7fb82d59a700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c6af90 0x564f78c6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.733+0000 7fb82d59a700 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c6af90 0x564f78c6d380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9d3e602662ed9e server_cookie=80f0ad73ee568db0 in_seq=0 out_seq=0 2023-08-11T14:07:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.733+0000 7fb8117fa700 10 client.26114 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.757+0000 7fb8117fa700 1 -- 172.21.15.3:0/1829614242 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f78af77c0 con 0x564f78c6af90 2023-08-11T14:07:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.757+0000 7fb8117fa700 10 client.26114 handle_command_reply: tid=0 2023-08-11T14:07:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.760+0000 7fb84382bb80 1 client.26114 shutdown 2023-08-11T14:07:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.761+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 msgr2=0x564f78bfcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.761+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb82000b780 tx=0x7fb8200092e0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.762+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 shutdown_connections 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.762+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564f78bad790 0x564f78c000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.762+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564f78c6af90 0x564f78c6d380 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.763+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f78babf40 0x564f78bfcb50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.763+0000 7fb84382bb80 1 --2- 172.21.15.3:0/1829614242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f78bac8a0 0x564f78bfd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.763+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 >> 172.21.15.3:0/1829614242 conn(0x564f78ad98f0 msgr2=0x564f78ba8210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.763+0000 7fb8117fa700 0 client.26114 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:18.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.763+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 shutdown_connections 2023-08-11T14:07:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.766+0000 7fb84382bb80 1 -- 172.21.15.3:0/1829614242 wait complete. 2023-08-11T14:07:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.780+0000 7fb84382bb80 20 client.26114 trim_cache size 0 max 16384 2023-08-11T14:07:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:07:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:07:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:07:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:07:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 msgr2=0x7fb8140a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 0x7fb8140a24a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb824097580 tx=0x7fb824000bc0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 msgr2=0x7fb82819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d950 tx=0x7fb82400dd10 comp rx=0 tx=0).stop 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 shutdown_connections 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8281044d0 0x7fb82819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8280ff190 0x7fb82819ce60 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d950 tx=0x7fb82400dd10 comp rx=0 tx=0).stop 2023-08-11T14:07:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb81409ffe0 0x7fb8140a24a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 --2- 172.21.15.3:0/739230380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb828101b00 0x7fb82819d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 >> 172.21.15.3:0/739230380 conn(0x7fb8280fad70 msgr2=0x7fb8280ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 shutdown_connections 2023-08-11T14:07:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:18.782+0000 7fb82ffff700 1 -- 172.21.15.3:0/739230380 wait complete. 2023-08-11T14:07:19.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:19 smithi187 ceph-mon[130973]: pgmap v449: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2023-08-11T14:07:19.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:19 smithi003 ceph-mon[97195]: pgmap v449: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2023-08-11T14:07:19.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:19 smithi103 ceph-mon[130434]: pgmap v449: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 56 MiB/s wr, 2.17k op/s 2023-08-11T14:07:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:20 smithi003 ceph-mon[97195]: pgmap v450: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 3.06k op/s 2023-08-11T14:07:20.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:20 smithi187 ceph-mon[130973]: pgmap v450: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 3.06k op/s 2023-08-11T14:07:20.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:20 smithi103 ceph-mon[130434]: pgmap v450: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 3.06k op/s 2023-08-11T14:07:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 25.0 s, 1692.4 tps, lat 60.883 ms stddev 89.957 2023-08-11T14:07:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:23 smithi003 ceph-mon[97195]: pgmap v451: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 51 MiB/s wr, 3.58k op/s 2023-08-11T14:07:23.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:23 smithi187 ceph-mon[130973]: pgmap v451: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 51 MiB/s wr, 3.58k op/s 2023-08-11T14:07:23.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:23 smithi103 ceph-mon[130434]: pgmap v451: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 51 MiB/s wr, 3.58k op/s 2023-08-11T14:07:24.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:24 smithi187 ceph-mon[130973]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 48 MiB/s wr, 3.53k op/s 2023-08-11T14:07:24.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:24 smithi003 ceph-mon[97195]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 48 MiB/s wr, 3.53k op/s 2023-08-11T14:07:24.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:24 smithi103 ceph-mon[130434]: pgmap v452: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 48 MiB/s wr, 3.53k op/s 2023-08-11T14:07:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:27 smithi103 ceph-mon[130434]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 52 MiB/s wr, 3.52k op/s 2023-08-11T14:07:27.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:27 smithi003 ceph-mon[97195]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 52 MiB/s wr, 3.52k op/s 2023-08-11T14:07:27.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:27 smithi187 ceph-mon[130973]: pgmap v453: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 52 MiB/s wr, 3.52k op/s 2023-08-11T14:07:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 30.0 s, 1774.8 tps, lat 56.099 ms stddev 119.485 2023-08-11T14:07:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:28 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:28.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:28 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:28.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:28 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:07:29.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:29 smithi103 ceph-mon[130434]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 3.16k op/s 2023-08-11T14:07:29.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:29 smithi003 ceph-mon[97195]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 3.16k op/s 2023-08-11T14:07:29.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:29 smithi187 ceph-mon[130973]: pgmap v454: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 51 MiB/s wr, 3.16k op/s 2023-08-11T14:07:31.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:31 smithi003 ceph-mon[97195]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 42 MiB/s wr, 2.34k op/s 2023-08-11T14:07:31.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:31 smithi187 ceph-mon[130973]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 42 MiB/s wr, 2.34k op/s 2023-08-11T14:07:31.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:31 smithi103 ceph-mon[130434]: pgmap v455: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 42 MiB/s wr, 2.34k op/s 2023-08-11T14:07:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.120+0000 7f17d05ff700 1 -- 172.21.15.3:0/693284472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810cc60 msgr2=0x7f17c810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.120+0000 7f17d05ff700 1 --2- 172.21.15.3:0/693284472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810cc60 0x7f17c810d0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17c40098f0 tx=0x7f17c4009c00 comp rx=0 tx=0).stop 2023-08-11T14:07:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.129+0000 7f17d05ff700 1 -- 172.21.15.3:0/693284472 shutdown_connections 2023-08-11T14:07:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.129+0000 7f17d05ff700 1 --2- 172.21.15.3:0/693284472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810cc60 0x7f17c810d0e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.129+0000 7f17d05ff700 1 --2- 172.21.15.3:0/693284472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17c8107920 0x7f17c8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.129+0000 7f17d05ff700 1 --2- 172.21.15.3:0/693284472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810a290 0x7f17c810c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.129+0000 7f17d05ff700 1 -- 172.21.15.3:0/693284472 >> 172.21.15.3:0/693284472 conn(0x7f17c806d750 msgr2=0x7f17c806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.140+0000 7f17d05ff700 1 -- 172.21.15.3:0/693284472 shutdown_connections 2023-08-11T14:07:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.147+0000 7f17d05ff700 1 -- 172.21.15.3:0/693284472 wait complete. 2023-08-11T14:07:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.148+0000 7f17d05ff700 1 Processor -- start 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.148+0000 7f17d05ff700 1 -- start start 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17c8107920 0x7f17c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810cc60 0x7f17c819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c819e7c0 con 0x7f17c810a290 2023-08-11T14:07:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c819e930 con 0x7f17c8107920 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.149+0000 7f17d05ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c819eaf0 con 0x7f17c810cc60 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.151+0000 7f17ceb9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810cc60 0x7f17c819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35506/0 (socket says 172.21.15.3:35506) 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 -- 172.21.15.3:0/3914132366 learned_addr learned my addr 172.21.15.3:0/3914132366 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 -- 172.21.15.3:0/3914132366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810cc60 msgr2=0x7f17c819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810cc60 0x7f17c819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 -- 172.21.15.3:0/3914132366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17c8107920 msgr2=0x7f17c819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.152+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17c8107920 0x7f17c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.153+0000 7f17cdb9a700 1 -- 172.21.15.3:0/3914132366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c4009520 con 0x7f17c810a290 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.153+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b840 tx=0x7f17c000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.154+0000 7f17bf7fe700 1 -- 172.21.15.3:0/3914132366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000d890 con 0x7f17c810a290 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.154+0000 7f17bf7fe700 1 -- 172.21.15.3:0/3914132366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c000e430 con 0x7f17c810a290 2023-08-11T14:07:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.154+0000 7f17bf7fe700 1 -- 172.21.15.3:0/3914132366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0017660 con 0x7f17c810a290 2023-08-11T14:07:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.154+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c819edd0 con 0x7f17c810a290 2023-08-11T14:07:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.154+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81c7570 con 0x7f17c810a290 2023-08-11T14:07:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.161+0000 7f17e3f60b80 1 Processor -- start 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.161+0000 7f17e3f60b80 1 -- start start 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x55727208d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727208dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271fe55d0 con 0x55727208c8a0 2023-08-11T14:07:32.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271fe2ef0 con 0x55727208d790 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.162+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271fde7e0 con 0x55727208bf40 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.163+0000 7f17ceb9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.163+0000 7f17ceb9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46970/0 (socket says 172.21.15.3:46970) 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.163+0000 7f17ceb9c700 1 -- 172.21.15.3:0/4009267497 learned_addr learned my addr 172.21.15.3:0/4009267497 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.163+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727208dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.166+0000 7f17ce39b700 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x55727208d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.167+0000 7f17ceb9c700 1 -- 172.21.15.3:0/4009267497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 msgr2=0x55727208dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.167+0000 7f17ceb9c700 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727208dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.167+0000 7f17ceb9c700 1 -- 172.21.15.3:0/4009267497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 msgr2=0x55727208d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.167+0000 7f17ceb9c700 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x55727208d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.167+0000 7f17ceb9c700 1 -- 172.21.15.3:0/4009267497 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557271e5e5d0 con 0x55727208bf40 2023-08-11T14:07:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.169+0000 7f17ceb9c700 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009810 tx=0x7f17c4010d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cfe7eed5c84fb04 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.170+0000 7f17bf7fe700 1 -- 172.21.15.3:0/3914132366 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f17c000dc70 con 0x7f17c810a290 2023-08-11T14:07:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.172+0000 7f17bf7fe700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 0x7f17b40a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.172+0000 7f17bf7fe700 1 -- 172.21.15.3:0/3914132366 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f17c00d2070 con 0x7f17c810a290 2023-08-11T14:07:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.175+0000 7f17ce39b700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 0x7f17b40a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.178+0000 7f17ce39b700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 0x7f17b40a2590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005e50 tx=0x7f17b800a3d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.190+0000 7f17bcbfe700 1 -- 172.21.15.3:0/4009267497 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c401d070 con 0x55727208bf40 2023-08-11T14:07:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.190+0000 7f17bcbfe700 1 -- 172.21.15.3:0/4009267497 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c400f920 con 0x55727208bf40 2023-08-11T14:07:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.190+0000 7f17bcbfe700 1 -- 172.21.15.3:0/4009267497 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40052a0 con 0x55727208bf40 2023-08-11T14:07:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.191+0000 7f17e3f60b80 1 -- 172.21.15.3:0/4009267497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 msgr2=0x55727208c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.191+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009810 tx=0x7f17c4010d50 comp rx=0 tx=0).stop 2023-08-11T14:07:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.198+0000 7f17e3f60b80 1 -- 172.21.15.3:0/4009267497 shutdown_connections 2023-08-11T14:07:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.198+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727208dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.198+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x55727208c360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.198+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/4009267497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x55727208d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.198+0000 7f17e3f60b80 1 -- 172.21.15.3:0/4009267497 >> 172.21.15.3:0/4009267497 conn(0x557271fb98f0 msgr2=0x557272088920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.205+0000 7f17e3f60b80 1 -- 172.21.15.3:0/4009267497 shutdown_connections 2023-08-11T14:07:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.209+0000 7f17e3f60b80 1 -- 172.21.15.3:0/4009267497 wait complete. 2023-08-11T14:07:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.210+0000 7f17e3f60b80 1 -- start start 2023-08-11T14:07:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.210+0000 7f17e3f60b80 1 Processor -- start 2023-08-11T14:07:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x557272095c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x557272096140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727209d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271f7ca60 con 0x55727208c8a0 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271f7c760 con 0x55727208d790 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557271e608c0 con 0x55727208bf40 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.211+0000 7f17e3f60b80 10 client.0 _ll_register_callbacks cb 0x557271f7fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.212+0000 7f17ceb9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727209d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.215+0000 7f17ceb9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727209d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40272/0 (socket says 172.21.15.3:40272) 2023-08-11T14:07:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.215+0000 7f17ceb9c700 1 -- 172.21.15.3:0/486381488 learned_addr learned my addr 172.21.15.3:0/486381488 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.226+0000 7f17ce39b700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x557272095c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.228+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x557272096140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.233+0000 7f17ce39b700 1 -- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 msgr2=0x55727209d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.233+0000 7f17ce39b700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727209d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.233+0000 7f17ce39b700 1 -- 172.21.15.3:0/486381488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 msgr2=0x557272096140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.233+0000 7f17ce39b700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x557272096140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.233+0000 7f17ce39b700 1 -- 172.21.15.3:0/486381488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c810dc60 con 0x55727208bf40 2023-08-11T14:07:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.241+0000 7f17ce39b700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x557272095c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f17c819db40 tx=0x7f17b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28b1e6f4cc9c8d37 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.243+0000 7f17b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:07:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.243+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800aea0 con 0x55727208bf40 2023-08-11T14:07:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.243+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f17b8010e40 con 0x55727208bf40 2023-08-11T14:07:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.244+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55727207f7e0 con 0x55727208bf40 2023-08-11T14:07:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.252+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b801f800 con 0x55727208bf40 2023-08-11T14:07:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.253+0000 7f17e3f60b80 10 client.26098 fetch_fsmap learned FSMap version 33 2023-08-11T14:07:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.253+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557271e0ad80 con 0x55727208bf40 2023-08-11T14:07:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.264+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55727207f7e0 con 0x55727208bf40 2023-08-11T14:07:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.264+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17b8028b90 con 0x55727208bf40 2023-08-11T14:07:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:07:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 20 client.26098 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720a1a60 0x5572720a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 4 client.26098 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:07:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.265+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55727205b480 con 0x5572720a1a60 2023-08-11T14:07:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.272+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720a1a60 0x5572720a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.297+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720a1a60 0x5572720a3e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5231fdd55ace78 server_cookie=c20d86288392d8bf in_seq=0 out_seq=0 2023-08-11T14:07:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.297+0000 7f17b1ffb700 10 client.26098 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.309+0000 7f17b1ffb700 1 -- 172.21.15.3:0/486381488 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55727205b480 con 0x5572720a1a60 2023-08-11T14:07:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.309+0000 7f17b1ffb700 10 client.26098 handle_command_reply: tid=0 2023-08-11T14:07:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.310+0000 7f17e3f60b80 1 client.26098 shutdown 2023-08-11T14:07:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.319+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 msgr2=0x557272095c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.319+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x557272095c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f17c819db40 tx=0x7f17b8004750 comp rx=0 tx=0).stop 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 shutdown_connections 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720a1a60 0x5572720a3e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208d790 0x55727209d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208bf40 0x557272095c00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/486381488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208c8a0 0x557272096140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 >> 172.21.15.3:0/486381488 conn(0x557271fb98f0 msgr2=0x557272087710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17b1ffb700 0 client.26098 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 shutdown_connections 2023-08-11T14:07:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.320+0000 7f17e3f60b80 1 -- 172.21.15.3:0/486381488 wait complete. 2023-08-11T14:07:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.322+0000 7f17e3f60b80 20 client.26098 trim_cache size 0 max 16384 2023-08-11T14:07:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 Processor -- start 2023-08-11T14:07:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 -- start start 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208c8a0 0x55727209d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208d790 0x557272088860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557272081ea0 con 0x55727208d790 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572720a8c80 con 0x55727208bf40 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.358+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557272087d50 con 0x55727208c8a0 2023-08-11T14:07:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.359+0000 7f17ce39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40282/0 (socket says 172.21.15.3:40282) 2023-08-11T14:07:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 -- 172.21.15.3:0/2420365480 learned_addr learned my addr 172.21.15.3:0/2420365480 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 -- 172.21.15.3:0/2420365480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208c8a0 msgr2=0x55727209d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208c8a0 0x55727209d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 -- 172.21.15.3:0/2420365480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208d790 msgr2=0x557272088860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208d790 0x557272088860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.360+0000 7f17ce39b700 1 -- 172.21.15.3:0/2420365480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557271df3c00 con 0x55727208bf40 2023-08-11T14:07:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.361+0000 7f17ce39b700 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005940 tx=0x7f17b800aa40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565397343a8af1d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.364+0000 7f17bd3ff700 1 -- 172.21.15.3:0/2420365480 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800fab0 con 0x55727208bf40 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.364+0000 7f17bd3ff700 1 -- 172.21.15.3:0/2420365480 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b8014070 con 0x55727208bf40 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.364+0000 7f17bd3ff700 1 -- 172.21.15.3:0/2420365480 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801fc80 con 0x55727208bf40 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.367+0000 7f17e3f60b80 1 -- 172.21.15.3:0/2420365480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 msgr2=0x557272088320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.367+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005940 tx=0x7f17b800aa40 comp rx=0 tx=0).stop 2023-08-11T14:07:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.368+0000 7f17e3f60b80 1 -- 172.21.15.3:0/2420365480 shutdown_connections 2023-08-11T14:07:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.368+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208d790 0x557272088860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.368+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727208bf40 0x557272088320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.368+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/2420365480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208c8a0 0x55727209d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.368+0000 7f17e3f60b80 1 -- 172.21.15.3:0/2420365480 >> 172.21.15.3:0/2420365480 conn(0x557271fb98f0 msgr2=0x5572720906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.369+0000 7f17e3f60b80 1 -- 172.21.15.3:0/2420365480 shutdown_connections 2023-08-11T14:07:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.369+0000 7f17e3f60b80 1 -- 172.21.15.3:0/2420365480 wait complete. 2023-08-11T14:07:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.370+0000 7f17e3f60b80 1 -- start start 2023-08-11T14:07:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.370+0000 7f17e3f60b80 1 Processor -- start 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208d790 0x55727209b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727209c3e0 0x55727209c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727209cda0 con 0x55727208bf40 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727209cf10 con 0x55727209c3e0 2023-08-11T14:07:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727209d0d0 con 0x55727208d790 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.376+0000 7f17e3f60b80 10 client.0 _ll_register_callbacks cb 0x7f17b800adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.377+0000 7f17ce39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.377+0000 7f17ce39b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35552/0 (socket says 172.21.15.3:35552) 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.377+0000 7f17ce39b700 1 -- 172.21.15.3:0/23743481 learned_addr learned my addr 172.21.15.3:0/23743481 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.377+0000 7f17ceb9c700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727209c3e0 0x55727209c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.378+0000 7f17ce39b700 1 -- 172.21.15.3:0/23743481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208d790 msgr2=0x55727209b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:07:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.378+0000 7f17ce39b700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208d790 0x55727209b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.378+0000 7f17ce39b700 1 -- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727209c3e0 msgr2=0x55727209c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.378+0000 7f17ce39b700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727209c3e0 0x55727209c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.378+0000 7f17ce39b700 1 -- 172.21.15.3:0/23743481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557271e0ad80 con 0x55727208bf40 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.379+0000 7f17ce39b700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000bc0 tx=0x7f17b801fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4407419160e26e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.379+0000 7f17b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.381+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8028850 con 0x55727208bf40 2023-08-11T14:07:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.381+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f17b8038070 con 0x55727208bf40 2023-08-11T14:07:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.381+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c400f920 con 0x55727208bf40 2023-08-11T14:07:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.382+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b800aea0 con 0x55727208bf40 2023-08-11T14:07:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.385+0000 7f17e3f60b80 10 client.16065 fetch_fsmap learned FSMap version 33 2023-08-11T14:07:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.385+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557271e311f0 con 0x55727208bf40 2023-08-11T14:07:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.389+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c400f920 con 0x55727208bf40 2023-08-11T14:07:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.389+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17b8027b20 con 0x55727208bf40 2023-08-11T14:07:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.389+0000 7f17e3f60b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:07:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.389+0000 7f17e3f60b80 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.389+0000 7f17e3f60b80 20 client.16065 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.390+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720f6710 0x5572720f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.390+0000 7f17e3f60b80 4 client.16065 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:07:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.390+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557271fd77c0 con 0x5572720f6710 2023-08-11T14:07:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.395+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720f6710 0x5572720f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.400+0000 7f17cdb9a700 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720f6710 0x5572720f8b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0cd01c8a3518f0 server_cookie=a0c46890fa4add5d in_seq=0 out_seq=0 2023-08-11T14:07:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:32.400+0000 7f17b1ffb700 10 client.16065 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:33.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 35.0 s, 1825.7 tps, lat 44.329 ms stddev 30.268 2023-08-11T14:07:33.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:33 smithi003 ceph-mon[97195]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2023-08-11T14:07:33.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:33 smithi187 ceph-mon[130973]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2023-08-11T14:07:33.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:33 smithi103 ceph-mon[130434]: pgmap v456: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 1.47k op/s 2023-08-11T14:07:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.297+0000 7f17b1ffb700 1 -- 172.21.15.3:0/23743481 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557271fd77c0 con 0x5572720f6710 2023-08-11T14:07:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.297+0000 7f17b1ffb700 10 client.16065 handle_command_reply: tid=0 2023-08-11T14:07:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.297+0000 7f17e3f60b80 1 client.16065 shutdown 2023-08-11T14:07:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.297+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 msgr2=0x55727209ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.297+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17b8000bc0 tx=0x7f17b801fca0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 shutdown_connections 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5572720f6710 0x5572720f8b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55727209c3e0 0x55727209c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55727208bf40 0x55727209ae90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 --2- 172.21.15.3:0/23743481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55727208d790 0x55727209b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 >> 172.21.15.3:0/23743481 conn(0x557271fb98f0 msgr2=0x557272094990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17b1ffb700 0 client.16065 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 shutdown_connections 2023-08-11T14:07:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.298+0000 7f17e3f60b80 1 -- 172.21.15.3:0/23743481 wait complete. 2023-08-11T14:07:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.299+0000 7f17e3f60b80 20 client.16065 trim_cache size 0 max 16384 2023-08-11T14:07:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:07:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:07:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:07:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.301+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 msgr2=0x7f17b40a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.301+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 0x7f17b40a2590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005e50 tx=0x7f17b800a3d0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.301+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 msgr2=0x7f17c819d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.301+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b840 tx=0x7f17c000bc00 comp rx=0 tx=0).stop 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 shutdown_connections 2023-08-11T14:07:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c810cc60 0x7f17c819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f17b40a00d0 0x7f17b40a2590 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17c8107920 0x7f17c819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 --2- 172.21.15.3:0/3914132366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f17c810a290 0x7f17c819d2a0 secure :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b840 tx=0x7f17c000bc00 comp rx=0 tx=0).stop 2023-08-11T14:07:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.303+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 >> 172.21.15.3:0/3914132366 conn(0x7f17c806d750 msgr2=0x7f17c81083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.307+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 shutdown_connections 2023-08-11T14:07:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:34.309+0000 7f17d05ff700 1 -- 172.21.15.3:0/3914132366 wait complete. 2023-08-11T14:07:34.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:34 smithi187 ceph-mon[130973]: pgmap v457: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.37k op/s 2023-08-11T14:07:34.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:34 smithi003 ceph-mon[97195]: pgmap v457: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.37k op/s 2023-08-11T14:07:34.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:34 smithi103 ceph-mon[130434]: pgmap v457: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 MiB/s wr, 1.37k op/s 2023-08-11T14:07:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:37 smithi103 ceph-mon[130434]: pgmap v458: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.08k op/s 2023-08-11T14:07:37.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:37 smithi003 ceph-mon[97195]: pgmap v458: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.08k op/s 2023-08-11T14:07:37.802 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:37 smithi187 ceph-mon[130973]: pgmap v458: 129 pgs: 129 active+clean; 8.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 MiB/s wr, 2.08k op/s 2023-08-11T14:07:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 40.0 s, 466.3 tps, lat 243.137 ms stddev 339.280 2023-08-11T14:07:38.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:38 smithi003 ceph-mon[97195]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.95k op/s 2023-08-11T14:07:38.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:38 smithi187 ceph-mon[130973]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.95k op/s 2023-08-11T14:07:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:38 smithi103 ceph-mon[130434]: pgmap v459: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 2.95k op/s 2023-08-11T14:07:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:41 smithi103 ceph-mon[130434]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 MiB/s wr, 3.47k op/s 2023-08-11T14:07:41.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:41 smithi187 ceph-mon[130973]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 MiB/s wr, 3.47k op/s 2023-08-11T14:07:41.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:41 smithi003 ceph-mon[97195]: pgmap v460: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 MiB/s wr, 3.47k op/s 2023-08-11T14:07:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 45.0 s, 2042.0 tps, lat 52.187 ms stddev 73.954 2023-08-11T14:07:43.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:43 smithi103 ceph-mon[130434]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.62k op/s 2023-08-11T14:07:43.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:43 smithi003 ceph-mon[97195]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.62k op/s 2023-08-11T14:07:43.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:43 smithi187 ceph-mon[130973]: pgmap v461: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 50 MiB/s wr, 3.62k op/s 2023-08-11T14:07:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:45 smithi103 ceph-mon[130434]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 57 MiB/s wr, 3.89k op/s 2023-08-11T14:07:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:45 smithi003 ceph-mon[97195]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 57 MiB/s wr, 3.89k op/s 2023-08-11T14:07:45.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:45 smithi187 ceph-mon[130973]: pgmap v462: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 57 MiB/s wr, 3.89k op/s 2023-08-11T14:07:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.595+0000 7f72f0dff700 1 -- 172.21.15.3:0/1782839085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec073380 msgr2=0x7f72ec073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.595+0000 7f72f0dff700 1 --2- 172.21.15.3:0/1782839085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec073380 0x7f72ec073800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098f0 tx=0x7f72dc009c00 comp rx=0 tx=0).stop 2023-08-11T14:07:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.596+0000 7f72f0dff700 1 -- 172.21.15.3:0/1782839085 shutdown_connections 2023-08-11T14:07:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.596+0000 7f72f0dff700 1 --2- 172.21.15.3:0/1782839085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec10cf00 0x7f72ec10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.596+0000 7f72f0dff700 1 --2- 172.21.15.3:0/1782839085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec074960 0x7f72ec072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.596+0000 7f72f0dff700 1 --2- 172.21.15.3:0/1782839085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec073380 0x7f72ec073800 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.596+0000 7f72f0dff700 1 -- 172.21.15.3:0/1782839085 >> 172.21.15.3:0/1782839085 conn(0x7f72ec0fbe60 msgr2=0x7f72ec0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.606+0000 7f72f0dff700 1 -- 172.21.15.3:0/1782839085 shutdown_connections 2023-08-11T14:07:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.607+0000 7f72f0dff700 1 -- 172.21.15.3:0/1782839085 wait complete. 2023-08-11T14:07:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.608+0000 7f72f0dff700 1 Processor -- start 2023-08-11T14:07:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.608+0000 7f72f0dff700 1 -- start start 2023-08-11T14:07:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.608+0000 7f72f0dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 0x7f72ec06f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.608+0000 7f72f0dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 0x7f72ec06f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.609+0000 7f72f0dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec10cf00 0x7f72ec070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.609+0000 7f72f0dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec070d90 con 0x7f72ec10cf00 2023-08-11T14:07:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.610+0000 7f72f0dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec070f00 con 0x7f72ec074960 2023-08-11T14:07:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.610+0000 7f72f0dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec0710c0 con 0x7f72ec073380 2023-08-11T14:07:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.609+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 0x7f72ec06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.611+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 0x7f72ec06f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59788/0 (socket says 172.21.15.3:59788) 2023-08-11T14:07:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.611+0000 7f72ea59c700 1 -- 172.21.15.3:0/902918181 learned_addr learned my addr 172.21.15.3:0/902918181 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.611+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 0x7f72ec06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.614+0000 7f72ead9d700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec10cf00 0x7f72ec070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.615+0000 7f72e9d9b700 1 -- 172.21.15.3:0/902918181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 msgr2=0x7f72ec06f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.615+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 0x7f72ec06f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.615+0000 7f72e9d9b700 1 -- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec10cf00 msgr2=0x7f72ec070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.615+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec10cf00 0x7f72ec070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.615+0000 7f72e9d9b700 1 -- 172.21.15.3:0/902918181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009520 con 0x7f72ec074960 2023-08-11T14:07:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.616+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 0x7f72ec06f860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc00fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.617+0000 7f72db7fe700 1 -- 172.21.15.3:0/902918181 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc01c070 con 0x7f72ec074960 2023-08-11T14:07:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.617+0000 7f72db7fe700 1 -- 172.21.15.3:0/902918181 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc005530 con 0x7f72ec074960 2023-08-11T14:07:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.617+0000 7f72db7fe700 1 -- 172.21.15.3:0/902918181 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc003e00 con 0x7f72ec074960 2023-08-11T14:07:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.617+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec071340 con 0x7f72ec074960 2023-08-11T14:07:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.617+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec071bd0 con 0x7f72ec074960 2023-08-11T14:07:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.625+0000 7f72db7fe700 1 -- 172.21.15.3:0/902918181 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f72dc003680 con 0x7f72ec074960 2023-08-11T14:07:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.630+0000 7f72db7fe700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 0x7f72d40a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.630+0000 7f72db7fe700 1 -- 172.21.15.3:0/902918181 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f72dc0cc7e0 con 0x7f72ec074960 2023-08-11T14:07:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.631+0000 7f73046deb80 1 Processor -- start 2023-08-11T14:07:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.639+0000 7f72ea59c700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 0x7f72d40a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.639+0000 7f73046deb80 1 -- start start 2023-08-11T14:07:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fd91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fd9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74f315d0 con 0x557a74fd9790 2023-08-11T14:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74f2eef0 con 0x557a74fd88a0 2023-08-11T14:07:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.640+0000 7f73046deb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74f2a7e0 con 0x557a74fd7f40 2023-08-11T14:07:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.642+0000 7f72e9d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fd91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59792/0 (socket says 172.21.15.3:59792) 2023-08-11T14:07:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 -- 172.21.15.3:0/2889789194 learned_addr learned my addr 172.21.15.3:0/2889789194 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 -- 172.21.15.3:0/2889789194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 msgr2=0x557a74fd91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fd91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 -- 172.21.15.3:0/2889789194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 msgr2=0x557a74fd9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fd9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.651+0000 7f72ea59c700 1 -- 172.21.15.3:0/2889789194 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a74daa5d0 con 0x557a74fd7f40 2023-08-11T14:07:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.653+0000 7f72ea59c700 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72e0014890 tx=0x7f72e0014ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78745b14dc3fd6df server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.654+0000 7f72ea59c700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 0x7f72d40a25e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72e0014fa0 tx=0x7f72e0012000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.654+0000 7f72d8ff9700 1 -- 172.21.15.3:0/2889789194 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0009240 con 0x557a74fd7f40 2023-08-11T14:07:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.654+0000 7f72d8ff9700 1 -- 172.21.15.3:0/2889789194 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e0016e90 con 0x557a74fd7f40 2023-08-11T14:07:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.654+0000 7f72d8ff9700 1 -- 172.21.15.3:0/2889789194 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0018db0 con 0x557a74fd7f40 2023-08-11T14:07:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.656+0000 7f73046deb80 1 -- 172.21.15.3:0/2889789194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 msgr2=0x557a74fd8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.656+0000 7f73046deb80 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72e0014890 tx=0x7f72e0014ba0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.660+0000 7f73046deb80 1 -- 172.21.15.3:0/2889789194 shutdown_connections 2023-08-11T14:07:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.661+0000 7f73046deb80 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fd9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.661+0000 7f73046deb80 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd7f40 0x557a74fd8360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.661+0000 7f73046deb80 1 --2- 172.21.15.3:0/2889789194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fd91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.661+0000 7f73046deb80 1 -- 172.21.15.3:0/2889789194 >> 172.21.15.3:0/2889789194 conn(0x557a74f058f0 msgr2=0x557a74fd4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.664+0000 7f73046deb80 1 -- 172.21.15.3:0/2889789194 shutdown_connections 2023-08-11T14:07:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.664+0000 7f73046deb80 1 -- 172.21.15.3:0/2889789194 wait complete. 2023-08-11T14:07:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.665+0000 7f73046deb80 1 -- start start 2023-08-11T14:07:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 Processor -- start 2023-08-11T14:07:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fe1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fe91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74ec8a60 con 0x557a74fd9790 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74ec8760 con 0x557a74fd7f40 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74dac8c0 con 0x557a74fd88a0 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.666+0000 7f73046deb80 10 client.0 _ll_register_callbacks cb 0x557a74ecba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.667+0000 7f72e9d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.667+0000 7f72e9d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59808/0 (socket says 172.21.15.3:59808) 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.667+0000 7f72e9d9b700 1 -- 172.21.15.3:0/922603806 learned_addr learned my addr 172.21.15.3:0/922603806 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72ea59c700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fe1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72e9d9b700 1 -- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 msgr2=0x557a74fe1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fe1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72e9d9b700 1 -- 172.21.15.3:0/922603806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 msgr2=0x557a74fe91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fe91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.668+0000 7f72e9d9b700 1 -- 172.21.15.3:0/922603806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00144c0 con 0x557a74fd88a0 2023-08-11T14:07:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.669+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0969b0 tx=0x7f72dc0963b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d08a8e9131adea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.669+0000 7f72d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:07:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.669+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0b7ea0 con 0x557a74fd88a0 2023-08-11T14:07:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.669+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f72dc099f40 con 0x557a74fd88a0 2023-08-11T14:07:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.669+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a74fcb7e0 con 0x557a74fd88a0 2023-08-11T14:07:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.670+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72dc09dd70 con 0x557a74fd88a0 2023-08-11T14:07:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.670+0000 7f73046deb80 10 client.26107 fetch_fsmap learned FSMap version 33 2023-08-11T14:07:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.670+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557a74d56d80 con 0x557a74fd88a0 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.671+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc0c6020 con 0x557a74fd88a0 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f72dc0b4070 con 0x557a74fd88a0 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 10 client.26107 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 20 client.26107 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a74fed7b0 0x557a74fefbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 4 client.26107 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:07:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.672+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a74fa7480 con 0x557a74fed7b0 2023-08-11T14:07:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.673+0000 7f72ea59c700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a74fed7b0 0x557a74fefbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.674+0000 7f72ea59c700 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a74fed7b0 0x557a74fefbd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a59e3a0fdad146 server_cookie=a861ec31c82e54b7 in_seq=0 out_seq=0 2023-08-11T14:07:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.674+0000 7f72d1ffb700 10 client.26107 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.675+0000 7f72d1ffb700 1 -- 172.21.15.3:0/922603806 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x557a74fa7480 con 0x557a74fed7b0 2023-08-11T14:07:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.675+0000 7f72d1ffb700 10 client.26107 handle_command_reply: tid=0 2023-08-11T14:07:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.675+0000 7f73046deb80 1 client.26107 shutdown 2023-08-11T14:07:47.677 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:47 smithi103 ceph-mon[130434]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.58k op/s 2023-08-11T14:07:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 msgr2=0x557a74fe2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0969b0 tx=0x7f72dc0963b0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 shutdown_connections 2023-08-11T14:07:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd9790 0x557a74fe91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a74fed7b0 0x557a74fefbd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fe1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 --2- 172.21.15.3:0/922603806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd88a0 0x557a74fe2160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.687+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 >> 172.21.15.3:0/922603806 conn(0x557a74f058f0 msgr2=0x557a74fd3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.688+0000 7f72d1ffb700 0 client.26107 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.688+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 shutdown_connections 2023-08-11T14:07:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.688+0000 7f73046deb80 1 -- 172.21.15.3:0/922603806 wait complete. 2023-08-11T14:07:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.689+0000 7f73046deb80 20 client.26107 trim_cache size 0 max 16384 2023-08-11T14:07:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.732+0000 7f73046deb80 1 Processor -- start 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 -- start start 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fdc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd88a0 0x557a74fdcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fcdea0 con 0x557a74fd88a0 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fd8c80 con 0x557a74fd7f40 2023-08-11T14:07:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.733+0000 7f73046deb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fe2da0 con 0x557a74fd9790 2023-08-11T14:07:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.734+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.734+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59820/0 (socket says 172.21.15.3:59820) 2023-08-11T14:07:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.734+0000 7f72ead9d700 1 -- 172.21.15.3:0/3248110730 learned_addr learned my addr 172.21.15.3:0/3248110730 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.735+0000 7f72ead9d700 1 -- 172.21.15.3:0/3248110730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 msgr2=0x557a74fdc6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:07:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.735+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fdc6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.735+0000 7f72ead9d700 1 -- 172.21.15.3:0/3248110730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd88a0 msgr2=0x557a74fdcbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:07:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.735+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd88a0 0x557a74fdcbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.735+0000 7f72ead9d700 1 -- 172.21.15.3:0/3248110730 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a74d3fc00 con 0x557a74fd9790 2023-08-11T14:07:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.737+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009bf0 tx=0x7f72e4009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f01a484ade905 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.741+0000 7f72d97fa700 1 -- 172.21.15.3:0/3248110730 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4014410 con 0x557a74fd9790 2023-08-11T14:07:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.741+0000 7f72d97fa700 1 -- 172.21.15.3:0/3248110730 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e400f050 con 0x557a74fd9790 2023-08-11T14:07:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.741+0000 7f72d97fa700 1 -- 172.21.15.3:0/3248110730 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4013790 con 0x557a74fd9790 2023-08-11T14:07:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.743+0000 7f73046deb80 1 -- 172.21.15.3:0/3248110730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 msgr2=0x557a74fe9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.743+0000 7f73046deb80 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009bf0 tx=0x7f72e4009f00 comp rx=0 tx=0).stop 2023-08-11T14:07:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.745+0000 7f73046deb80 1 -- 172.21.15.3:0/3248110730 shutdown_connections 2023-08-11T14:07:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.745+0000 7f73046deb80 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe9160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.745+0000 7f73046deb80 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd7f40 0x557a74fdc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.745+0000 7f73046deb80 1 --2- 172.21.15.3:0/3248110730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd88a0 0x557a74fdcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.745+0000 7f73046deb80 1 -- 172.21.15.3:0/3248110730 >> 172.21.15.3:0/3248110730 conn(0x557a74f058f0 msgr2=0x557a74fdfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.746+0000 7f73046deb80 1 -- 172.21.15.3:0/3248110730 shutdown_connections 2023-08-11T14:07:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.746+0000 7f73046deb80 1 -- 172.21.15.3:0/3248110730 wait complete. 2023-08-11T14:07:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.746+0000 7f73046deb80 1 -- start start 2023-08-11T14:07:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 Processor -- start 2023-08-11T14:07:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd7f40 0x557a74fe5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fe5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fe8ba0 con 0x557a74fd7f40 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fe8d10 con 0x557a74fd88a0 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a74fe8ed0 con 0x557a74fd9790 2023-08-11T14:07:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.747+0000 7f73046deb80 10 client.0 _ll_register_callbacks cb 0x557a74ecba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:07:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.748+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.748+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:59828/0 (socket says 172.21.15.3:59828) 2023-08-11T14:07:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.748+0000 7f72ead9d700 1 -- 172.21.15.3:0/3434966062 learned_addr learned my addr 172.21.15.3:0/3434966062 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:07:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.748+0000 7f72e9d9b700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fe5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 -- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 msgr2=0x557a74fe5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fe5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 -- 172.21.15.3:0/3434966062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd7f40 msgr2=0x557a74fe5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:07:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd7f40 0x557a74fe5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 -- 172.21.15.3:0/3434966062 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e40098a0 con 0x557a74fd9790 2023-08-11T14:07:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72ead9d700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72e4007f60 tx=0x7f72e4007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0f177e8b62e379 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:07:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.750+0000 7f72d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:07:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.751+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4014b70 con 0x557a74fd9790 2023-08-11T14:07:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.751+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f72e4014cd0 con 0x557a74fd9790 2023-08-11T14:07:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.757+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72e0016e90 con 0x557a74fd9790 2023-08-11T14:07:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.760+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e4013510 con 0x557a74fd9790 2023-08-11T14:07:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.763+0000 7f73046deb80 10 client.26113 fetch_fsmap learned FSMap version 33 2023-08-11T14:07:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.763+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557a74d7d1f0 con 0x557a74fd9790 2023-08-11T14:07:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72e0016e90 con 0x557a74fd9790 2023-08-11T14:07:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f72e4005320 con 0x557a74fd9790 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 10 client.26113 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 20 client.26113 populate_metadata read hostname 'smithi003' 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a75042640 0x557a75044a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 4 client.26113 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:07:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.764+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a74f237c0 con 0x557a75042640 2023-08-11T14:07:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.766+0000 7f72ea59c700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a75042640 0x557a75044a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:07:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.769+0000 7f72ea59c700 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a75042640 0x557a75044a60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f66149e100fd5a6 server_cookie=6f2ebbe00da7a90a in_seq=0 out_seq=0 2023-08-11T14:07:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.769+0000 7f72d1ffb700 10 client.26113 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:47.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:47 smithi187 ceph-mon[130973]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.58k op/s 2023-08-11T14:07:47.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:47 smithi003 ceph-mon[97195]: pgmap v463: 129 pgs: 129 active+clean; 8.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 49 MiB/s wr, 3.58k op/s 2023-08-11T14:07:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.870+0000 7f72d1ffb700 1 -- 172.21.15.3:0/3434966062 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a74f237c0 con 0x557a75042640 2023-08-11T14:07:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.870+0000 7f72d1ffb700 10 client.26113 handle_command_reply: tid=0 2023-08-11T14:07:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.870+0000 7f73046deb80 1 client.26113 shutdown 2023-08-11T14:07:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.877+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 msgr2=0x557a74fe8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.877+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f72e4007f60 tx=0x7f72e4007f90 comp rx=0 tx=0).stop 2023-08-11T14:07:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.885+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 shutdown_connections 2023-08-11T14:07:47.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.885+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a74fd9790 0x557a74fe8660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.885+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a74fd7f40 0x557a74fe5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.885+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x557a75042640 0x557a75044a60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.886+0000 7f73046deb80 1 --2- 172.21.15.3:0/3434966062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a74fd88a0 0x557a74fe5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.886+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 >> 172.21.15.3:0/3434966062 conn(0x557a74f058f0 msgr2=0x557a74fdc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.886+0000 7f72d1ffb700 0 client.26113 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:07:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.896+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 shutdown_connections 2023-08-11T14:07:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.898+0000 7f73046deb80 1 -- 172.21.15.3:0/3434966062 wait complete. 2023-08-11T14:07:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.898+0000 7f73046deb80 20 client.26113 trim_cache size 0 max 16384 2023-08-11T14:07:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:07:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:07:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:07:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:07:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.901+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 msgr2=0x7f72d40a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.901+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 0x7f72d40a25e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72e0014fa0 tx=0x7f72e0012000 comp rx=0 tx=0).stop 2023-08-11T14:07:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.901+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 msgr2=0x7f72ec06f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.901+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 0x7f72ec06f860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc00fa90 comp rx=0 tx=0).stop 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 shutdown_connections 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72ec10cf00 0x7f72ec070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec073380 0x7f72ec06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72d40a0120 0x7f72d40a25e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 --2- 172.21.15.3:0/902918181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72ec074960 0x7f72ec06f860 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc00fa90 comp rx=0 tx=0).stop 2023-08-11T14:07:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.904+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 >> 172.21.15.3:0/902918181 conn(0x7f72ec0fbe60 msgr2=0x7f72ec0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:07:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.906+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 shutdown_connections 2023-08-11T14:07:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:07:47.909+0000 7f72f0dff700 1 -- 172.21.15.3:0/902918181 wait complete. 2023-08-11T14:07:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 50.0 s, 1201.6 tps, lat 82.393 ms stddev 279.166 2023-08-11T14:07:49.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:49 smithi187 ceph-mon[130973]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 3.25k op/s 2023-08-11T14:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:49 smithi003 ceph-mon[97195]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 3.25k op/s 2023-08-11T14:07:49.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:49 smithi103 ceph-mon[130434]: pgmap v464: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 MiB/s wr, 3.25k op/s 2023-08-11T14:07:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:50 smithi103 ceph-mon[130434]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 2.54k op/s 2023-08-11T14:07:51.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:50 smithi003 ceph-mon[97195]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 2.54k op/s 2023-08-11T14:07:51.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:50 smithi187 ceph-mon[130973]: pgmap v465: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 2.54k op/s 2023-08-11T14:07:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 55.0 s, 1933.7 tps, lat 47.917 ms stddev 48.965 2023-08-11T14:07:53.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:53 smithi003 ceph-mon[97195]: pgmap v466: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-08-11T14:07:53.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:53 smithi187 ceph-mon[130973]: pgmap v466: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-08-11T14:07:53.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:53 smithi103 ceph-mon[130434]: pgmap v466: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 2.00k op/s 2023-08-11T14:07:55.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:55 smithi003 ceph-mon[97195]: pgmap v467: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 2.19k op/s 2023-08-11T14:07:55.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:55 smithi187 ceph-mon[130973]: pgmap v467: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 2.19k op/s 2023-08-11T14:07:55.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:55 smithi103 ceph-mon[130434]: pgmap v467: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 2.19k op/s 2023-08-11T14:07:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:57 smithi003 ceph-mon[97195]: pgmap v468: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 42 MiB/s wr, 2.44k op/s 2023-08-11T14:07:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:57 smithi187 ceph-mon[130973]: pgmap v468: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 42 MiB/s wr, 2.44k op/s 2023-08-11T14:07:57.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:57 smithi103 ceph-mon[130434]: pgmap v468: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 42 MiB/s wr, 2.44k op/s 2023-08-11T14:07:58.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 60.0 s, 719.1 tps, lat 149.977 ms stddev 202.414 2023-08-11T14:07:58.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:07:58 smithi103 ceph-mon[130434]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 MiB/s wr, 3.04k op/s 2023-08-11T14:07:59.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:07:58 smithi003 ceph-mon[97195]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 MiB/s wr, 3.04k op/s 2023-08-11T14:07:59.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:07:58 smithi187 ceph-mon[130973]: pgmap v469: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 MiB/s wr, 3.04k op/s 2023-08-11T14:08:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.195+0000 7fc50afff700 1 -- 172.21.15.3:0/3472849301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 msgr2=0x7fc50407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.195+0000 7fc50afff700 1 --2- 172.21.15.3:0/3472849301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc50407a500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00b1a0 tx=0x7fc4fc00b4b0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.200+0000 7fc50afff700 1 -- 172.21.15.3:0/3472849301 shutdown_connections 2023-08-11T14:08:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.200+0000 7fc50afff700 1 --2- 172.21.15.3:0/3472849301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc50407a500 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.200+0000 7fc50afff700 1 --2- 172.21.15.3:0/3472849301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040727c0 0x7fc504072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.200+0000 7fc50afff700 1 --2- 172.21.15.3:0/3472849301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040756a0 0x7fc504077b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.200+0000 7fc50afff700 1 -- 172.21.15.3:0/3472849301 >> 172.21.15.3:0/3472849301 conn(0x7fc50406d750 msgr2=0x7fc50406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.202+0000 7fc50afff700 1 -- 172.21.15.3:0/3472849301 shutdown_connections 2023-08-11T14:08:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.202+0000 7fc50afff700 1 -- 172.21.15.3:0/3472849301 wait complete. 2023-08-11T14:08:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.204+0000 7fc50afff700 1 Processor -- start 2023-08-11T14:08:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 -- start start 2023-08-11T14:08:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 0x7fc5041b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 0x7fc5041b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc5041b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5041b5900 con 0x7fc5040756a0 2023-08-11T14:08:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5041b5a70 con 0x7fc5040727c0 2023-08-11T14:08:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc50afff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5041b5c30 con 0x7fc504078070 2023-08-11T14:08:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.205+0000 7fc508d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 0x7fc5041b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc508d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 0x7fc5041b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47754/0 (socket says 172.21.15.3:47754) 2023-08-11T14:08:01.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc508d9b700 1 -- 172.21.15.3:0/2996078049 learned_addr learned my addr 172.21.15.3:0/2996078049 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 0x7fc5041b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc50959c700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc5041b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 -- 172.21.15.3:0/2996078049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 msgr2=0x7fc5041b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc5041b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 -- 172.21.15.3:0/2996078049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 msgr2=0x7fc5041b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 0x7fc5041b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc4fbfff700 1 -- 172.21.15.3:0/2996078049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc009d00 con 0x7fc5040756a0 2023-08-11T14:08:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.206+0000 7fc50959c700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc5041b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:08:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.207+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 0x7fc5041b2ba0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400a8d0 tx=0x7fc4f400abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.216+0000 7fc4f9ffb700 1 -- 172.21.15.3:0/2996078049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4010070 con 0x7fc5040756a0 2023-08-11T14:08:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.216+0000 7fc4f9ffb700 1 -- 172.21.15.3:0/2996078049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4f4007ab0 con 0x7fc5040756a0 2023-08-11T14:08:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.216+0000 7fc4f9ffb700 1 -- 172.21.15.3:0/2996078049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4014c20 con 0x7fc5040756a0 2023-08-11T14:08:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.218+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5041b5f10 con 0x7fc5040756a0 2023-08-11T14:08:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.218+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5041b63a0 con 0x7fc5040756a0 2023-08-11T14:08:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.227+0000 7fc51e82bb80 1 Processor -- start 2023-08-11T14:08:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.227+0000 7fc51e82bb80 1 -- start start 2023-08-11T14:08:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f428360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f429c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f3815d0 con 0x55a74f427f40 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f37eef0 con 0x55a74f4288a0 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.228+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f37a7e0 con 0x55a74f429790 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.229+0000 7fc508d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.229+0000 7fc508d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47770/0 (socket says 172.21.15.3:47770) 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.229+0000 7fc508d9b700 1 -- 172.21.15.3:0/4107179456 learned_addr learned my addr 172.21.15.3:0/4107179456 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.234+0000 7fc50959c700 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f428360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.235+0000 7fc508d9b700 1 -- 172.21.15.3:0/4107179456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 msgr2=0x55a74f429c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:08:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.235+0000 7fc508d9b700 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f429c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.235+0000 7fc508d9b700 1 -- 172.21.15.3:0/4107179456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 msgr2=0x55a74f428360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.235+0000 7fc508d9b700 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f428360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.235+0000 7fc508d9b700 1 -- 172.21.15.3:0/4107179456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a74f1fa5d0 con 0x55a74f4288a0 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.236+0000 7fc508d9b700 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc50000aca0 tx=0x7fc5000071c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef00ccea7f1b0294 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.236+0000 7fc4f9ffb700 1 -- 172.21.15.3:0/2996078049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc4f40075f0 con 0x7fc5040756a0 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.236+0000 7fc4eaffd700 1 -- 172.21.15.3:0/4107179456 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50000f040 con 0x55a74f4288a0 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.237+0000 7fc4eaffd700 1 -- 172.21.15.3:0/4107179456 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5000044f0 con 0x55a74f4288a0 2023-08-11T14:08:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.237+0000 7fc4eaffd700 1 -- 172.21.15.3:0/4107179456 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500015470 con 0x55a74f4288a0 2023-08-11T14:08:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.238+0000 7fc51e82bb80 1 -- 172.21.15.3:0/4107179456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 msgr2=0x55a74f4291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.238+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc50000aca0 tx=0x7fc5000071c0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.239+0000 7fc4f9ffb700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 0x7fc4ec0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.239+0000 7fc4f9ffb700 1 -- 172.21.15.3:0/2996078049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc4f40ca590 con 0x7fc5040756a0 2023-08-11T14:08:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.240+0000 7fc51e82bb80 1 -- 172.21.15.3:0/4107179456 shutdown_connections 2023-08-11T14:08:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.240+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f429c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.240+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f428360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.240+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/4107179456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4288a0 0x55a74f4291b0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc50000aca0 tx=0x7fc5000071c0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.240+0000 7fc51e82bb80 1 -- 172.21.15.3:0/4107179456 >> 172.21.15.3:0/4107179456 conn(0x55a74f3558f0 msgr2=0x55a74f424920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.241+0000 7fc51e82bb80 1 -- 172.21.15.3:0/4107179456 shutdown_connections 2023-08-11T14:08:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.242+0000 7fc50959c700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 0x7fc4ec0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.242+0000 7fc51e82bb80 1 -- 172.21.15.3:0/4107179456 wait complete. 2023-08-11T14:08:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.243+0000 7fc51e82bb80 1 -- start start 2023-08-11T14:08:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.243+0000 7fc51e82bb80 1 Processor -- start 2023-08-11T14:08:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f431f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f4324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f318a60 con 0x55a74f427f40 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f318760 con 0x55a74f4374d0 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f1fc8c0 con 0x55a74f429790 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc51e82bb80 10 client.0 _ll_register_callbacks cb 0x55a74f31ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.244+0000 7fc50959c700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 0x7fc4ec0a2560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc504083b30 tx=0x7fc4fc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.245+0000 7fc50959c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.245+0000 7fc50959c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47786/0 (socket says 172.21.15.3:47786) 2023-08-11T14:08:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.245+0000 7fc50959c700 1 -- 172.21.15.3:0/1877897234 learned_addr learned my addr 172.21.15.3:0/1877897234 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.245+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f4324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc508d9b700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f431f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc50959c700 1 -- 172.21.15.3:0/1877897234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 msgr2=0x55a74f4324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc50959c700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f4324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc50959c700 1 -- 172.21.15.3:0/1877897234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 msgr2=0x55a74f431f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc50959c700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f431f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc50959c700 1 -- 172.21.15.3:0/1877897234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50000a8d0 con 0x55a74f4374d0 2023-08-11T14:08:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc508d9b700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f431f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:08:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.246+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f4324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:08:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.247+0000 7fc50959c700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc003d00 tx=0x7fc4fc00b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d016d18cd3d71fb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.247+0000 7fc4e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.247+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc010030 con 0x55a74f4374d0 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.247+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc4fc0056c0 con 0x55a74f4374d0 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.247+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a74f41b7e0 con 0x55a74f4374d0 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4fc005880 con 0x55a74f4374d0 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc51e82bb80 10 client.26150 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a74f1a6d80 con 0x55a74f4374d0 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a74f41b7e0 con 0x55a74f4374d0 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc4fc0183f0 con 0x55a74f4374d0 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc51e82bb80 10 client.26150 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc51e82bb80 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.248+0000 7fc51e82bb80 20 client.26150 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.249+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f43e170 0x55a74f440590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.249+0000 7fc51e82bb80 4 client.26150 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:08:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.249+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a74f3f7480 con 0x55a74f43e170 2023-08-11T14:08:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.249+0000 7fc508d9b700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f43e170 0x55a74f440590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.250+0000 7fc4e3fff700 10 client.26150 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.250+0000 7fc508d9b700 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f43e170 0x55a74f440590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958687040ffc7813 server_cookie=37c588db34229e75 in_seq=0 out_seq=0 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.250+0000 7fc4e3fff700 1 -- 172.21.15.3:0/1877897234 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a74f3f7480 con 0x55a74f43e170 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.250+0000 7fc4e3fff700 10 client.26150 handle_command_reply: tid=0 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 client.26150 shutdown 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 msgr2=0x55a74f4398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc003d00 tx=0x7fc4fc00b4b0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 shutdown_connections 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f43e170 0x55a74f440590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f4374d0 0x55a74f4398c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f431f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/1877897234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f429790 0x55a74f4324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 >> 172.21.15.3:0/1877897234 conn(0x55a74f3558f0 msgr2=0x55a74f423710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc4e3fff700 0 client.26150 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.251+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 shutdown_connections 2023-08-11T14:08:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.252+0000 7fc51e82bb80 1 -- 172.21.15.3:0/1877897234 wait complete. 2023-08-11T14:08:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.254+0000 7fc51e82bb80 20 client.26150 trim_cache size 0 max 16384 2023-08-11T14:08:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.280+0000 7fc51e82bb80 1 Processor -- start 2023-08-11T14:08:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 -- start start 2023-08-11T14:08:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f437490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f42c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f41b7e0 con 0x55a74f427f40 2023-08-11T14:08:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f42c450 con 0x55a74f429790 2023-08-11T14:08:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.281+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f42d280 con 0x55a74f42ce20 2023-08-11T14:08:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58886/0 (socket says 172.21.15.3:58886) 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 -- 172.21.15.3:0/550817640 learned_addr learned my addr 172.21.15.3:0/550817640 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 -- 172.21.15.3:0/550817640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 msgr2=0x55a74f42c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f42c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 -- 172.21.15.3:0/550817640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 msgr2=0x55a74f437490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f437490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.282+0000 7fc50959c700 1 -- 172.21.15.3:0/550817640 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a74f18fc00 con 0x55a74f42ce20 2023-08-11T14:08:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.283+0000 7fc50959c700 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0090a0 tx=0x7fc4fc0096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14dfbebc0747ad68 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.283+0000 7fc4eb7fe700 1 -- 172.21.15.3:0/550817640 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc005330 con 0x55a74f42ce20 2023-08-11T14:08:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.284+0000 7fc4eb7fe700 1 -- 172.21.15.3:0/550817640 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4fc011040 con 0x55a74f42ce20 2023-08-11T14:08:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.284+0000 7fc4eb7fe700 1 -- 172.21.15.3:0/550817640 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc020420 con 0x55a74f42ce20 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 -- 172.21.15.3:0/550817640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 msgr2=0x55a74f431f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0090a0 tx=0x7fc4fc0096e0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 -- 172.21.15.3:0/550817640 shutdown_connections 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f431f60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f437490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/550817640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f42c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.285+0000 7fc51e82bb80 1 -- 172.21.15.3:0/550817640 >> 172.21.15.3:0/550817640 conn(0x55a74f3558f0 msgr2=0x55a74f42bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.286+0000 7fc51e82bb80 1 -- 172.21.15.3:0/550817640 shutdown_connections 2023-08-11T14:08:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.286+0000 7fc51e82bb80 1 -- 172.21.15.3:0/550817640 wait complete. 2023-08-11T14:08:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.287+0000 7fc51e82bb80 1 -- start start 2023-08-11T14:08:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.287+0000 7fc51e82bb80 1 Processor -- start 2023-08-11T14:08:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f434890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f434dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f435de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f436320 con 0x55a74f427f40 2023-08-11T14:08:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f436490 con 0x55a74f429790 2023-08-11T14:08:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74f436650 con 0x55a74f42ce20 2023-08-11T14:08:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.288+0000 7fc51e82bb80 10 client.0 _ll_register_callbacks cb 0x7fc4fc007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.289+0000 7fc50959c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f435de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.289+0000 7fc50959c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f435de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58888/0 (socket says 172.21.15.3:58888) 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.289+0000 7fc50959c700 1 -- 172.21.15.3:0/3933539750 learned_addr learned my addr 172.21.15.3:0/3933539750 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.289+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f434dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 -- 172.21.15.3:0/3933539750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 msgr2=0x55a74f435de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f435de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 -- 172.21.15.3:0/3933539750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 msgr2=0x55a74f434890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f434890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 -- 172.21.15.3:0/3933539750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a74f1a6d80 con 0x55a74f429790 2023-08-11T14:08:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.290+0000 7fc4fbfff700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f434dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55a74f428f70 tx=0x7fc4f4097af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13f32513b04249e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.294+0000 7fc4e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.294+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4098360 con 0x55a74f429790 2023-08-11T14:08:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.294+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc4f4097c60 con 0x55a74f429790 2023-08-11T14:08:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.294+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5000044f0 con 0x55a74f429790 2023-08-11T14:08:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.294+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4f409ba90 con 0x55a74f429790 2023-08-11T14:08:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.295+0000 7fc51e82bb80 10 client.26153 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.295+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a74f1cd1f0 con 0x55a74f429790 2023-08-11T14:08:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.295+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5000044f0 con 0x55a74f429790 2023-08-11T14:08:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc4f409f5e0 con 0x55a74f429790 2023-08-11T14:08:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 10 client.26153 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 10 client.26153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 20 client.26153 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f492430 0x55a74f494820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 4 client.26153 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:08:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a74f3737c0 con 0x55a74f492430 2023-08-11T14:08:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.296+0000 7fc508d9b700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f492430 0x55a74f494820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.297+0000 7fc508d9b700 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f492430 0x55a74f494820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91974d9e82b1641 server_cookie=fa83f76371c1405d in_seq=0 out_seq=0 2023-08-11T14:08:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.297+0000 7fc4e3fff700 10 client.26153 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.335+0000 7fc4e3fff700 1 -- 172.21.15.3:0/3933539750 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a74f3737c0 con 0x55a74f492430 2023-08-11T14:08:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.335+0000 7fc4e3fff700 10 client.26153 handle_command_reply: tid=0 2023-08-11T14:08:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.335+0000 7fc51e82bb80 1 client.26153 shutdown 2023-08-11T14:08:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 msgr2=0x55a74f434dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f434dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55a74f428f70 tx=0x7fc4f4097af0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 shutdown_connections 2023-08-11T14:08:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a74f42ce20 0x55a74f435de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a74f427f40 0x55a74f434890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a74f492430 0x55a74f494820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 --2- 172.21.15.3:0/3933539750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a74f429790 0x55a74f434dd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 >> 172.21.15.3:0/3933539750 conn(0x55a74f3558f0 msgr2=0x55a74f431f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.336+0000 7fc4e3fff700 0 client.26153 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.337+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 shutdown_connections 2023-08-11T14:08:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.337+0000 7fc51e82bb80 1 -- 172.21.15.3:0/3933539750 wait complete. 2023-08-11T14:08:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.337+0000 7fc51e82bb80 20 client.26153 trim_cache size 0 max 16384 2023-08-11T14:08:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:08:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:08:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:08:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:08:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 msgr2=0x7fc4ec0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 0x7fc4ec0a2560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc504083b30 tx=0x7fc4fc00d040 comp rx=0 tx=0).stop 2023-08-11T14:08:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 msgr2=0x7fc5041b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 0x7fc5041b2ba0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400a8d0 tx=0x7fc4f400abe0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 shutdown_connections 2023-08-11T14:08:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc504078070 0x7fc5041b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc4ec0a00a0 0x7fc4ec0a2560 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5040727c0 0x7fc5041b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 --2- 172.21.15.3:0/2996078049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5040756a0 0x7fc5041b2ba0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 >> 172.21.15.3:0/2996078049 conn(0x7fc50406d750 msgr2=0x7fc504076ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 shutdown_connections 2023-08-11T14:08:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:01.338+0000 7fc50afff700 1 -- 172.21.15.3:0/2996078049 wait complete. 2023-08-11T14:08:01.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:01 smithi003 ceph-mon[97195]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 42 MiB/s wr, 2.88k op/s 2023-08-11T14:08:01.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:01 smithi187 ceph-mon[130973]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 42 MiB/s wr, 2.88k op/s 2023-08-11T14:08:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:01 smithi103 ceph-mon[130434]: pgmap v470: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 42 MiB/s wr, 2.88k op/s 2023-08-11T14:08:03.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 65.0 s, 2252.8 tps, lat 44.470 ms stddev 44.759 2023-08-11T14:08:03.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:03 smithi003 ceph-mon[97195]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 2.72k op/s 2023-08-11T14:08:03.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:03 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:08:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:03 smithi187 ceph-mon[130973]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 2.72k op/s 2023-08-11T14:08:03.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:03 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:08:03.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:03 smithi103 ceph-mon[130434]: pgmap v471: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 MiB/s wr, 2.72k op/s 2023-08-11T14:08:03.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:03 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:08:05.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:05 smithi103 ceph-mon[130434]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 2.69k op/s 2023-08-11T14:08:05.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:08:05.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:08:05.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:08:06.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:05 smithi003 ceph-mon[97195]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 2.69k op/s 2023-08-11T14:08:06.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:08:06.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:08:06.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:08:06.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:05 smithi187 ceph-mon[130973]: pgmap v472: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 2.69k op/s 2023-08-11T14:08:06.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:08:06.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:08:06.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:08:06.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:06 smithi103 ceph-mon[130434]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 2.29k op/s 2023-08-11T14:08:07.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:06 smithi003 ceph-mon[97195]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 2.29k op/s 2023-08-11T14:08:07.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:06 smithi187 ceph-mon[130973]: pgmap v473: 129 pgs: 129 active+clean; 8.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 MiB/s wr, 2.29k op/s 2023-08-11T14:08:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 70.0 s, 841.1 tps, lat 118.886 ms stddev 201.241 2023-08-11T14:08:09.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:09 smithi103 ceph-mon[130434]: pgmap v474: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 MiB/s wr, 2.01k op/s 2023-08-11T14:08:10.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:09 smithi187 ceph-mon[130973]: pgmap v474: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 MiB/s wr, 2.01k op/s 2023-08-11T14:08:10.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:09 smithi003 ceph-mon[97195]: pgmap v474: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 MiB/s wr, 2.01k op/s 2023-08-11T14:08:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:10 smithi103 ceph-mon[130434]: pgmap v475: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 33 MiB/s wr, 1.48k op/s 2023-08-11T14:08:11.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:10 smithi003 ceph-mon[97195]: pgmap v475: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 33 MiB/s wr, 1.48k op/s 2023-08-11T14:08:11.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:10 smithi187 ceph-mon[130973]: pgmap v475: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 33 MiB/s wr, 1.48k op/s 2023-08-11T14:08:13.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 75.0 s, 1893.7 tps, lat 52.971 ms stddev 44.365 2023-08-11T14:08:13.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:13 smithi003 ceph-mon[97195]: pgmap v476: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-08-11T14:08:13.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:13 smithi103 ceph-mon[130434]: pgmap v476: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-08-11T14:08:14.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:13 smithi187 ceph-mon[130973]: pgmap v476: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 1.19k op/s 2023-08-11T14:08:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.647+0000 7f99a1bff700 1 -- 172.21.15.3:0/495645154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10cc60 msgr2=0x7f999c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.647+0000 7f99a1bff700 1 --2- 172.21.15.3:0/495645154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10cc60 0x7f999c10d0e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f999400b0a0 tx=0x7f999400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.658+0000 7f99a1bff700 1 -- 172.21.15.3:0/495645154 shutdown_connections 2023-08-11T14:08:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.658+0000 7f99a1bff700 1 --2- 172.21.15.3:0/495645154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10cc60 0x7f999c10d0e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.658+0000 7f99a1bff700 1 --2- 172.21.15.3:0/495645154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f999c107920 0x7f999c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.658+0000 7f99a1bff700 1 --2- 172.21.15.3:0/495645154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10a290 0x7f999c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.658+0000 7f99a1bff700 1 -- 172.21.15.3:0/495645154 >> 172.21.15.3:0/495645154 conn(0x7f999c06d750 msgr2=0x7f999c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.672+0000 7f99a1bff700 1 -- 172.21.15.3:0/495645154 shutdown_connections 2023-08-11T14:08:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.675+0000 7f99a1bff700 1 -- 172.21.15.3:0/495645154 wait complete. 2023-08-11T14:08:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.676+0000 7f99a1bff700 1 Processor -- start 2023-08-11T14:08:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.677+0000 7f99a1bff700 1 -- start start 2023-08-11T14:08:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f999c107920 0x7f999c118390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10a290 0x7f999c1188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c119230 con 0x7f999c107920 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c117210 con 0x7f999c10cc60 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.684+0000 7f99a1bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1173d0 con 0x7f999c10a290 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.685+0000 7f999bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.685+0000 7f999bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:36516/0 (socket says 172.21.15.3:36516) 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.685+0000 7f999bfff700 1 -- 172.21.15.3:0/1656496345 learned_addr learned my addr 172.21.15.3:0/1656496345 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.685+0000 7f999affd700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10a290 0x7f999c1188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.687+0000 7f999bfff700 1 -- 172.21.15.3:0/1656496345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10a290 msgr2=0x7f999c1188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.687+0000 7f999bfff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10a290 0x7f999c1188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.687+0000 7f999bfff700 1 -- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f999c107920 msgr2=0x7f999c118390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.687+0000 7f999bfff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f999c107920 0x7f999c118390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.687+0000 7f999bfff700 1 -- 172.21.15.3:0/1656496345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009d00 con 0x7f999c10cc60 2023-08-11T14:08:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.689+0000 7f999bfff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9994009300 tx=0x7f9994009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.690+0000 7f9998ff9700 1 -- 172.21.15.3:0/1656496345 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400e070 con 0x7f999c10cc60 2023-08-11T14:08:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.690+0000 7f9998ff9700 1 -- 172.21.15.3:0/1656496345 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9994003e10 con 0x7f999c10cc60 2023-08-11T14:08:14.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.690+0000 7f9998ff9700 1 -- 172.21.15.3:0/1656496345 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999401a8f0 con 0x7f999c10cc60 2023-08-11T14:08:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.693+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c117650 con 0x7f999c10cc60 2023-08-11T14:08:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.693+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c1b6600 con 0x7f999c10cc60 2023-08-11T14:08:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.704+0000 7f99b544fb80 1 Processor -- start 2023-08-11T14:08:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.705+0000 7f99b544fb80 1 -- start start 2023-08-11T14:08:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.705+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 0x560a51fb4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.705+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fb51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 0x560a51fb5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f99b544fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51f0d5d0 con 0x560a51fb5790 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f99b544fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51f0aef0 con 0x560a51fb48a0 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f99b544fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51f067e0 con 0x560a51fb3f40 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f999affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 0x560a51fb4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f999affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 0x560a51fb5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f999affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 0x560a51fb5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54500/0 (socket says 172.21.15.3:54500) 2023-08-11T14:08:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.706+0000 7f999affd700 1 -- 172.21.15.3:0/3118823056 learned_addr learned my addr 172.21.15.3:0/3118823056 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.707+0000 7f999b7fe700 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fb51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.708+0000 7f999affd700 1 -- 172.21.15.3:0/3118823056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 msgr2=0x560a51fb51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.708+0000 7f999affd700 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fb51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.708+0000 7f999affd700 1 -- 172.21.15.3:0/3118823056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 msgr2=0x560a51fb5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.708+0000 7f999affd700 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 0x560a51fb5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.708+0000 7f999affd700 1 -- 172.21.15.3:0/3118823056 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a51d865d0 con 0x560a51fb3f40 2023-08-11T14:08:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.710+0000 7f9998ff9700 1 -- 172.21.15.3:0/1656496345 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9994005770 con 0x7f999c10cc60 2023-08-11T14:08:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.710+0000 7f999affd700 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 0x560a51fb4360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f999000bf00 tx=0x7f999000b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ef9c2d72312821 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.711+0000 7f9981ffb700 1 -- 172.21.15.3:0/3118823056 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999001e640 con 0x560a51fb3f40 2023-08-11T14:08:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.711+0000 7f9981ffb700 1 -- 172.21.15.3:0/3118823056 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999001ec60 con 0x560a51fb3f40 2023-08-11T14:08:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.711+0000 7f9981ffb700 1 -- 172.21.15.3:0/3118823056 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990010aa0 con 0x560a51fb3f40 2023-08-11T14:08:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.712+0000 7f99b544fb80 1 -- 172.21.15.3:0/3118823056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 msgr2=0x560a51fb4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.712+0000 7f99b544fb80 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 0x560a51fb4360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f999000bf00 tx=0x7f999000b650 comp rx=0 tx=0).stop 2023-08-11T14:08:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.713+0000 7f9998ff9700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 0x7f99840a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.713+0000 7f9998ff9700 1 -- 172.21.15.3:0/1656496345 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f99940cbdf0 con 0x7f999c10cc60 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.714+0000 7f99b544fb80 1 -- 172.21.15.3:0/3118823056 shutdown_connections 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.714+0000 7f99b544fb80 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb5790 0x560a51fb5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.714+0000 7f99b544fb80 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb3f40 0x560a51fb4360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.714+0000 7f99b544fb80 1 --2- 172.21.15.3:0/3118823056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fb51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.714+0000 7f99b544fb80 1 -- 172.21.15.3:0/3118823056 >> 172.21.15.3:0/3118823056 conn(0x560a51ee18f0 msgr2=0x560a51fb0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.715+0000 7f99b544fb80 1 -- 172.21.15.3:0/3118823056 shutdown_connections 2023-08-11T14:08:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.715+0000 7f99b544fb80 1 -- 172.21.15.3:0/3118823056 wait complete. 2023-08-11T14:08:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.715+0000 7f99b544fb80 1 -- start start 2023-08-11T14:08:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.716+0000 7f99b544fb80 1 Processor -- start 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51ea4a60 con 0x560a51fb3f40 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51ea4760 con 0x560a51fb48a0 2023-08-11T14:08:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51d888c0 con 0x560a51fb5790 2023-08-11T14:08:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.717+0000 7f99b544fb80 10 client.0 _ll_register_callbacks cb 0x560a51ea7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.718+0000 7f999affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.718+0000 7f999affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:36532/0 (socket says 172.21.15.3:36532) 2023-08-11T14:08:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.718+0000 7f999affd700 1 -- 172.21.15.3:0/2310170290 learned_addr learned my addr 172.21.15.3:0/2310170290 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.718+0000 7f999bfff700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 -- 172.21.15.3:0/2310170290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 msgr2=0x560a51fc5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 -- 172.21.15.3:0/2310170290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 msgr2=0x560a51fbdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 -- 172.21.15.3:0/2310170290 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99900123f0 con 0x560a51fb48a0 2023-08-11T14:08:14.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.719+0000 7f999affd700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f9990010480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6317d65bd7cddaf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.723+0000 7f999affd700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 0x7f99840a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.726+0000 7f997affd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.726+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990022050 con 0x560a51fb48a0 2023-08-11T14:08:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.726+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f999001e950 con 0x560a51fb48a0 2023-08-11T14:08:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.727+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a51fa77e0 con 0x560a51fb48a0 2023-08-11T14:08:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.727+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9990036020 con 0x560a51fb48a0 2023-08-11T14:08:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.728+0000 7f99b544fb80 10 client.26165 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:14.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.728+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560a51d32d80 con 0x560a51fb48a0 2023-08-11T14:08:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.732+0000 7f999affd700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 0x7f99840a2590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9990005a80 tx=0x7f9990033040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.732+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a51fa77e0 con 0x560a51fb48a0 2023-08-11T14:08:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.732+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f99900308a0 con 0x560a51fb48a0 2023-08-11T14:08:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 10 client.26165 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 20 client.26165 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a51fc99c0 0x560a51fcbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 4 client.26165 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:08:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.733+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a51f83480 con 0x560a51fc99c0 2023-08-11T14:08:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.734+0000 7f999b7fe700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a51fc99c0 0x560a51fcbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.736+0000 7f999b7fe700 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a51fc99c0 0x560a51fcbde0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba78677383f76d8 server_cookie=696b0ee6c193f681 in_seq=0 out_seq=0 2023-08-11T14:08:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.736+0000 7f997affd700 10 client.26165 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.747+0000 7f997affd700 1 -- 172.21.15.3:0/2310170290 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560a51f83480 con 0x560a51fc99c0 2023-08-11T14:08:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.747+0000 7f997affd700 10 client.26165 handle_command_reply: tid=0 2023-08-11T14:08:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.747+0000 7f99b544fb80 1 client.26165 shutdown 2023-08-11T14:08:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 msgr2=0x560a51fbe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f9990010480 comp rx=0 tx=0).stop 2023-08-11T14:08:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 shutdown_connections 2023-08-11T14:08:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a51fc99c0 0x560a51fcbde0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 --2- 172.21.15.3:0/2310170290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb48a0 0x560a51fbe1f0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f9990010480 comp rx=0 tx=0).stop 2023-08-11T14:08:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.750+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 >> 172.21.15.3:0/2310170290 conn(0x560a51ee18f0 msgr2=0x560a51faf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.755+0000 7f997affd700 0 client.26165 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.755+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 shutdown_connections 2023-08-11T14:08:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.757+0000 7f99b544fb80 1 -- 172.21.15.3:0/2310170290 wait complete. 2023-08-11T14:08:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.758+0000 7f99b544fb80 20 client.26165 trim_cache size 0 max 16384 2023-08-11T14:08:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.788+0000 7f99b544fb80 1 Processor -- start 2023-08-11T14:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.791+0000 7f99b544fb80 1 -- start start 2023-08-11T14:08:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fbe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fbebb0 con 0x560a51fb3f40 2023-08-11T14:08:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fbd200 con 0x560a51fc51f0 2023-08-11T14:08:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.792+0000 7f99b544fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fa77c0 con 0x560a51fb5790 2023-08-11T14:08:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.797+0000 7f999bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.797+0000 7f999bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:36538/0 (socket says 172.21.15.3:36538) 2023-08-11T14:08:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.797+0000 7f999bfff700 1 -- 172.21.15.3:0/1647573968 learned_addr learned my addr 172.21.15.3:0/1647573968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.797+0000 7f999b7fe700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.800+0000 7f999affd700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fbe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.804+0000 7f999bfff700 1 -- 172.21.15.3:0/1647573968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 msgr2=0x560a51fbe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999bfff700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fbe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999bfff700 1 -- 172.21.15.3:0/1647573968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 msgr2=0x560a51fbdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999bfff700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999bfff700 1 -- 172.21.15.3:0/1647573968 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a51d1bc00 con 0x560a51fc51f0 2023-08-11T14:08:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999affd700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fbe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:08:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.805+0000 7f999b7fe700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:08:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.809+0000 7f999bfff700 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9994099540 tx=0x7f999400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef629075226d6881 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.811+0000 7f99827fc700 1 -- 172.21.15.3:0/1647573968 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940995b0 con 0x560a51fc51f0 2023-08-11T14:08:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.811+0000 7f99827fc700 1 -- 172.21.15.3:0/1647573968 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999409d3e0 con 0x560a51fc51f0 2023-08-11T14:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.811+0000 7f99827fc700 1 -- 172.21.15.3:0/1647573968 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994020da0 con 0x560a51fc51f0 2023-08-11T14:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.813+0000 7f99b544fb80 1 -- 172.21.15.3:0/1647573968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 msgr2=0x560a51fb8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.813+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9994099540 tx=0x7f999400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.815+0000 7f99b544fb80 1 -- 172.21.15.3:0/1647573968 shutdown_connections 2023-08-11T14:08:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.815+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fc51f0 0x560a51fb8760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.815+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fb3f40 0x560a51fbdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.815+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1647573968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fbe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.815+0000 7f99b544fb80 1 -- 172.21.15.3:0/1647573968 >> 172.21.15.3:0/1647573968 conn(0x560a51ee18f0 msgr2=0x560a51fc5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.816+0000 7f99b544fb80 1 -- 172.21.15.3:0/1647573968 shutdown_connections 2023-08-11T14:08:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.818+0000 7f99b544fb80 1 -- 172.21.15.3:0/1647573968 wait complete. 2023-08-11T14:08:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.818+0000 7f99b544fb80 1 -- start start 2023-08-11T14:08:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.819+0000 7f99b544fb80 1 Processor -- start 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb3f40 0x560a51fc4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fc2fb0 con 0x560a51fc51f0 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fc30f0 con 0x560a51fb3f40 2023-08-11T14:08:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a51fc32b0 con 0x560a51fb5790 2023-08-11T14:08:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.823+0000 7f99b544fb80 10 client.0 _ll_register_callbacks cb 0x7f998c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.824+0000 7f999bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.824+0000 7f999bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54538/0 (socket says 172.21.15.3:54538) 2023-08-11T14:08:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.824+0000 7f999bfff700 1 -- 172.21.15.3:0/1425872015 learned_addr learned my addr 172.21.15.3:0/1425872015 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.825+0000 7f999bfff700 1 -- 172.21.15.3:0/1425872015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 msgr2=0x560a51fc49a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:08:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.825+0000 7f999bfff700 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc49a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.825+0000 7f999bfff700 1 -- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb3f40 msgr2=0x560a51fc4460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:08:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.825+0000 7f999bfff700 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb3f40 0x560a51fc4460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.825+0000 7f999bfff700 1 -- 172.21.15.3:0/1425872015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a51d32d80 con 0x560a51fc51f0 2023-08-11T14:08:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.826+0000 7f999bfff700 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99940a0e00 tx=0x7f9994009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa1bf7917c006c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.831+0000 7f997affd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:08:14.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.831+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940b72a0 con 0x560a51fc51f0 2023-08-11T14:08:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.831+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9994097950 con 0x560a51fc51f0 2023-08-11T14:08:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.833+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f999001ec60 con 0x560a51fc51f0 2023-08-11T14:08:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.843+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99940acbe0 con 0x560a51fc51f0 2023-08-11T14:08:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.843+0000 7f99b544fb80 10 client.16089 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.843+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f999c1b6600 con 0x560a51fc51f0 2023-08-11T14:08:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.847+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999001ec60 con 0x560a51fc51f0 2023-08-11T14:08:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.847+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9994099810 con 0x560a51fc51f0 2023-08-11T14:08:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 20 client.16089 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a5201e450 0x560a52020840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 4 client.16089 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:08:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.848+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a51eff7c0 con 0x560a5201e450 2023-08-11T14:08:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.857+0000 7f999b7fe700 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a5201e450 0x560a52020840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.859+0000 7f999b7fe700 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a5201e450 0x560a52020840 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7854fc8345ca0c30 server_cookie=61fac01722c8f865 in_seq=0 out_seq=0 2023-08-11T14:08:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:14.859+0000 7f997affd700 10 client.16089 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:14.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:14 smithi103 ceph-mon[130434]: pgmap v477: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.37k op/s 2023-08-11T14:08:15.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:14 smithi187 ceph-mon[130973]: pgmap v477: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.37k op/s 2023-08-11T14:08:15.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:14 smithi003 ceph-mon[97195]: pgmap v477: 129 pgs: 129 active+clean; 8.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 1.37k op/s 2023-08-11T14:08:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.045+0000 7f997affd700 1 -- 172.21.15.3:0/1425872015 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a51eff7c0 con 0x560a5201e450 2023-08-11T14:08:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.045+0000 7f997affd700 10 client.16089 handle_command_reply: tid=0 2023-08-11T14:08:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.045+0000 7f99b544fb80 1 client.16089 shutdown 2023-08-11T14:08:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 msgr2=0x560a51fc2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f99940a0e00 tx=0x7f9994009640 comp rx=0 tx=0).stop 2023-08-11T14:08:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 shutdown_connections 2023-08-11T14:08:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560a51fc51f0 0x560a51fc2a70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560a5201e450 0x560a52020840 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a51fb3f40 0x560a51fc4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 --2- 172.21.15.3:0/1425872015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a51fb5790 0x560a51fc49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.047+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 >> 172.21.15.3:0/1425872015 conn(0x560a51ee18f0 msgr2=0x560a51fb8760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.048+0000 7f997affd700 0 client.16089 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.049+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 shutdown_connections 2023-08-11T14:08:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.049+0000 7f99b544fb80 1 -- 172.21.15.3:0/1425872015 wait complete. 2023-08-11T14:08:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.049+0000 7f99b544fb80 20 client.16089 trim_cache size 0 max 16384 2023-08-11T14:08:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:08:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:08:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:08:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:08:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.052+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 msgr2=0x7f99840a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.052+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 0x7f99840a2590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9990005a80 tx=0x7f9990033040 comp rx=0 tx=0).stop 2023-08-11T14:08:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.052+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 msgr2=0x7f999c116cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.052+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9994009300 tx=0x7f9994009330 comp rx=0 tx=0).stop 2023-08-11T14:08:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 shutdown_connections 2023-08-11T14:08:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f99840a00d0 0x7f99840a2590 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f999c10cc60 0x7f999c116cd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f999c107920 0x7f999c118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 --2- 172.21.15.3:0/1656496345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f999c10a290 0x7f999c1188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.056+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 >> 172.21.15.3:0/1656496345 conn(0x7f999c06d750 msgr2=0x7f999c10aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.064+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 shutdown_connections 2023-08-11T14:08:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:15.070+0000 7f99a1bff700 1 -- 172.21.15.3:0/1656496345 wait complete. 2023-08-11T14:08:17.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:17 smithi103 ceph-mon[130434]: pgmap v478: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-08-11T14:08:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:17 smithi003 ceph-mon[97195]: pgmap v478: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-08-11T14:08:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:17 smithi187 ceph-mon[130973]: pgmap v478: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-08-11T14:08:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 80.0 s, 263.8 tps, lat 316.936 ms stddev 431.339 2023-08-11T14:08:19.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:19 smithi103 ceph-mon[130434]: pgmap v479: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.90k op/s 2023-08-11T14:08:20.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:19 smithi003 ceph-mon[97195]: pgmap v479: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.90k op/s 2023-08-11T14:08:20.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:19 smithi187 ceph-mon[130973]: pgmap v479: 129 pgs: 129 active+clean; 8.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 2.90k op/s 2023-08-11T14:08:21.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:21 smithi103 ceph-mon[130434]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 3.19k op/s 2023-08-11T14:08:22.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:21 smithi187 ceph-mon[130973]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 3.19k op/s 2023-08-11T14:08:22.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:21 smithi003 ceph-mon[97195]: pgmap v480: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 3.19k op/s 2023-08-11T14:08:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 85.0 s, 1862.9 tps, lat 61.882 ms stddev 127.522 2023-08-11T14:08:23.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:23 smithi103 ceph-mon[130434]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 3.08k op/s 2023-08-11T14:08:24.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:23 smithi003 ceph-mon[97195]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 3.08k op/s 2023-08-11T14:08:24.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:23 smithi187 ceph-mon[130973]: pgmap v481: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 3.08k op/s 2023-08-11T14:08:25.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:25 smithi103 ceph-mon[130434]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-08-11T14:08:26.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:25 smithi003 ceph-mon[97195]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-08-11T14:08:26.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:25 smithi187 ceph-mon[130973]: pgmap v482: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 3.19k op/s 2023-08-11T14:08:26.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:26 smithi103 ceph-mon[130434]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.98k op/s 2023-08-11T14:08:27.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:26 smithi187 ceph-mon[130973]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.98k op/s 2023-08-11T14:08:27.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:26 smithi003 ceph-mon[97195]: pgmap v483: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 43 MiB/s wr, 2.98k op/s 2023-08-11T14:08:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 90.0 s, 1638.6 tps, lat 61.781 ms stddev 128.734 2023-08-11T14:08:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.391+0000 7ffa617ff700 1 -- 172.21.15.3:0/884260493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 msgr2=0x7ffa5c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.391+0000 7ffa617ff700 1 --2- 172.21.15.3:0/884260493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c10d0e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b0a0 tx=0x7ffa5400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.392+0000 7ffa617ff700 1 -- 172.21.15.3:0/884260493 shutdown_connections 2023-08-11T14:08:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.392+0000 7ffa617ff700 1 --2- 172.21.15.3:0/884260493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c10d0e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.392+0000 7ffa617ff700 1 --2- 172.21.15.3:0/884260493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c107920 0x7ffa5c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.392+0000 7ffa617ff700 1 --2- 172.21.15.3:0/884260493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c10a290 0x7ffa5c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.392+0000 7ffa617ff700 1 -- 172.21.15.3:0/884260493 >> 172.21.15.3:0/884260493 conn(0x7ffa5c06d750 msgr2=0x7ffa5c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.398+0000 7ffa617ff700 1 -- 172.21.15.3:0/884260493 shutdown_connections 2023-08-11T14:08:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.399+0000 7ffa617ff700 1 -- 172.21.15.3:0/884260493 wait complete. 2023-08-11T14:08:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.400+0000 7ffa617ff700 1 Processor -- start 2023-08-11T14:08:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.407+0000 7ffa617ff700 1 -- start start 2023-08-11T14:08:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.407+0000 7ffa617ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c107920 0x7ffa5c1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.408+0000 7ffa617ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c10a290 0x7ffa5c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.408+0000 7ffa617ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.408+0000 7ffa617ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a7010 con 0x7ffa5c10a290 2023-08-11T14:08:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.408+0000 7ffa617ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a7180 con 0x7ffa5c107920 2023-08-11T14:08:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.408+0000 7ffa617ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c1a7340 con 0x7ffa5c10cc60 2023-08-11T14:08:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58368/0 (socket says 172.21.15.3:58368) 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/871761070 learned_addr learned my addr 172.21.15.3:0/871761070 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/871761070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c107920 msgr2=0x7ffa5c1a55f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c107920 0x7ffa5c1a55f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c10a290 msgr2=0x7ffa5c1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c10a290 0x7ffa5c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.409+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/871761070 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009d00 con 0x7ffa5c10cc60 2023-08-11T14:08:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.410+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa540062a0 tx=0x7ffa54004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.412+0000 7ffa43fff700 1 -- 172.21.15.3:0/871761070 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400e040 con 0x7ffa5c10cc60 2023-08-11T14:08:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.412+0000 7ffa43fff700 1 -- 172.21.15.3:0/871761070 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa54009440 con 0x7ffa5c10cc60 2023-08-11T14:08:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.412+0000 7ffa43fff700 1 -- 172.21.15.3:0/871761070 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004020 con 0x7ffa5c10cc60 2023-08-11T14:08:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.413+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c1a75c0 con 0x7ffa5c10cc60 2023-08-11T14:08:28.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.413+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c1adde0 con 0x7ffa5c10cc60 2023-08-11T14:08:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.424+0000 7ffa75080b80 1 Processor -- start 2023-08-11T14:08:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.424+0000 7ffa75080b80 1 -- start start 2023-08-11T14:08:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cb685d0 con 0x55663cc208b0 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cb65ef0 con 0x55663cc1ff50 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa75080b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cb617e0 con 0x55663cc217a0 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa43fff700 1 -- 172.21.15.3:0/871761070 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ffa54004180 con 0x7ffa5c10cc60 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.425+0000 7ffa5a7fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.426+0000 7ffa5a7fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41784/0 (socket says 172.21.15.3:41784) 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.426+0000 7ffa5a7fc700 1 -- 172.21.15.3:0/4187421237 learned_addr learned my addr 172.21.15.3:0/4187421237 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.426+0000 7ffa5a7fc700 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa5a7fc700 1 -- 172.21.15.3:0/4187421237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 msgr2=0x55663cc21c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa5a7fc700 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc21c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa5a7fc700 1 -- 172.21.15.3:0/4187421237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 msgr2=0x55663cc211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa5a7fc700 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa5a7fc700 1 -- 172.21.15.3:0/4187421237 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663c9e15d0 con 0x55663cc1ff50 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.427+0000 7ffa43fff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 0x7ffa440a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.428+0000 7ffa43fff700 1 -- 172.21.15.3:0/871761070 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffa540cccc0 con 0x7ffa5c10cc60 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.428+0000 7ffa5a7fc700 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55663cc20f80 tx=0x7ffa4c02c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15edd5070af3ff6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.429+0000 7ffa5affd700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 0x7ffa440a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.430+0000 7ffa417fa700 1 -- 172.21.15.3:0/4187421237 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c004220 con 0x55663cc1ff50 2023-08-11T14:08:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.430+0000 7ffa417fa700 1 -- 172.21.15.3:0/4187421237 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4c004380 con 0x55663cc1ff50 2023-08-11T14:08:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.430+0000 7ffa417fa700 1 -- 172.21.15.3:0/4187421237 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c032910 con 0x55663cc1ff50 2023-08-11T14:08:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.434+0000 7ffa5affd700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 0x7ffa440a2610 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa50005950 tx=0x7ffa500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.435+0000 7ffa75080b80 1 -- 172.21.15.3:0/4187421237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 msgr2=0x55663cc20370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.435+0000 7ffa75080b80 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55663cc20f80 tx=0x7ffa4c02c2d0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 -- 172.21.15.3:0/4187421237 shutdown_connections 2023-08-11T14:08:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc20370 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 --2- 172.21.15.3:0/4187421237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 -- 172.21.15.3:0/4187421237 >> 172.21.15.3:0/4187421237 conn(0x55663cb3c8f0 msgr2=0x55663cc0b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.437+0000 7ffa75080b80 1 -- 172.21.15.3:0/4187421237 shutdown_connections 2023-08-11T14:08:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.438+0000 7ffa75080b80 1 -- 172.21.15.3:0/4187421237 wait complete. 2023-08-11T14:08:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.438+0000 7ffa75080b80 1 -- start start 2023-08-11T14:08:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 Processor -- start 2023-08-11T14:08:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663caffa60 con 0x55663cc1ff50 2023-08-11T14:08:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663caff760 con 0x55663cc217a0 2023-08-11T14:08:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663c9e38c0 con 0x55663cc208b0 2023-08-11T14:08:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.439+0000 7ffa75080b80 10 client.0 _ll_register_callbacks cb 0x55663cb02a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.440+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.440+0000 7ffa5a7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.440+0000 7ffa5a7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58392/0 (socket says 172.21.15.3:58392) 2023-08-11T14:08:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.440+0000 7ffa5a7fc700 1 -- 172.21.15.3:0/3994372071 learned_addr learned my addr 172.21.15.3:0/3994372071 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.441+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/3994372071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 msgr2=0x55663cc10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.441+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.441+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/3994372071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc1ff50 msgr2=0x55663cc10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.441+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.441+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/3994372071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c0354d0 con 0x55663cc217a0 2023-08-11T14:08:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.445+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc177a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b830 tx=0x7ffa5409af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62938f1b80e41dcf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.447+0000 7ffa3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.447+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54025e50 con 0x55663cc217a0 2023-08-11T14:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.447+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffa5409b390 con 0x55663cc217a0 2023-08-11T14:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.447+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55663cc027e0 con 0x55663cc217a0 2023-08-11T14:08:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.447+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa5409f1c0 con 0x55663cc217a0 2023-08-11T14:08:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.451+0000 7ffa75080b80 10 client.26174 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.451+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55663c98dd80 con 0x55663cc217a0 2023-08-11T14:08:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.452+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55663cc027e0 con 0x55663cc217a0 2023-08-11T14:08:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.452+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa5409b5f0 con 0x55663cc217a0 2023-08-11T14:08:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 10 client.26174 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 20 client.26174 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc1be00 0x55663cc1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 4 client.26174 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:08:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.453+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55663cbde480 con 0x55663cc1be00 2023-08-11T14:08:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.470+0000 7ffa5affd700 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc1be00 0x55663cc1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.472+0000 7ffa5affd700 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc1be00 0x55663cc1e220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8fcd09637dd034 server_cookie=32e1207d68b45444 in_seq=0 out_seq=0 2023-08-11T14:08:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.472+0000 7ffa3eff5700 10 client.26174 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.475+0000 7ffa3eff5700 1 -- 172.21.15.3:0/3994372071 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55663cbde480 con 0x55663cc1be00 2023-08-11T14:08:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.475+0000 7ffa3eff5700 10 client.26174 handle_command_reply: tid=0 2023-08-11T14:08:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.475+0000 7ffa75080b80 1 client.26174 shutdown 2023-08-11T14:08:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.476+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 msgr2=0x55663cc177a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.476+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc177a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b830 tx=0x7ffa5409af50 comp rx=0 tx=0).stop 2023-08-11T14:08:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 shutdown_connections 2023-08-11T14:08:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc1be00 0x55663cc1e220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc177a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 --2- 172.21.15.3:0/3994372071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 >> 172.21.15.3:0/3994372071 conn(0x55663cb3c8f0 msgr2=0x55663cc0a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.477+0000 7ffa3eff5700 0 client.26174 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.478+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 shutdown_connections 2023-08-11T14:08:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.478+0000 7ffa75080b80 1 -- 172.21.15.3:0/3994372071 wait complete. 2023-08-11T14:08:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.479+0000 7ffa75080b80 20 client.26174 trim_cache size 0 max 16384 2023-08-11T14:08:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.509+0000 7ffa75080b80 1 Processor -- start 2023-08-11T14:08:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.509+0000 7ffa75080b80 1 -- start start 2023-08-11T14:08:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc0ec50 con 0x55663cc208b0 2023-08-11T14:08:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc027c0 con 0x55663cc217a0 2023-08-11T14:08:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.510+0000 7ffa75080b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc2cb60 con 0x55663cc1ff50 2023-08-11T14:08:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.511+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.511+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41798/0 (socket says 172.21.15.3:41798) 2023-08-11T14:08:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.511+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/2927317840 learned_addr learned my addr 172.21.15.3:0/2927317840 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.511+0000 7ffa5a7fc700 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5affd700 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/2927317840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc1ff50 msgr2=0x55663cc10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/2927317840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 msgr2=0x55663cc10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.512+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/2927317840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663c976c00 con 0x55663cc217a0 2023-08-11T14:08:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.518+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409b320 tx=0x7ffa5400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a370ea2186758706 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.524+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2927317840 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540b34e0 con 0x55663cc217a0 2023-08-11T14:08:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.524+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2927317840 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa5409b390 con 0x55663cc217a0 2023-08-11T14:08:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.524+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2927317840 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5409a950 con 0x55663cc217a0 2023-08-11T14:08:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.525+0000 7ffa75080b80 1 -- 172.21.15.3:0/2927317840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 msgr2=0x55663cc17730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.525+0000 7ffa75080b80 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409b320 tx=0x7ffa5400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.526+0000 7ffa75080b80 1 -- 172.21.15.3:0/2927317840 shutdown_connections 2023-08-11T14:08:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.526+0000 7ffa75080b80 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc217a0 0x55663cc17730 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.526+0000 7ffa75080b80 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc1ff50 0x55663cc10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.526+0000 7ffa75080b80 1 --2- 172.21.15.3:0/2927317840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.526+0000 7ffa75080b80 1 -- 172.21.15.3:0/2927317840 >> 172.21.15.3:0/2927317840 conn(0x55663cb3c8f0 msgr2=0x55663cc0ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.537+0000 7ffa75080b80 1 -- 172.21.15.3:0/2927317840 shutdown_connections 2023-08-11T14:08:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.538+0000 7ffa75080b80 1 -- 172.21.15.3:0/2927317840 wait complete. 2023-08-11T14:08:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.538+0000 7ffa75080b80 1 -- start start 2023-08-11T14:08:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 Processor -- start 2023-08-11T14:08:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc17840 con 0x55663cc208b0 2023-08-11T14:08:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc179b0 con 0x55663cc1ff50 2023-08-11T14:08:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663cc77b10 con 0x55663cc217a0 2023-08-11T14:08:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.539+0000 7ffa75080b80 10 client.0 _ll_register_callbacks cb 0x7ffa540976d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.540+0000 7ffa5affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.540+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.540+0000 7ffa5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58410/0 (socket says 172.21.15.3:58410) 2023-08-11T14:08:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.541+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/320529301 learned_addr learned my addr 172.21.15.3:0/320529301 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.545+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 msgr2=0x55663cc122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.545+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.545+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/320529301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 msgr2=0x55663cc12810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.545+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.545+0000 7ffa5b7fe700 1 -- 172.21.15.3:0/320529301 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663c98dd80 con 0x55663cc217a0 2023-08-11T14:08:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.561+0000 7ffa5b7fe700 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409b090 tx=0x7ffa5409bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52455028400f4032 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.562+0000 7ffa3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:08:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.573+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54025e50 con 0x55663cc217a0 2023-08-11T14:08:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.573+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffa5409f1c0 con 0x55663cc217a0 2023-08-11T14:08:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.573+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa4c004380 con 0x55663cc217a0 2023-08-11T14:08:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.589+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa540a2c80 con 0x55663cc217a0 2023-08-11T14:08:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.590+0000 7ffa75080b80 10 client.26158 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.590+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55663c9b41f0 con 0x55663cc217a0 2023-08-11T14:08:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.594+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4c004380 con 0x55663cc217a0 2023-08-11T14:08:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.594+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffa5409a2f0 con 0x55663cc217a0 2023-08-11T14:08:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 10 client.26158 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 20 client.26158 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc78c20 0x55663cc7b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 4 client.26158 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:08:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.600+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55663cb5a7c0 con 0x55663cc78c20 2023-08-11T14:08:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.601+0000 7ffa5affd700 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc78c20 0x55663cc7b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.612+0000 7ffa5affd700 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc78c20 0x55663cc7b010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88480dcfc3e783d0 server_cookie=b1deca1aebd50072 in_seq=0 out_seq=0 2023-08-11T14:08:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.613+0000 7ffa3eff5700 10 client.26158 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.704+0000 7ffa3eff5700 1 -- 172.21.15.3:0/320529301 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55663cb5a7c0 con 0x55663cc78c20 2023-08-11T14:08:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.704+0000 7ffa3eff5700 10 client.26158 handle_command_reply: tid=0 2023-08-11T14:08:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.704+0000 7ffa75080b80 1 client.26158 shutdown 2023-08-11T14:08:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 msgr2=0x55663cc775d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409b090 tx=0x7ffa5409bc00 comp rx=0 tx=0).stop 2023-08-11T14:08:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 shutdown_connections 2023-08-11T14:08:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55663cc217a0 0x55663cc775d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55663cc78c20 0x55663cc7b010 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55663cc1ff50 0x55663cc122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 --2- 172.21.15.3:0/320529301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55663cc208b0 0x55663cc12810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 >> 172.21.15.3:0/320529301 conn(0x55663cb3c8f0 msgr2=0x55663cc0fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.705+0000 7ffa3eff5700 0 client.26158 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.708+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 shutdown_connections 2023-08-11T14:08:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.708+0000 7ffa75080b80 1 -- 172.21.15.3:0/320529301 wait complete. 2023-08-11T14:08:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:08:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:08:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:08:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:08:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.715+0000 7ffa75080b80 20 client.26158 trim_cache size 0 max 16384 2023-08-11T14:08:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.716+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 msgr2=0x7ffa440a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.716+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 0x7ffa440a2610 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa50005950 tx=0x7ffa500058e0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.716+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 msgr2=0x7ffa5c1a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.716+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa540062a0 tx=0x7ffa54004d30 comp rx=0 tx=0).stop 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 shutdown_connections 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5c10cc60 0x7ffa5c1a69b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa440a0150 0x7ffa440a2610 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c107920 0x7ffa5c1a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 --2- 172.21.15.3:0/871761070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c10a290 0x7ffa5c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.717+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 >> 172.21.15.3:0/871761070 conn(0x7ffa5c06d750 msgr2=0x7ffa5c10bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.722+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 shutdown_connections 2023-08-11T14:08:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:28.724+0000 7ffa617ff700 1 -- 172.21.15.3:0/871761070 wait complete. 2023-08-11T14:08:29.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:29 smithi103 ceph-mon[130434]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 43 MiB/s wr, 2.64k op/s 2023-08-11T14:08:30.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:29 smithi003 ceph-mon[97195]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 43 MiB/s wr, 2.64k op/s 2023-08-11T14:08:30.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:29 smithi187 ceph-mon[130973]: pgmap v484: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 43 MiB/s wr, 2.64k op/s 2023-08-11T14:08:30.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:30 smithi103 ceph-mon[130434]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.62k op/s 2023-08-11T14:08:31.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:30 smithi003 ceph-mon[97195]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.62k op/s 2023-08-11T14:08:31.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:30 smithi187 ceph-mon[130973]: pgmap v485: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.62k op/s 2023-08-11T14:08:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 95.0 s, 1897.2 tps, lat 52.730 ms stddev 38.664 2023-08-11T14:08:33.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:33 smithi103 ceph-mon[130434]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 31 MiB/s wr, 1.06k op/s 2023-08-11T14:08:34.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:33 smithi003 ceph-mon[97195]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 31 MiB/s wr, 1.06k op/s 2023-08-11T14:08:34.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:33 smithi187 ceph-mon[130973]: pgmap v486: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 31 MiB/s wr, 1.06k op/s 2023-08-11T14:08:35.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:35 smithi103 ceph-mon[130434]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 35 MiB/s wr, 1.01k op/s 2023-08-11T14:08:36.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:35 smithi003 ceph-mon[97195]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 35 MiB/s wr, 1.01k op/s 2023-08-11T14:08:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:35 smithi187 ceph-mon[130973]: pgmap v487: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 35 MiB/s wr, 1.01k op/s 2023-08-11T14:08:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:36 smithi103 ceph-mon[130434]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 1.40k op/s 2023-08-11T14:08:37.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:36 smithi003 ceph-mon[97195]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 1.40k op/s 2023-08-11T14:08:37.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:36 smithi187 ceph-mon[130973]: pgmap v488: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 35 MiB/s wr, 1.40k op/s 2023-08-11T14:08:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 100.0 s, 411.0 tps, lat 212.041 ms stddev 384.165 2023-08-11T14:08:39.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:39 smithi103 ceph-mon[130434]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-08-11T14:08:40.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:39 smithi003 ceph-mon[97195]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-08-11T14:08:40.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:39 smithi187 ceph-mon[130973]: pgmap v489: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 2.17k op/s 2023-08-11T14:08:41.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:40 smithi003 ceph-mon[97195]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 MiB/s wr, 2.65k op/s 2023-08-11T14:08:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:40 smithi187 ceph-mon[130973]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 MiB/s wr, 2.65k op/s 2023-08-11T14:08:41.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:40 smithi103 ceph-mon[130434]: pgmap v490: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 MiB/s wr, 2.65k op/s 2023-08-11T14:08:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.050+0000 7fa6671ff700 1 -- 172.21.15.3:0/4033248241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600ff190 msgr2=0x7fa6601015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.050+0000 7fa6671ff700 1 --2- 172.21.15.3:0/4033248241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600ff190 0x7fa6601015c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa654009990 tx=0x7fa654009ca0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.054+0000 7fa6671ff700 1 -- 172.21.15.3:0/4033248241 shutdown_connections 2023-08-11T14:08:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.054+0000 7fa6671ff700 1 --2- 172.21.15.3:0/4033248241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6601044d0 0x7fa660104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.054+0000 7fa6671ff700 1 --2- 172.21.15.3:0/4033248241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6600ff190 0x7fa6601015c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.054+0000 7fa6671ff700 1 --2- 172.21.15.3:0/4033248241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa660101b00 0x7fa660103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.054+0000 7fa6671ff700 1 -- 172.21.15.3:0/4033248241 >> 172.21.15.3:0/4033248241 conn(0x7fa6600fad70 msgr2=0x7fa6600fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.055+0000 7fa6671ff700 1 -- 172.21.15.3:0/4033248241 shutdown_connections 2023-08-11T14:08:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.055+0000 7fa6671ff700 1 -- 172.21.15.3:0/4033248241 wait complete. 2023-08-11T14:08:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.056+0000 7fa6671ff700 1 Processor -- start 2023-08-11T14:08:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.056+0000 7fa6671ff700 1 -- start start 2023-08-11T14:08:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6600ff190 0x7fa66010a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa660101b00 0x7fa66010aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010c0d0 con 0x7fa6600ff190 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010c240 con 0x7fa660101b00 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa6671ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010c400 con 0x7fa6601044d0 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa66579c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa66579c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50754/0 (socket says 172.21.15.3:50754) 2023-08-11T14:08:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.057+0000 7fa66579c700 1 -- 172.21.15.3:0/3363157303 learned_addr learned my addr 172.21.15.3:0/3363157303 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.058+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6600ff190 0x7fa66010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.058+0000 7fa66579c700 1 -- 172.21.15.3:0/3363157303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa660101b00 msgr2=0x7fa66010aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.058+0000 7fa66579c700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa660101b00 0x7fa66010aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.058+0000 7fa66579c700 1 -- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6600ff190 msgr2=0x7fa66010a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.058+0000 7fa66579c700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6600ff190 0x7fa66010a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.059+0000 7fa66579c700 1 -- 172.21.15.3:0/3363157303 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540095f0 con 0x7fa6601044d0 2023-08-11T14:08:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.059+0000 7fa66579c700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b780 tx=0x7fa65800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.060+0000 7fa65dffb700 1 -- 172.21.15.3:0/3363157303 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800f840 con 0x7fa6601044d0 2023-08-11T14:08:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.060+0000 7fa65dffb700 1 -- 172.21.15.3:0/3363157303 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa65800fe60 con 0x7fa6601044d0 2023-08-11T14:08:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.060+0000 7fa65dffb700 1 -- 172.21.15.3:0/3363157303 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800d3d0 con 0x7fa6601044d0 2023-08-11T14:08:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.061+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66010c6e0 con 0x7fa6601044d0 2023-08-11T14:08:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.061+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66010cf10 con 0x7fa6601044d0 2023-08-11T14:08:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.073+0000 7fa67aa2bb80 1 Processor -- start 2023-08-11T14:08:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.073+0000 7fa67aa2bb80 1 -- start start 2023-08-11T14:08:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615365d0 con 0x55bc615dcf40 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc61533ef0 con 0x55bc615dd8a0 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.074+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc6152f7e0 con 0x55bc615de790 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.076+0000 7fa664f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.076+0000 7fa664f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615de1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52322/0 (socket says 172.21.15.3:52322) 2023-08-11T14:08:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.076+0000 7fa664f9b700 1 -- 172.21.15.3:0/260176098 learned_addr learned my addr 172.21.15.3:0/260176098 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.076+0000 7fa65ffff700 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.077+0000 7fa65dffb700 1 -- 172.21.15.3:0/3363157303 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa65800f9a0 con 0x7fa6601044d0 2023-08-11T14:08:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.079+0000 7fa65ffff700 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.079+0000 7fa65ffff700 1 -- 172.21.15.3:0/260176098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 msgr2=0x55bc615de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.080+0000 7fa65ffff700 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.080+0000 7fa65ffff700 1 -- 172.21.15.3:0/260176098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 msgr2=0x55bc615dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.080+0000 7fa65ffff700 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.080+0000 7fa65ffff700 1 -- 172.21.15.3:0/260176098 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc613af5d0 con 0x55bc615de790 2023-08-11T14:08:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.082+0000 7fa65ffff700 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615dec10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00bf90 tx=0x7fa64c00b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1bb9b5a60e4bab server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.083+0000 7fa64affd700 1 -- 172.21.15.3:0/260176098 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004230 con 0x55bc615de790 2023-08-11T14:08:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.083+0000 7fa65dffb700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 0x7fa6500a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.083+0000 7fa64affd700 1 -- 172.21.15.3:0/260176098 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64c00de10 con 0x55bc615de790 2023-08-11T14:08:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.083+0000 7fa64affd700 1 -- 172.21.15.3:0/260176098 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c006630 con 0x55bc615de790 2023-08-11T14:08:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.083+0000 7fa65dffb700 1 -- 172.21.15.3:0/3363157303 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa6580cb7b0 con 0x7fa6601044d0 2023-08-11T14:08:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 0x7fa6500a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/260176098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 msgr2=0x55bc615dec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615dec10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00bf90 tx=0x7fa64c00b650 comp rx=0 tx=0).stop 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/260176098 shutdown_connections 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615dec10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/260176098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.084+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/260176098 >> 172.21.15.3:0/260176098 conn(0x55bc6150a8f0 msgr2=0x55bc615d9920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.085+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/260176098 shutdown_connections 2023-08-11T14:08:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.085+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/260176098 wait complete. 2023-08-11T14:08:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.085+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 0x7fa6500a2610 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55bc615d92f0 tx=0x7fa654003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.085+0000 7fa67aa2bb80 1 -- start start 2023-08-11T14:08:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.086+0000 7fa67aa2bb80 1 Processor -- start 2023-08-11T14:08:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.089+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dcf40 0x55bc615e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.089+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.089+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615de790 0x55bc615ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.089+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc614cda60 con 0x55bc615de790 2023-08-11T14:08:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.089+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc614cd760 con 0x55bc615dd8a0 2023-08-11T14:08:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc613b18c0 con 0x55bc615dcf40 2023-08-11T14:08:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa67aa2bb80 10 client.0 _ll_register_callbacks cb 0x55bc614d0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa66579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615de790 0x55bc615ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52332/0 (socket says 172.21.15.3:52332) 2023-08-11T14:08:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa65ffff700 1 -- 172.21.15.3:0/1719454339 learned_addr learned my addr 172.21.15.3:0/1719454339 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.090+0000 7fa664f9b700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dcf40 0x55bc615e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 -- 172.21.15.3:0/1719454339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dcf40 msgr2=0x55bc615e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dcf40 0x55bc615e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 -- 172.21.15.3:0/1719454339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615de790 msgr2=0x55bc615ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615de790 0x55bc615ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 -- 172.21.15.3:0/1719454339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c00bcc0 con 0x55bc615dd8a0 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa64c006110 tx=0x7fa64c0065e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a14a4b35a67f4d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.091+0000 7fa63ffff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00b390 con 0x55bc615dd8a0 2023-08-11T14:08:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa64c000e40 con 0x55bc615dd8a0 2023-08-11T14:08:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc615d07e0 con 0x55bc615dd8a0 2023-08-11T14:08:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64c004b20 con 0x55bc615dd8a0 2023-08-11T14:08:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa67aa2bb80 10 client.26186 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bc6135bd80 con 0x55bc615dd8a0 2023-08-11T14:08:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.092+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc615d07e0 con 0x55bc615dd8a0 2023-08-11T14:08:42.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa64c00a070 con 0x55bc615dd8a0 2023-08-11T14:08:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 10 client.26186 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 20 client.26186 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc61603a00 0x55bc61605e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 4 client.26186 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:08:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc615ac480 con 0x55bc61603a00 2023-08-11T14:08:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.093+0000 7fa664f9b700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc61603a00 0x55bc61605e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.094+0000 7fa664f9b700 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc61603a00 0x55bc61605e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787f489a76f3d001 server_cookie=d8357c8e5e365af5 in_seq=0 out_seq=0 2023-08-11T14:08:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.094+0000 7fa63ffff700 10 client.26186 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa63ffff700 1 -- 172.21.15.3:0/1719454339 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55bc615ac480 con 0x55bc61603a00 2023-08-11T14:08:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa63ffff700 10 client.26186 handle_command_reply: tid=0 2023-08-11T14:08:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 client.26186 shutdown 2023-08-11T14:08:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 msgr2=0x55bc615e71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa64c006110 tx=0x7fa64c0065e0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 shutdown_connections 2023-08-11T14:08:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615de790 0x55bc615ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dcf40 0x55bc615e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc61603a00 0x55bc61605e20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1719454339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615e71c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.095+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 >> 172.21.15.3:0/1719454339 conn(0x55bc6150a8f0 msgr2=0x55bc615d86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.096+0000 7fa63ffff700 0 client.26186 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.096+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 shutdown_connections 2023-08-11T14:08:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.096+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1719454339 wait complete. 2023-08-11T14:08:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.096+0000 7fa67aa2bb80 20 client.26186 trim_cache size 0 max 16384 2023-08-11T14:08:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.120+0000 7fa67aa2bb80 1 Processor -- start 2023-08-11T14:08:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 -- start start 2023-08-11T14:08:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615d2f20 con 0x55bc615dcf40 2023-08-11T14:08:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615d8d80 con 0x55bc615dd8a0 2023-08-11T14:08:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.123+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615e4960 con 0x55bc615de790 2023-08-11T14:08:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52340/0 (socket says 172.21.15.3:52340) 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 -- 172.21.15.3:0/1139252171 learned_addr learned my addr 172.21.15.3:0/1139252171 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa66579c700 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 -- 172.21.15.3:0/1139252171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 msgr2=0x55bc615e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 -- 172.21.15.3:0/1139252171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 msgr2=0x55bc615dd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa65ffff700 1 -- 172.21.15.3:0/1139252171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc61344c00 con 0x55bc615dd8a0 2023-08-11T14:08:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.125+0000 7fa66579c700 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:08:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.126+0000 7fa65ffff700 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d500 tx=0x7fa64c0226f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317683dcf1bceaeb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.126+0000 7fa64b7fe700 1 -- 172.21.15.3:0/1139252171 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00b390 con 0x55bc615dd8a0 2023-08-11T14:08:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.126+0000 7fa64b7fe700 1 -- 172.21.15.3:0/1139252171 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64c004e90 con 0x55bc615dd8a0 2023-08-11T14:08:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.126+0000 7fa64b7fe700 1 -- 172.21.15.3:0/1139252171 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0045f0 con 0x55bc615dd8a0 2023-08-11T14:08:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.127+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1139252171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 msgr2=0x55bc615ee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.127+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00d500 tx=0x7fa64c0226f0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.129+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1139252171 shutdown_connections 2023-08-11T14:08:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.129+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615de790 0x55bc615e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.129+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615dd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.129+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/1139252171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615dd8a0 0x55bc615ee1c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.129+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1139252171 >> 172.21.15.3:0/1139252171 conn(0x55bc6150a8f0 msgr2=0x55bc615e63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.130+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1139252171 shutdown_connections 2023-08-11T14:08:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.131+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/1139252171 wait complete. 2023-08-11T14:08:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.131+0000 7fa67aa2bb80 1 -- start start 2023-08-11T14:08:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.131+0000 7fa67aa2bb80 1 Processor -- start 2023-08-11T14:08:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dd8a0 0x55bc615ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615de790 0x55bc615ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615ed750 con 0x55bc615dcf40 2023-08-11T14:08:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615ed8c0 con 0x55bc615de790 2023-08-11T14:08:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc615eda80 con 0x55bc615dd8a0 2023-08-11T14:08:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa67aa2bb80 10 client.0 _ll_register_callbacks cb 0x55bc614d0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa664f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa664f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39362/0 (socket says 172.21.15.3:39362) 2023-08-11T14:08:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa664f9b700 1 -- 172.21.15.3:0/3141833275 learned_addr learned my addr 172.21.15.3:0/3141833275 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa66579c700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615de790 0x55bc615ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.132+0000 7fa65ffff700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dd8a0 0x55bc615ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 -- 172.21.15.3:0/3141833275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dd8a0 msgr2=0x55bc615ec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dd8a0 0x55bc615ec200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 -- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615de790 msgr2=0x55bc615ed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615de790 0x55bc615ed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 -- 172.21.15.3:0/3141833275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc6135bd80 con 0x55bc615dcf40 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.133+0000 7fa664f9b700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa654006010 tx=0x7fa654004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156b876c37638c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.134+0000 7fa63ffff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.134+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654053510 con 0x55bc615dcf40 2023-08-11T14:08:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.134+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa654053bc0 con 0x55bc615dcf40 2023-08-11T14:08:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.134+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa64c00de10 con 0x55bc615dcf40 2023-08-11T14:08:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.135+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa654053da0 con 0x55bc615dcf40 2023-08-11T14:08:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.135+0000 7fa67aa2bb80 10 client.16107 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.135+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bc613821f0 con 0x55bc615dcf40 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64c00de10 con 0x55bc615dcf40 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa654045870 con 0x55bc615dcf40 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 10 client.16107 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 20 client.16107 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc616583c0 0x55bc6165a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 4 client.16107 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:08:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.137+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc615287c0 con 0x55bc616583c0 2023-08-11T14:08:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.138+0000 7fa65ffff700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc616583c0 0x55bc6165a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.142+0000 7fa65ffff700 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc616583c0 0x55bc6165a7e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baeaa45c19f1a529 server_cookie=b5d890e062bf8a1d in_seq=0 out_seq=0 2023-08-11T14:08:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.142+0000 7fa63ffff700 10 client.16107 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.232+0000 7fa63ffff700 1 -- 172.21.15.3:0/3141833275 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc615287c0 con 0x55bc616583c0 2023-08-11T14:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.232+0000 7fa63ffff700 10 client.16107 handle_command_reply: tid=0 2023-08-11T14:08:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.232+0000 7fa67aa2bb80 1 client.16107 shutdown 2023-08-11T14:08:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.240+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 msgr2=0x55bc615ebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.240+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa654006010 tx=0x7fa654004d50 comp rx=0 tx=0).stop 2023-08-11T14:08:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 shutdown_connections 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55bc616583c0 0x55bc6165a7e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc615de790 0x55bc615ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bc615dcf40 0x55bc615ebcc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 --2- 172.21.15.3:0/3141833275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc615dd8a0 0x55bc615ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.242+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 >> 172.21.15.3:0/3141833275 conn(0x55bc6150a8f0 msgr2=0x55bc615e16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.243+0000 7fa63ffff700 0 client.16107 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 shutdown_connections 2023-08-11T14:08:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa67aa2bb80 1 -- 172.21.15.3:0/3141833275 wait complete. 2023-08-11T14:08:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa67aa2bb80 20 client.16107 trim_cache size 0 max 16384 2023-08-11T14:08:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:08:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:08:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:08:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:08:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 msgr2=0x7fa6500a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 0x7fa6500a2610 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55bc615d92f0 tx=0x7fa654003680 comp rx=0 tx=0).stop 2023-08-11T14:08:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 msgr2=0x7fa66010ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa65800b780 tx=0x7fa65800bb40 comp rx=0 tx=0).stop 2023-08-11T14:08:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 shutdown_connections 2023-08-11T14:08:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6601044d0 0x7fa66010ba70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6600ff190 0x7fa66010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa6500a0150 0x7fa6500a2610 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 --2- 172.21.15.3:0/3363157303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa660101b00 0x7fa66010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 >> 172.21.15.3:0/3363157303 conn(0x7fa6600fad70 msgr2=0x7fa660102810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 shutdown_connections 2023-08-11T14:08:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:42.246+0000 7fa6671ff700 1 -- 172.21.15.3:0/3363157303 wait complete. 2023-08-11T14:08:43.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 105.0 s, 682.4 tps, lat 155.384 ms stddev 285.449 2023-08-11T14:08:43.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:43 smithi103 ceph-mon[130434]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.94k op/s 2023-08-11T14:08:44.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:43 smithi003 ceph-mon[97195]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.94k op/s 2023-08-11T14:08:44.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:43 smithi187 ceph-mon[130973]: pgmap v491: 129 pgs: 129 active+clean; 8.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 MiB/s wr, 2.94k op/s 2023-08-11T14:08:45.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:45 smithi103 ceph-mon[130434]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 3.49k op/s 2023-08-11T14:08:46.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:45 smithi187 ceph-mon[130973]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 3.49k op/s 2023-08-11T14:08:46.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:45 smithi003 ceph-mon[97195]: pgmap v492: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 44 MiB/s wr, 3.49k op/s 2023-08-11T14:08:47.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:47 smithi103 ceph-mon[130434]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 MiB/s wr, 3.67k op/s 2023-08-11T14:08:48.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:47 smithi003 ceph-mon[97195]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 MiB/s wr, 3.67k op/s 2023-08-11T14:08:48.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:47 smithi187 ceph-mon[130973]: pgmap v493: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 MiB/s wr, 3.67k op/s 2023-08-11T14:08:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 110.0 s, 1735.0 tps, lat 58.231 ms stddev 102.587 2023-08-11T14:08:48.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:48 smithi103 ceph-mon[130434]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 3.07k op/s 2023-08-11T14:08:49.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:48 smithi003 ceph-mon[97195]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 3.07k op/s 2023-08-11T14:08:49.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:48 smithi187 ceph-mon[130973]: pgmap v494: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 3.07k op/s 2023-08-11T14:08:51.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:51 smithi003 ceph-mon[97195]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 2.67k op/s 2023-08-11T14:08:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:51 smithi187 ceph-mon[130973]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 2.67k op/s 2023-08-11T14:08:51.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:51 smithi103 ceph-mon[130434]: pgmap v495: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 38 MiB/s wr, 2.67k op/s 2023-08-11T14:08:53.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 115.0 s, 979.8 tps, lat 101.863 ms stddev 214.729 2023-08-11T14:08:53.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:53 smithi003 ceph-mon[97195]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 MiB/s wr, 2.16k op/s 2023-08-11T14:08:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:53 smithi187 ceph-mon[130973]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 MiB/s wr, 2.16k op/s 2023-08-11T14:08:53.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:53 smithi103 ceph-mon[130434]: pgmap v496: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 34 MiB/s wr, 2.16k op/s 2023-08-11T14:08:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.521+0000 7f00291ff700 1 -- 172.21.15.3:0/236914356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240727c0 msgr2=0x7f0024072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.521+0000 7f00291ff700 1 --2- 172.21.15.3:0/236914356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240727c0 0x7f0024072be0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0014007570 tx=0x7f0014007880 comp rx=0 tx=0).stop 2023-08-11T14:08:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.522+0000 7f00291ff700 1 -- 172.21.15.3:0/236914356 shutdown_connections 2023-08-11T14:08:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.522+0000 7f00291ff700 1 --2- 172.21.15.3:0/236914356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f002407a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.522+0000 7f00291ff700 1 --2- 172.21.15.3:0/236914356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240727c0 0x7f0024072be0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.522+0000 7f00291ff700 1 --2- 172.21.15.3:0/236914356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00240756a0 0x7f0024077b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.522+0000 7f00291ff700 1 -- 172.21.15.3:0/236914356 >> 172.21.15.3:0/236914356 conn(0x7f002406d750 msgr2=0x7f002406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.524+0000 7f00291ff700 1 -- 172.21.15.3:0/236914356 shutdown_connections 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.524+0000 7f00291ff700 1 -- 172.21.15.3:0/236914356 wait complete. 2023-08-11T14:08:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.527+0000 7f00291ff700 1 Processor -- start 2023-08-11T14:08:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 -- start start 2023-08-11T14:08:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00240727c0 0x7f002412bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 0x7f002412e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f0024130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024130e10 con 0x7f00240727c0 2023-08-11T14:08:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024130f80 con 0x7f00240756a0 2023-08-11T14:08:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.528+0000 7f00291ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024131140 con 0x7f0024078070 2023-08-11T14:08:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.534+0000 7f001a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 0x7f002412e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.534+0000 7f002359e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f0024130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.534+0000 7f002359e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f0024130640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46288/0 (socket says 172.21.15.3:46288) 2023-08-11T14:08:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.534+0000 7f002359e700 1 -- 172.21.15.3:0/3167838206 learned_addr learned my addr 172.21.15.3:0/3167838206 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.535+0000 7f001a59c700 1 -- 172.21.15.3:0/3167838206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 msgr2=0x7f0024130640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.535+0000 7f001a59c700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f0024130640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.535+0000 7f001a59c700 1 -- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00240727c0 msgr2=0x7f002412bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.535+0000 7f001a59c700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00240727c0 0x7f002412bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.535+0000 7f001a59c700 1 -- 172.21.15.3:0/3167838206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014007220 con 0x7f00240756a0 2023-08-11T14:08:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.536+0000 7f001a59c700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 0x7f002412e0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b450 tx=0x7f000c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.540+0000 7f0020d99700 1 -- 172.21.15.3:0/3167838206 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c011830 con 0x7f00240756a0 2023-08-11T14:08:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.540+0000 7f0020d99700 1 -- 172.21.15.3:0/3167838206 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c011e50 con 0x7f00240756a0 2023-08-11T14:08:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.540+0000 7f0020d99700 1 -- 172.21.15.3:0/3167838206 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c010e10 con 0x7f00240756a0 2023-08-11T14:08:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.544+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0024131420 con 0x7f00240756a0 2023-08-11T14:08:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.545+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0024131930 con 0x7f00240756a0 2023-08-11T14:08:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.550+0000 7f003cbc3b80 1 Processor -- start 2023-08-11T14:08:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 -- start start 2023-08-11T14:08:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea9141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea914c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea86c5d0 con 0x5632ea914790 2023-08-11T14:08:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea869ef0 con 0x5632ea912f40 2023-08-11T14:08:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.551+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea8657e0 con 0x5632ea9138a0 2023-08-11T14:08:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60032/0 (socket says 172.21.15.3:60032) 2023-08-11T14:08:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 -- 172.21.15.3:0/1162755152 learned_addr learned my addr 172.21.15.3:0/1162755152 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 -- 172.21.15.3:0/1162755152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 msgr2=0x5632ea9141b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:08:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea9141b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 -- 172.21.15.3:0/1162755152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 msgr2=0x5632ea914c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea914c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.552+0000 7f002359e700 1 -- 172.21.15.3:0/1162755152 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ea6e55d0 con 0x5632ea912f40 2023-08-11T14:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.553+0000 7f002359e700 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e270 tx=0x7f001c009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ceceafe38f4803 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.553+0000 7f001959a700 1 -- 172.21.15.3:0/1162755152 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c00ee40 con 0x5632ea912f40 2023-08-11T14:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.553+0000 7f001959a700 1 -- 172.21.15.3:0/1162755152 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x5632ea912f40 2023-08-11T14:08:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.553+0000 7f001959a700 1 -- 172.21.15.3:0/1162755152 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0137a0 con 0x5632ea912f40 2023-08-11T14:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.557+0000 7f0020d99700 1 -- 172.21.15.3:0/3167838206 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f000c010510 con 0x7f00240756a0 2023-08-11T14:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.559+0000 7f0020d99700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 0x7f00100a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.559+0000 7f0020d99700 1 -- 172.21.15.3:0/3167838206 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f000c0cae20 con 0x7f00240756a0 2023-08-11T14:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.561+0000 7f003cbc3b80 1 -- 172.21.15.3:0/1162755152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 msgr2=0x5632ea913360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.561+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f001c00e270 tx=0x7f001c009710 comp rx=0 tx=0).stop 2023-08-11T14:08:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f003cbc3b80 1 -- 172.21.15.3:0/1162755152 shutdown_connections 2023-08-11T14:08:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea914c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea912f40 0x5632ea913360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/1162755152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea9141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f003cbc3b80 1 -- 172.21.15.3:0/1162755152 >> 172.21.15.3:0/1162755152 conn(0x5632ea8408f0 msgr2=0x5632ea90f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.577+0000 7f0022d9d700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 0x7f00100a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.578+0000 7f003cbc3b80 1 -- 172.21.15.3:0/1162755152 shutdown_connections 2023-08-11T14:08:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.578+0000 7f003cbc3b80 1 -- 172.21.15.3:0/1162755152 wait complete. 2023-08-11T14:08:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.578+0000 7f003cbc3b80 1 -- start start 2023-08-11T14:08:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 Processor -- start 2023-08-11T14:08:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea91d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea914790 0x5632ea9242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea803a60 con 0x5632ea912f40 2023-08-11T14:08:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea803760 con 0x5632ea9138a0 2023-08-11T14:08:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.579+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea6e78c0 con 0x5632ea914790 2023-08-11T14:08:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.580+0000 7f003cbc3b80 10 client.0 _ll_register_callbacks cb 0x5632ea806a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.584+0000 7f0022d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.584+0000 7f0022d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56692/0 (socket says 172.21.15.3:56692) 2023-08-11T14:08:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.584+0000 7f0022d9d700 1 -- 172.21.15.3:0/2086618767 learned_addr learned my addr 172.21.15.3:0/2086618767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.586+0000 7f0022d9d700 1 -- 172.21.15.3:0/2086618767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea914790 msgr2=0x5632ea9242d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:08:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.586+0000 7f0022d9d700 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea914790 0x5632ea9242d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.586+0000 7f0022d9d700 1 -- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 msgr2=0x5632ea91d290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:08:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.586+0000 7f0022d9d700 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea91d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.586+0000 7f0022d9d700 1 -- 172.21.15.3:0/2086618767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c009dc0 con 0x5632ea912f40 2023-08-11T14:08:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.592+0000 7f0022d9d700 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0014016000 tx=0x7f0014000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32530623084d6dd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.592+0000 7f000a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:08:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.592+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001400f040 con 0x5632ea912f40 2023-08-11T14:08:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.592+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f00140051a0 con 0x5632ea912f40 2023-08-11T14:08:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.592+0000 7f0022d9d700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 0x7f00100a2610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0014005df0 tx=0x7f0014008af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.593+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632ea9067e0 con 0x5632ea912f40 2023-08-11T14:08:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.593+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001400aab0 con 0x5632ea912f40 2023-08-11T14:08:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.593+0000 7f003cbc3b80 10 client.16110 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.593+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632ea691d80 con 0x5632ea912f40 2023-08-11T14:08:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.596+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632ea9067e0 con 0x5632ea912f40 2023-08-11T14:08:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.596+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0014032070 con 0x5632ea912f40 2023-08-11T14:08:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 20 client.16110 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea928c20 0x5632ea92b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 4 client.16110 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:08:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.598+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632ea8e2480 con 0x5632ea928c20 2023-08-11T14:08:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.606+0000 7f001a59c700 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea928c20 0x5632ea92b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.621+0000 7f001a59c700 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea928c20 0x5632ea92b040 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf8dcc3f8ebf247 server_cookie=32ab3589c1e3cf2a in_seq=0 out_seq=0 2023-08-11T14:08:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.622+0000 7f000a7fc700 10 client.16110 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.623+0000 7f000a7fc700 1 -- 172.21.15.3:0/2086618767 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5632ea8e2480 con 0x5632ea928c20 2023-08-11T14:08:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.623+0000 7f000a7fc700 10 client.16110 handle_command_reply: tid=0 2023-08-11T14:08:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.623+0000 7f003cbc3b80 1 client.16110 shutdown 2023-08-11T14:08:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.624+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 msgr2=0x5632ea91cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.624+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0014016000 tx=0x7f0014000bc0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.627+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 shutdown_connections 2023-08-11T14:08:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.627+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea914790 0x5632ea9242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.627+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91cd50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.627+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea928c20 0x5632ea92b040 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.628+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/2086618767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea91d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.628+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 >> 172.21.15.3:0/2086618767 conn(0x5632ea8408f0 msgr2=0x5632ea90e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.628+0000 7f000a7fc700 0 client.16110 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.635+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 shutdown_connections 2023-08-11T14:08:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.636+0000 7f003cbc3b80 1 -- 172.21.15.3:0/2086618767 wait complete. 2023-08-11T14:08:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.637+0000 7f003cbc3b80 20 client.16110 trim_cache size 0 max 16384 2023-08-11T14:08:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 Processor -- start 2023-08-11T14:08:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 -- start start 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea912f40 0x5632ea9177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea924290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea914610 con 0x5632ea914790 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea912b30 con 0x5632ea9138a0 2023-08-11T14:08:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.666+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea908ea0 con 0x5632ea912f40 2023-08-11T14:08:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60048/0 (socket says 172.21.15.3:60048) 2023-08-11T14:08:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 -- 172.21.15.3:0/639104343 learned_addr learned my addr 172.21.15.3:0/639104343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f002359e700 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea924290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 -- 172.21.15.3:0/639104343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea912f40 msgr2=0x5632ea9177f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea912f40 0x5632ea9177f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 -- 172.21.15.3:0/639104343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 msgr2=0x5632ea924290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea924290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 -- 172.21.15.3:0/639104343 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ea67ac00 con 0x5632ea9138a0 2023-08-11T14:08:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.667+0000 7f001a59c700 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f000c009fd0 tx=0x7f000c0ca040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc8d10be305c27c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.668+0000 7f0019d9b700 1 -- 172.21.15.3:0/639104343 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c098c70 con 0x5632ea9138a0 2023-08-11T14:08:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.668+0000 7f0019d9b700 1 -- 172.21.15.3:0/639104343 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c098570 con 0x5632ea9138a0 2023-08-11T14:08:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.668+0000 7f0019d9b700 1 -- 172.21.15.3:0/639104343 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0969a0 con 0x5632ea9138a0 2023-08-11T14:08:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.669+0000 7f003cbc3b80 1 -- 172.21.15.3:0/639104343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 msgr2=0x5632ea917d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.669+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f000c009fd0 tx=0x7f000c0ca040 comp rx=0 tx=0).stop 2023-08-11T14:08:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 -- 172.21.15.3:0/639104343 shutdown_connections 2023-08-11T14:08:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea914790 0x5632ea924290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea912f40 0x5632ea9177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/639104343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea9138a0 0x5632ea917d30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 -- 172.21.15.3:0/639104343 >> 172.21.15.3:0/639104343 conn(0x5632ea8408f0 msgr2=0x5632ea91b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.675+0000 7f003cbc3b80 1 -- 172.21.15.3:0/639104343 shutdown_connections 2023-08-11T14:08:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.679+0000 7f003cbc3b80 1 -- 172.21.15.3:0/639104343 wait complete. 2023-08-11T14:08:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.679+0000 7f003cbc3b80 1 -- start start 2023-08-11T14:08:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.680+0000 7f003cbc3b80 1 Processor -- start 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.680+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea920260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 0x5632ea923280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea9237c0 con 0x5632ea912f40 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea923930 con 0x5632ea914790 2023-08-11T14:08:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ea923af0 con 0x5632ea9138a0 2023-08-11T14:08:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f003cbc3b80 10 client.0 _ll_register_callbacks cb 0x7f001400a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:08:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f0022d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f0022d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56712/0 (socket says 172.21.15.3:56712) 2023-08-11T14:08:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.681+0000 7f0022d9d700 1 -- 172.21.15.3:0/3692647043 learned_addr learned my addr 172.21.15.3:0/3692647043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:08:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f001a59c700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea920260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 0x5632ea923280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 -- 172.21.15.3:0/3692647043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 msgr2=0x5632ea920260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea920260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 -- 172.21.15.3:0/3692647043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 msgr2=0x5632ea91fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:08:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.682+0000 7f002359e700 1 -- 172.21.15.3:0/3692647043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0024131930 con 0x5632ea914790 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f0022d9d700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f001a59c700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea920260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f002359e700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 0x5632ea923280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f001c000f80 tx=0x7f001c003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37f01f740baab4b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f000a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c014890 con 0x5632ea914790 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f001c0149f0 con 0x5632ea914790 2023-08-11T14:08:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.683+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f001c00f040 con 0x5632ea914790 2023-08-11T14:08:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.684+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f001c014bb0 con 0x5632ea914790 2023-08-11T14:08:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.684+0000 7f003cbc3b80 10 client.26207 fetch_fsmap learned FSMap version 33 2023-08-11T14:08:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.684+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632ea6b81f0 con 0x5632ea914790 2023-08-11T14:08:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.684+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x5632ea914790 2023-08-11T14:08:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f001c004040 con 0x5632ea914790 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 10 client.26207 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 10 client.26207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 20 client.26207 populate_metadata read hostname 'smithi003' 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea97d370 0x5632ea97f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 4 client.26207 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:08:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.686+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632ea85e7c0 con 0x5632ea97d370 2023-08-11T14:08:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.691+0000 7f0022d9d700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea97d370 0x5632ea97f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:08:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.692+0000 7f0022d9d700 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea97d370 0x5632ea97f760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e618ada92425040 server_cookie=24d13035b023f907 in_seq=0 out_seq=0 2023-08-11T14:08:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.692+0000 7f000a7fc700 10 client.26207 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.749+0000 7f000a7fc700 1 -- 172.21.15.3:0/3692647043 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632ea85e7c0 con 0x5632ea97d370 2023-08-11T14:08:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.749+0000 7f000a7fc700 10 client.26207 handle_command_reply: tid=0 2023-08-11T14:08:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.749+0000 7f003cbc3b80 1 client.26207 shutdown 2023-08-11T14:08:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.759+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 msgr2=0x5632ea923280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.759+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 0x5632ea923280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f001c000f80 tx=0x7f001c003e10 comp rx=0 tx=0).stop 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 shutdown_connections 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632ea97d370 0x5632ea97f760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632ea914790 0x5632ea923280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5632ea912f40 0x5632ea91fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 --2- 172.21.15.3:0/3692647043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632ea9138a0 0x5632ea920260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f000a7fc700 0 client.26207 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:08:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.763+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 >> 172.21.15.3:0/3692647043 conn(0x5632ea8408f0 msgr2=0x5632ea91b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.764+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 shutdown_connections 2023-08-11T14:08:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.764+0000 7f003cbc3b80 1 -- 172.21.15.3:0/3692647043 wait complete. 2023-08-11T14:08:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.764+0000 7f003cbc3b80 20 client.26207 trim_cache size 0 max 16384 2023-08-11T14:08:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:08:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:08:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:08:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:08:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.768+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 msgr2=0x7f00100a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.768+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 0x7f00100a2610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0014005df0 tx=0x7f0014008af0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.768+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 msgr2=0x7f002412e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:08:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.768+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 0x7f002412e0b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b450 tx=0x7f000c00b760 comp rx=0 tx=0).stop 2023-08-11T14:08:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 shutdown_connections 2023-08-11T14:08:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0024078070 0x7f0024130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00240727c0 0x7f002412bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f00100a0150 0x7f00100a2610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 --2- 172.21.15.3:0/3167838206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00240756a0 0x7f002412e0b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:08:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.788+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 >> 172.21.15.3:0/3167838206 conn(0x7f002406d750 msgr2=0x7f0024076fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:08:55.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:55 smithi187 ceph-mon[130973]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-08-11T14:08:55.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:55 smithi003 ceph-mon[97195]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-08-11T14:08:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.803+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 shutdown_connections 2023-08-11T14:08:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:08:55.804+0000 7f00291ff700 1 -- 172.21.15.3:0/3167838206 wait complete. 2023-08-11T14:08:55.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:55 smithi103 ceph-mon[130434]: pgmap v497: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 34 MiB/s wr, 1.93k op/s 2023-08-11T14:08:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:57 smithi187 ceph-mon[130973]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.49k op/s 2023-08-11T14:08:57.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:57 smithi003 ceph-mon[97195]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.49k op/s 2023-08-11T14:08:57.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:57 smithi103 ceph-mon[130434]: pgmap v498: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 1.49k op/s 2023-08-11T14:08:58.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 120.0 s, 1328.7 tps, lat 79.341 ms stddev 97.081 2023-08-11T14:08:59.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:08:59 smithi103 ceph-mon[130434]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-08-11T14:09:00.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:08:59 smithi003 ceph-mon[97195]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-08-11T14:09:00.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:08:59 smithi187 ceph-mon[130973]: pgmap v499: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-08-11T14:09:01.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:01 smithi103 ceph-mon[130434]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-08-11T14:09:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:01 smithi003 ceph-mon[97195]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-08-11T14:09:02.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:01 smithi187 ceph-mon[130973]: pgmap v500: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.33k op/s 2023-08-11T14:09:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 125.0 s, 1049.7 tps, lat 94.859 ms stddev 197.740 2023-08-11T14:09:03.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:03 smithi103 ceph-mon[130434]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-08-11T14:09:04.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:03 smithi003 ceph-mon[97195]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-08-11T14:09:04.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:03 smithi187 ceph-mon[130973]: pgmap v501: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 1.09k op/s 2023-08-11T14:09:06.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:05 smithi187 ceph-mon[130973]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2023-08-11T14:09:06.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:05 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:09:06.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:05 smithi003 ceph-mon[97195]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2023-08-11T14:09:06.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:05 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:09:06.185 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:05 smithi103 ceph-mon[130434]: pgmap v502: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 31 MiB/s wr, 1.67k op/s 2023-08-11T14:09:06.185 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:05 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:09:07.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:06 smithi003 ceph-mon[97195]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 2.08k op/s 2023-08-11T14:09:07.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:06 smithi187 ceph-mon[130973]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 2.08k op/s 2023-08-11T14:09:07.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:06 smithi103 ceph-mon[130434]: pgmap v503: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 2.08k op/s 2023-08-11T14:09:08.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:09:08.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:09:08.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:07 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:09:08.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:09:08.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:09:08.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:07 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:09:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 130.0 s, 571.8 tps, lat 176.342 ms stddev 250.045 2023-08-11T14:09:08.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:09:08.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:09:08.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:07 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:09:09.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:08 smithi003 ceph-mon[97195]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 2.22k op/s 2023-08-11T14:09:09.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:08 smithi187 ceph-mon[130973]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 2.22k op/s 2023-08-11T14:09:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.085+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1123912200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f80ff820 msgr2=0x7fc1f80ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.085+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1123912200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f80ff820 0x7fc1f80ffc40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc1f001e360 tx=0x7fc1f001e670 comp rx=0 tx=0).stop 2023-08-11T14:09:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.089+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1123912200 shutdown_connections 2023-08-11T14:09:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.089+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1123912200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f81013b0 0x7fc1f8107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.089+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1123912200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f80ff820 0x7fc1f80ffc40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.089+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1123912200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1f8100a10 0x7fc1f8100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.090+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1123912200 >> 172.21.15.3:0/1123912200 conn(0x7fc1f80fad70 msgr2=0x7fc1f80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.092+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1123912200 shutdown_connections 2023-08-11T14:09:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.092+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1123912200 wait complete. 2023-08-11T14:09:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.094+0000 7fc1ff9ff700 1 Processor -- start 2023-08-11T14:09:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 -- start start 2023-08-11T14:09:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1f80ff820 0x7fc1f8071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f8100a10 0x7fc1f80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f818dbf0 con 0x7fc1f8100a10 2023-08-11T14:09:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f818dd60 con 0x7fc1f80ff820 2023-08-11T14:09:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.095+0000 7fc1ff9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f818df20 con 0x7fc1f81013b0 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.096+0000 7fc1fdf9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.096+0000 7fc1fdf9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46714/0 (socket says 172.21.15.3:46714) 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.096+0000 7fc1fdf9c700 1 -- 172.21.15.3:0/1772771547 learned_addr learned my addr 172.21.15.3:0/1772771547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 -- 172.21.15.3:0/1772771547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1f80ff820 msgr2=0x7fc1f8071490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1f80ff820 0x7fc1f8071490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 -- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f8100a10 msgr2=0x7fc1f80719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f8100a10 0x7fc1f80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 -- 172.21.15.3:0/1772771547 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f001ad10 con 0x7fc1f81013b0 2023-08-11T14:09:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.097+0000 7fc1fdf9c700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800e940 tx=0x7fc1e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.098+0000 7fc1ee7fc700 1 -- 172.21.15.3:0/1772771547 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e800cbd0 con 0x7fc1f81013b0 2023-08-11T14:09:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.098+0000 7fc1ee7fc700 1 -- 172.21.15.3:0/1772771547 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e800cd30 con 0x7fc1f81013b0 2023-08-11T14:09:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.098+0000 7fc1ee7fc700 1 -- 172.21.15.3:0/1772771547 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80056c0 con 0x7fc1f81013b0 2023-08-11T14:09:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.099+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f818e200 con 0x7fc1f81013b0 2023-08-11T14:09:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.099+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f8194a40 con 0x7fc1f81013b0 2023-08-11T14:09:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.106+0000 7fc213243b80 1 Processor -- start 2023-08-11T14:09:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc1ee7fc700 1 -- 172.21.15.3:0/1772771547 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc1e8004750 con 0x7fc1f81013b0 2023-08-11T14:09:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 -- start start 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9bf40 0x56150cf9c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9c8a0 0x56150cf9d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cef55d0 con 0x56150cf9bf40 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cef2ef0 con 0x56150cf9d790 2023-08-11T14:09:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.107+0000 7fc213243b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150ceee7e0 con 0x56150cf9c8a0 2023-08-11T14:09:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.109+0000 7fc1ee7fc700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 0x7fc1e40a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.109+0000 7fc1ee7fc700 1 -- 172.21.15.3:0/1772771547 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc1e8014070 con 0x7fc1f81013b0 2023-08-11T14:09:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.110+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.110+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40460/0 (socket says 172.21.15.3:40460) 2023-08-11T14:09:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.110+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/2349205104 learned_addr learned my addr 172.21.15.3:0/2349205104 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.111+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9c8a0 0x56150cf9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.119+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/2349205104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9c8a0 msgr2=0x56150cf9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.119+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9c8a0 0x56150cf9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.119+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/2349205104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9bf40 msgr2=0x56150cf9c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.119+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9bf40 0x56150cf9c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.119+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/2349205104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56150cd6e5d0 con 0x56150cf9d790 2023-08-11T14:09:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.120+0000 7fc1fdf9c700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 0x7fc1e40a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.120+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f400aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5699f997d37be7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.122+0000 7fc1e37fe700 1 -- 172.21.15.3:0/2349205104 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400fb90 con 0x56150cf9d790 2023-08-11T14:09:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.122+0000 7fc1e37fe700 1 -- 172.21.15.3:0/2349205104 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f400fcf0 con 0x56150cf9d790 2023-08-11T14:09:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.122+0000 7fc1e37fe700 1 -- 172.21.15.3:0/2349205104 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4010630 con 0x56150cf9d790 2023-08-11T14:09:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.123+0000 7fc213243b80 1 -- 172.21.15.3:0/2349205104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 msgr2=0x56150cf9dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.123+0000 7fc213243b80 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f400aef0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.125+0000 7fc213243b80 1 -- 172.21.15.3:0/2349205104 shutdown_connections 2023-08-11T14:09:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.125+0000 7fc213243b80 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cf9dc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.125+0000 7fc213243b80 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9bf40 0x56150cf9c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.125+0000 7fc213243b80 1 --2- 172.21.15.3:0/2349205104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9c8a0 0x56150cf9d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.125+0000 7fc213243b80 1 -- 172.21.15.3:0/2349205104 >> 172.21.15.3:0/2349205104 conn(0x56150cec98f0 msgr2=0x56150cf98920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.126+0000 7fc1fdf9c700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 0x7fc1e40a2610 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8098da0 tx=0x7fc1e801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.126+0000 7fc213243b80 1 -- 172.21.15.3:0/2349205104 shutdown_connections 2023-08-11T14:09:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.127+0000 7fc213243b80 1 -- 172.21.15.3:0/2349205104 wait complete. 2023-08-11T14:09:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.127+0000 7fc213243b80 1 -- start start 2023-08-11T14:09:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.128+0000 7fc213243b80 1 Processor -- start 2023-08-11T14:09:09.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9d790 0x56150cfad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150ce8ca60 con 0x56150cf9d790 2023-08-11T14:09:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150ce8c760 con 0x56150cf9c8a0 2023-08-11T14:09:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cd708c0 con 0x56150cf9bf40 2023-08-11T14:09:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.129+0000 7fc213243b80 10 client.0 _ll_register_callbacks cb 0x56150ce8fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.130+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.130+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46726/0 (socket says 172.21.15.3:46726) 2023-08-11T14:09:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.131+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/4292660677 learned_addr learned my addr 172.21.15.3:0/4292660677 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.131+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 -- 172.21.15.3:0/4292660677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 msgr2=0x56150cfa5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 -- 172.21.15.3:0/4292660677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9d790 msgr2=0x56150cfad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9d790 0x56150cfad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 -- 172.21.15.3:0/4292660677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f400a8f0 con 0x56150cf9c8a0 2023-08-11T14:09:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.132+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1f001e330 tx=0x7fc1f0003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4132bcd7cf517bc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.133+0000 7fc1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:09:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.133+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0022ab0 con 0x56150cf9c8a0 2023-08-11T14:09:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.133+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc1f0022c10 con 0x56150cf9c8a0 2023-08-11T14:09:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.133+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56150cf8f7e0 con 0x56150cf9c8a0 2023-08-11T14:09:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.134+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f001e140 con 0x56150cf9c8a0 2023-08-11T14:09:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.134+0000 7fc213243b80 10 client.26216 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.134+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56150cd1ad80 con 0x56150cf9c8a0 2023-08-11T14:09:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.134+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56150cf8f7e0 con 0x56150cf9c8a0 2023-08-11T14:09:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.134+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1f0032030 con 0x56150cf9c8a0 2023-08-11T14:09:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 20 client.26216 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150cfb1ac0 0x56150cfb3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 4 client.26216 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:09:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56150cf6b480 con 0x56150cfb1ac0 2023-08-11T14:09:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.135+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150cfb1ac0 0x56150cfb3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.136+0000 7fc1e0ff9700 10 client.26216 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.136+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150cfb1ac0 0x56150cfb3ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511ebc6cad4f56ab server_cookie=47db525a008d399 in_seq=0 out_seq=0 2023-08-11T14:09:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.136+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/4292660677 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56150cf6b480 con 0x56150cfb1ac0 2023-08-11T14:09:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.136+0000 7fc1e0ff9700 10 client.26216 handle_command_reply: tid=0 2023-08-11T14:09:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 client.26216 shutdown 2023-08-11T14:09:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 msgr2=0x56150cfa6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1f001e330 tx=0x7fc1f0003c30 comp rx=0 tx=0).stop 2023-08-11T14:09:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 shutdown_connections 2023-08-11T14:09:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9d790 0x56150cfad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150cfb1ac0 0x56150cfb3ee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 --2- 172.21.15.3:0/4292660677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 >> 172.21.15.3:0/4292660677 conn(0x56150cec98f0 msgr2=0x56150cf97710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc1e0ff9700 0 client.26216 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.137+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 shutdown_connections 2023-08-11T14:09:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.138+0000 7fc213243b80 1 -- 172.21.15.3:0/4292660677 wait complete. 2023-08-11T14:09:09.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.138+0000 7fc213243b80 20 client.26216 trim_cache size 0 max 16384 2023-08-11T14:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 Processor -- start 2023-08-11T14:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 -- start start 2023-08-11T14:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9d790 0x56150cfad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cf91bd0 con 0x56150cf9c8a0 2023-08-11T14:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cfa1250 con 0x56150cf9bf40 2023-08-11T14:09:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.167+0000 7fc213243b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cfa68b0 con 0x56150cf9d790 2023-08-11T14:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40474/0 (socket says 172.21.15.3:40474) 2023-08-11T14:09:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/3081296182 learned_addr learned my addr 172.21.15.3:0/3081296182 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9d790 0x56150cfad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/3081296182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9d790 msgr2=0x56150cfad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9d790 0x56150cfad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/3081296182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 msgr2=0x56150cfa6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/3081296182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56150cd03c00 con 0x56150cf9bf40 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.168+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.169+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006010 tx=0x7fc1f4001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=213c93ba9b1bc8cd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.169+0000 7fc1e3fff700 1 -- 172.21.15.3:0/3081296182 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400fa90 con 0x56150cf9bf40 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.169+0000 7fc1e3fff700 1 -- 172.21.15.3:0/3081296182 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f402f070 con 0x56150cf9bf40 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.169+0000 7fc1e3fff700 1 -- 172.21.15.3:0/3081296182 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004a90 con 0x56150cf9bf40 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 -- 172.21.15.3:0/3081296182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 msgr2=0x56150cfa5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006010 tx=0x7fc1f4001e50 comp rx=0 tx=0).stop 2023-08-11T14:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 -- 172.21.15.3:0/3081296182 shutdown_connections 2023-08-11T14:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9d790 0x56150cfad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9bf40 0x56150cfa5c50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 --2- 172.21.15.3:0/3081296182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfa6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 -- 172.21.15.3:0/3081296182 >> 172.21.15.3:0/3081296182 conn(0x56150cec98f0 msgr2=0x56150cfa3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.170+0000 7fc213243b80 1 -- 172.21.15.3:0/3081296182 shutdown_connections 2023-08-11T14:09:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.171+0000 7fc213243b80 1 -- 172.21.15.3:0/3081296182 wait complete. 2023-08-11T14:09:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.171+0000 7fc213243b80 1 -- start start 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.171+0000 7fc213243b80 1 Processor -- start 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cfab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cfab6f0 con 0x56150cf9c8a0 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cfab860 con 0x56150cf9d790 2023-08-11T14:09:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56150cfaba20 con 0x56150cf9bf40 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.172+0000 7fc213243b80 10 client.0 _ll_register_callbacks cb 0x7fc1f001a3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.173+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.173+0000 7fc1fcf9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46732/0 (socket says 172.21.15.3:46732) 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.173+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/143402954 learned_addr learned my addr 172.21.15.3:0/143402954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.173+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cfab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 msgr2=0x56150cfab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cfab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/143402954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 msgr2=0x56150cfad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 -- 172.21.15.3:0/143402954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56150cd1ad80 con 0x56150cf9bf40 2023-08-11T14:09:09.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:08 smithi103 ceph-mon[130434]: pgmap v504: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 2.22k op/s 2023-08-11T14:09:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1fcf9a700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fe0 tx=0x7fc1f400a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960b19e60d7893ef server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.174+0000 7fc1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4029e30 con 0x56150cf9bf40 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc1f4004830 con 0x56150cf9bf40 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1f400fcf0 con 0x56150cf9bf40 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f40049f0 con 0x56150cf9bf40 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc213243b80 10 client.26197 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.175+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56150cd6e5d0 con 0x56150cf9bf40 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.176+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f400fcf0 con 0x56150cf9bf40 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.176+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1f4033040 con 0x56150cf9bf40 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.176+0000 7fc213243b80 10 client.26197 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.177+0000 7fc213243b80 10 client.26197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.177+0000 7fc213243b80 20 client.26197 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.177+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150d006520 0x56150d008940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.177+0000 7fc213243b80 4 client.26197 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.177+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56150cee77c0 con 0x56150d006520 2023-08-11T14:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.178+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150d006520 0x56150d008940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.181+0000 7fc1fd79b700 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150d006520 0x56150d008940 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92000e5d3b6060d5 server_cookie=5d2cedb8eca6593 in_seq=0 out_seq=0 2023-08-11T14:09:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.181+0000 7fc1e0ff9700 10 client.26197 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:09.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.307+0000 7fc1e0ff9700 1 -- 172.21.15.3:0/143402954 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56150cee77c0 con 0x56150d006520 2023-08-11T14:09:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.307+0000 7fc1e0ff9700 10 client.26197 handle_command_reply: tid=0 2023-08-11T14:09:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.307+0000 7fc213243b80 1 client.26197 shutdown 2023-08-11T14:09:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.317+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 msgr2=0x56150cfacb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.317+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fe0 tx=0x7fc1f400a590 comp rx=0 tx=0).stop 2023-08-11T14:09:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 shutdown_connections 2023-08-11T14:09:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56150d006520 0x56150d008940 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56150cf9d790 0x56150cfab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56150cf9bf40 0x56150cfacb10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 --2- 172.21.15.3:0/143402954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56150cf9c8a0 0x56150cfad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 >> 172.21.15.3:0/143402954 conn(0x56150cec98f0 msgr2=0x56150cfa06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.318+0000 7fc1e0ff9700 0 client.26197 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.322+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 shutdown_connections 2023-08-11T14:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.322+0000 7fc213243b80 1 -- 172.21.15.3:0/143402954 wait complete. 2023-08-11T14:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.323+0000 7fc213243b80 20 client.26197 trim_cache size 0 max 16384 2023-08-11T14:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:09:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:09:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:09:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.325+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 msgr2=0x7fc1e40a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.325+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 0x7fc1e40a2610 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8098da0 tx=0x7fc1e801d000 comp rx=0 tx=0).stop 2023-08-11T14:09:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.325+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 msgr2=0x7fc1f818d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.325+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800e940 tx=0x7fc1e800ed00 comp rx=0 tx=0).stop 2023-08-11T14:09:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 shutdown_connections 2023-08-11T14:09:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1f81013b0 0x7fc1f818d5f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc1e40a0150 0x7fc1e40a2610 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1f80ff820 0x7fc1f8071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 --2- 172.21.15.3:0/1772771547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f8100a10 0x7fc1f80719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.327+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 >> 172.21.15.3:0/1772771547 conn(0x7fc1f80fad70 msgr2=0x7fc1f8103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.328+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 shutdown_connections 2023-08-11T14:09:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:09.330+0000 7fc1ff9ff700 1 -- 172.21.15.3:0/1772771547 wait complete. 2023-08-11T14:09:11.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:11 smithi187 ceph-mon[130973]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.69k op/s 2023-08-11T14:09:11.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:11 smithi003 ceph-mon[97195]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.69k op/s 2023-08-11T14:09:11.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:11 smithi103 ceph-mon[130434]: pgmap v505: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 MiB/s wr, 2.69k op/s 2023-08-11T14:09:13.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 135.0 s, 2051.1 tps, lat 48.621 ms stddev 41.222 2023-08-11T14:09:13.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:13 smithi103 ceph-mon[130434]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.69k op/s 2023-08-11T14:09:14.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:13 smithi187 ceph-mon[130973]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.69k op/s 2023-08-11T14:09:14.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:13 smithi003 ceph-mon[97195]: pgmap v506: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 2.69k op/s 2023-08-11T14:09:15.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:15 smithi103 ceph-mon[130434]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 2.60k op/s 2023-08-11T14:09:16.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:15 smithi003 ceph-mon[97195]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 2.60k op/s 2023-08-11T14:09:16.051 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:15 smithi187 ceph-mon[130973]: pgmap v507: 129 pgs: 129 active+clean; 8.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 MiB/s wr, 2.60k op/s 2023-08-11T14:09:18.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:17 smithi187 ceph-mon[130973]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-08-11T14:09:18.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:17 smithi003 ceph-mon[97195]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-08-11T14:09:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 140.0 s, 871.2 tps, lat 114.594 ms stddev 227.330 2023-08-11T14:09:18.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:17 smithi103 ceph-mon[130434]: pgmap v508: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 2.07k op/s 2023-08-11T14:09:19.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:18 smithi103 ceph-mon[130434]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.53k op/s 2023-08-11T14:09:19.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:18 smithi187 ceph-mon[130973]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.53k op/s 2023-08-11T14:09:19.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:18 smithi003 ceph-mon[97195]: pgmap v509: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.53k op/s 2023-08-11T14:09:21.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:21 smithi187 ceph-mon[130973]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2023-08-11T14:09:21.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:21 smithi103 ceph-mon[130434]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2023-08-11T14:09:22.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:21 smithi003 ceph-mon[97195]: pgmap v510: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 1.46k op/s 2023-08-11T14:09:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 145.0 s, 408.4 tps, lat 243.725 ms stddev 359.784 2023-08-11T14:09:24.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:23 smithi187 ceph-mon[130973]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-08-11T14:09:24.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:23 smithi103 ceph-mon[130434]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-08-11T14:09:24.265 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:23 smithi003 ceph-mon[97195]: pgmap v511: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 MiB/s wr, 1.51k op/s 2023-08-11T14:09:26.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:25 smithi003 ceph-mon[97195]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2023-08-11T14:09:26.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:25 smithi187 ceph-mon[130973]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2023-08-11T14:09:26.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:25 smithi103 ceph-mon[130434]: pgmap v512: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2023-08-11T14:09:28.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:27 smithi003 ceph-mon[97195]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 30 MiB/s wr, 2.24k op/s 2023-08-11T14:09:28.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:27 smithi187 ceph-mon[130973]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 30 MiB/s wr, 2.24k op/s 2023-08-11T14:09:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 150.0 s, 523.5 tps, lat 191.415 ms stddev 310.244 2023-08-11T14:09:28.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:27 smithi103 ceph-mon[130434]: pgmap v513: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 30 MiB/s wr, 2.24k op/s 2023-08-11T14:09:30.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:29 smithi003 ceph-mon[97195]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-08-11T14:09:30.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:29 smithi187 ceph-mon[130973]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-08-11T14:09:30.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:29 smithi103 ceph-mon[130434]: pgmap v514: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 25 MiB/s wr, 2.22k op/s 2023-08-11T14:09:31.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:30 smithi103 ceph-mon[130434]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 26 MiB/s wr, 2.39k op/s 2023-08-11T14:09:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:30 smithi003 ceph-mon[97195]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 26 MiB/s wr, 2.39k op/s 2023-08-11T14:09:31.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:30 smithi187 ceph-mon[130973]: pgmap v515: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 26 MiB/s wr, 2.39k op/s 2023-08-11T14:09:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 -- 172.21.15.3:0/3466407762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd01066b0 msgr2=0x7fccd0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 --2- 172.21.15.3:0/3466407762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd01066b0 0x7fccd0108ac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009f90 tx=0x7fccc800b330 comp rx=0 tx=0).stop 2023-08-11T14:09:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 -- 172.21.15.3:0/3466407762 shutdown_connections 2023-08-11T14:09:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 --2- 172.21.15.3:0/3466407762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd01066b0 0x7fccd0108ac0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 --2- 172.21.15.3:0/3466407762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd0101450 0x7fccd0103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 --2- 172.21.15.3:0/3466407762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd0103d80 0x7fccd0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.509+0000 7fccd5dff700 1 -- 172.21.15.3:0/3466407762 >> 172.21.15.3:0/3466407762 conn(0x7fccd00fade0 msgr2=0x7fccd00fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.515+0000 7fccd5dff700 1 -- 172.21.15.3:0/3466407762 shutdown_connections 2023-08-11T14:09:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.515+0000 7fccd5dff700 1 -- 172.21.15.3:0/3466407762 wait complete. 2023-08-11T14:09:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.517+0000 7fccd5dff700 1 Processor -- start 2023-08-11T14:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.517+0000 7fccd5dff700 1 -- start start 2023-08-11T14:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd0103d80 0x7fccd010c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd01066b0 0x7fccd0072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd0072770 con 0x7fccd0103d80 2023-08-11T14:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00728e0 con 0x7fccd01066b0 2023-08-11T14:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fccd5dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd010cef0 con 0x7fccd0101450 2023-08-11T14:09:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fcccffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd01066b0 0x7fccd0072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39604/0 (socket says 172.21.15.3:39604) 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.518+0000 7fcccf7fe700 1 -- 172.21.15.3:0/1665779241 learned_addr learned my addr 172.21.15.3:0/1665779241 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 -- 172.21.15.3:0/1665779241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd01066b0 msgr2=0x7fccd0072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd01066b0 0x7fccd0072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 -- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd0103d80 msgr2=0x7fccd010c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd0103d80 0x7fccd010c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 -- 172.21.15.3:0/1665779241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009c40 con 0x7fccd0101450 2023-08-11T14:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.519+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9c0 tx=0x7fccc000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.520+0000 7fccccff9700 1 -- 172.21.15.3:0/1665779241 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000cbd0 con 0x7fccd0101450 2023-08-11T14:09:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.520+0000 7fccccff9700 1 -- 172.21.15.3:0/1665779241 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc000eeb0 con 0x7fccd0101450 2023-08-11T14:09:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.520+0000 7fccccff9700 1 -- 172.21.15.3:0/1665779241 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0005550 con 0x7fccd0101450 2023-08-11T14:09:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.520+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd010d1d0 con 0x7fccd0101450 2023-08-11T14:09:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.520+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd010d6e0 con 0x7fccd0101450 2023-08-11T14:09:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.526+0000 7fccccff9700 1 -- 172.21.15.3:0/1665779241 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fccc0004750 con 0x7fccd0101450 2023-08-11T14:09:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.528+0000 7fccccff9700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 0x7fccb80a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.528+0000 7fccccff9700 1 -- 172.21.15.3:0/1665779241 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccc0014070 con 0x7fccd0101450 2023-08-11T14:09:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.529+0000 7fccceffd700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 0x7fccb80a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.530+0000 7fccceffd700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 0x7fccb80a2610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005950 tx=0x7fccc40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.569+0000 7fcce96f3b80 1 Processor -- start 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.569+0000 7fcce96f3b80 1 -- start start 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.569+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 0x55c8bae02370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.569+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 0x55c8bae031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.569+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae03c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bad4a5d0 con 0x55c8bae01f50 2023-08-11T14:09:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bad47ef0 con 0x55c8bae028b0 2023-08-11T14:09:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bad437e0 con 0x55c8bae037a0 2023-08-11T14:09:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 0x55c8bae02370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 0x55c8bae02370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50274/0 (socket says 172.21.15.3:50274) 2023-08-11T14:09:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcccf7fe700 1 -- 172.21.15.3:0/2219962158 learned_addr learned my addr 172.21.15.3:0/2219962158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fcccffff700 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae03c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.570+0000 7fccceffd700 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 0x55c8bae031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 -- 172.21.15.3:0/2219962158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 msgr2=0x55c8bae03c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae03c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 -- 172.21.15.3:0/2219962158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 msgr2=0x55c8bae02370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 0x55c8bae02370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 -- 172.21.15.3:0/2219962158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8babc35d0 con 0x55c8bae028b0 2023-08-11T14:09:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.571+0000 7fccceffd700 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 0x55c8bae031c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fccd010cb30 tx=0x7fccc4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12dbab61dc330553 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.572+0000 7fccb5ffb700 1 -- 172.21.15.3:0/2219962158 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40108c0 con 0x55c8bae028b0 2023-08-11T14:09:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.572+0000 7fccb5ffb700 1 -- 172.21.15.3:0/2219962158 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc4025460 con 0x55c8bae028b0 2023-08-11T14:09:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.572+0000 7fccb5ffb700 1 -- 172.21.15.3:0/2219962158 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc401c680 con 0x55c8bae028b0 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.573+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2219962158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 msgr2=0x55c8bae031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.573+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 0x55c8bae031c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fccd010cb30 tx=0x7fccc4011000 comp rx=0 tx=0).stop 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.574+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2219962158 shutdown_connections 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.574+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae03c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.574+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae01f50 0x55c8bae02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.574+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2219962158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae028b0 0x55c8bae031c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.574+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2219962158 >> 172.21.15.3:0/2219962158 conn(0x55c8bad1e8f0 msgr2=0x55c8baded920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.576+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2219962158 shutdown_connections 2023-08-11T14:09:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.576+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2219962158 wait complete. 2023-08-11T14:09:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.577+0000 7fcce96f3b80 1 -- start start 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.577+0000 7fcce96f3b80 1 Processor -- start 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae13230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bace1a60 con 0x55c8bae028b0 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bace1760 con 0x55c8bae037a0 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8babc58c0 con 0x55c8bae01f50 2023-08-11T14:09:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.578+0000 7fcce96f3b80 10 client.0 _ll_register_callbacks cb 0x55c8bace4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39630/0 (socket says 172.21.15.3:39630) 2023-08-11T14:09:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 -- 172.21.15.3:0/836563624 learned_addr learned my addr 172.21.15.3:0/836563624 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccffff700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae13230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 -- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 msgr2=0x55c8bae13230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fccceffd700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae13230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 -- 172.21.15.3:0/836563624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 msgr2=0x55c8bae0c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae0c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.579+0000 7fcccf7fe700 1 -- 172.21.15.3:0/836563624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc400e7f0 con 0x55c8bae01f50 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fcccffff700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae13230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fccceffd700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fccc000cfd0 tx=0x7fccc0099540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870c8e1bf65bbe2a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fccaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc009cc70 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccc00a0c50 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.580+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8bade47e0 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc00b8070 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 10 client.26209 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c8bab6fd80 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8bade47e0 con 0x55c8bae01f50 2023-08-11T14:09:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc00a07c0 con 0x55c8bae01f50 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 20 client.26209 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8badf3cf0 0x55c8badf6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 4 client.26209 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.581+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8badc0480 con 0x55c8badf3cf0 2023-08-11T14:09:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.582+0000 7fccceffd700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8badf3cf0 0x55c8badf6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.582+0000 7fccceffd700 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8badf3cf0 0x55c8badf6110 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf6f7ab4023cd38 server_cookie=f6579eda8ba1897b in_seq=0 out_seq=0 2023-08-11T14:09:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.582+0000 7fccaeffd700 10 client.26209 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.583+0000 7fccaeffd700 1 -- 172.21.15.3:0/836563624 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c8badc0480 con 0x55c8badf3cf0 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.583+0000 7fccaeffd700 10 client.26209 handle_command_reply: tid=0 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.583+0000 7fcce96f3b80 1 client.26209 shutdown 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.583+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 msgr2=0x55c8bae0bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.583+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fccc000cfd0 tx=0x7fccc0099540 comp rx=0 tx=0).stop 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 shutdown_connections 2023-08-11T14:09:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8badf3cf0 0x55c8badf6110 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae13230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae0bcb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/836563624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae0c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 >> 172.21.15.3:0/836563624 conn(0x55c8bad1e8f0 msgr2=0x55c8badec710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fccaeffd700 0 client.26209 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 shutdown_connections 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.584+0000 7fcce96f3b80 1 -- 172.21.15.3:0/836563624 wait complete. 2023-08-11T14:09:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.586+0000 7fcce96f3b80 20 client.26209 trim_cache size 0 max 16384 2023-08-11T14:09:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.627+0000 7fcce96f3b80 1 Processor -- start 2023-08-11T14:09:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 -- start start 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae01f50 0x55c8badfaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bae0b200 con 0x55c8bae028b0 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bade47c0 con 0x55c8bae01f50 2023-08-11T14:09:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.628+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bae02fb0 con 0x55c8bae037a0 2023-08-11T14:09:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:39646/0 (socket says 172.21.15.3:39646) 2023-08-11T14:09:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 -- 172.21.15.3:0/2443737593 learned_addr learned my addr 172.21.15.3:0/2443737593 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 -- 172.21.15.3:0/2443737593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae01f50 msgr2=0x55c8badfaa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:09:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae01f50 0x55c8badfaa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 -- 172.21.15.3:0/2443737593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 msgr2=0x55c8bae131f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:09:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae131f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.629+0000 7fcccffff700 1 -- 172.21.15.3:0/2443737593 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8bab58c00 con 0x55c8bae037a0 2023-08-11T14:09:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.631+0000 7fcccffff700 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009f90 tx=0x7fccc8007420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ddd2b3fcf11a761 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.631+0000 7fccb67fc700 1 -- 172.21.15.3:0/2443737593 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800e040 con 0x55c8bae037a0 2023-08-11T14:09:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.631+0000 7fccb67fc700 1 -- 172.21.15.3:0/2443737593 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc8021070 con 0x55c8bae037a0 2023-08-11T14:09:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.631+0000 7fccb67fc700 1 -- 172.21.15.3:0/2443737593 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800be90 con 0x55c8bae037a0 2023-08-11T14:09:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.632+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2443737593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 msgr2=0x55c8bae066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.632+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009f90 tx=0x7fccc8007420 comp rx=0 tx=0).stop 2023-08-11T14:09:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.633+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2443737593 shutdown_connections 2023-08-11T14:09:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.633+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae037a0 0x55c8bae066f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.633+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae01f50 0x55c8badfaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.633+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2443737593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.633+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2443737593 >> 172.21.15.3:0/2443737593 conn(0x55c8bad1e8f0 msgr2=0x55c8bae0a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.634+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2443737593 shutdown_connections 2023-08-11T14:09:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.634+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2443737593 wait complete. 2023-08-11T14:09:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.634+0000 7fcce96f3b80 1 -- start start 2023-08-11T14:09:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.635+0000 7fcce96f3b80 1 Processor -- start 2023-08-11T14:09:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.635+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.635+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcce96f3b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bae12720 con 0x55c8bae028b0 2023-08-11T14:09:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bae12890 con 0x55c8bae037a0 2023-08-11T14:09:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcce96f3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8bae12a50 con 0x55c8bae01f50 2023-08-11T14:09:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcce96f3b80 10 client.0 _ll_register_callbacks cb 0x55c8bace4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.636+0000 7fcccffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.637+0000 7fcccffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42742/0 (socket says 172.21.15.3:42742) 2023-08-11T14:09:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.637+0000 7fcccffff700 1 -- 172.21.15.3:0/2114842204 learned_addr learned my addr 172.21.15.3:0/2114842204 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.637+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.637+0000 7fccceffd700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 -- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 msgr2=0x55c8bae121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 -- 172.21.15.3:0/2114842204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 msgr2=0x55c8bae111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 -- 172.21.15.3:0/2114842204 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01096b0 con 0x55c8bae01f50 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fcccf7fe700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae10c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fccc00994a0 tx=0x7fccc00987a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d724d6178a375677 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.638+0000 7fccaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.639+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0098180 con 0x55c8bae01f50 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.639+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccc00a0c50 con 0x55c8bae01f50 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.639+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc4025460 con 0x55c8bae01f50 2023-08-11T14:09:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.640+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc009cc70 con 0x55c8bae01f50 2023-08-11T14:09:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.646+0000 7fcce96f3b80 10 client.26215 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.646+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c8bab961f0 con 0x55c8bae01f50 2023-08-11T14:09:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc4025460 con 0x55c8bae01f50 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc00b8160 con 0x55c8bae01f50 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 20 client.26215 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8bae7d210 0x55c8bae7f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 4 client.26215 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:09:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.666+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8bad3c7c0 con 0x55c8bae7d210 2023-08-11T14:09:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.669+0000 7fccceffd700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8bae7d210 0x55c8bae7f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.676+0000 7fccceffd700 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8bae7d210 0x55c8bae7f630 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c62aa52f1ae763b server_cookie=fb4760ee293fe454 in_seq=0 out_seq=0 2023-08-11T14:09:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.676+0000 7fccaeffd700 10 client.26215 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fccaeffd700 1 -- 172.21.15.3:0/2114842204 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8bad3c7c0 con 0x55c8bae7d210 2023-08-11T14:09:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fccaeffd700 10 client.26215 handle_command_reply: tid=0 2023-08-11T14:09:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 client.26215 shutdown 2023-08-11T14:09:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 msgr2=0x55c8bae10c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae10c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fccc00994a0 tx=0x7fccc00987a0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 shutdown_connections 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c8bae7d210 0x55c8bae7f630 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8bae037a0 0x55c8bae121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c8bae01f50 0x55c8bae10c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 --2- 172.21.15.3:0/2114842204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c8bae028b0 0x55c8bae111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.743+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 >> 172.21.15.3:0/2114842204 conn(0x55c8bad1e8f0 msgr2=0x55c8bae066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fccaeffd700 0 client.26215 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 shutdown_connections 2023-08-11T14:09:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fcce96f3b80 1 -- 172.21.15.3:0/2114842204 wait complete. 2023-08-11T14:09:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fcce96f3b80 20 client.26215 trim_cache size 0 max 16384 2023-08-11T14:09:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:09:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:09:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:09:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:09:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 msgr2=0x7fccb80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 0x7fccb80a2610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005950 tx=0x7fccc40058e0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 msgr2=0x7fccd010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.744+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9c0 tx=0x7fccc000ecd0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 shutdown_connections 2023-08-11T14:09:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccb80a0150 0x7fccb80a2610 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd01066b0 0x7fccd0072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd0101450 0x7fccd010bae0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 --2- 172.21.15.3:0/1665779241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd0103d80 0x7fccd010c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 >> 172.21.15.3:0/1665779241 conn(0x7fccd00fade0 msgr2=0x7fccd0106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 shutdown_connections 2023-08-11T14:09:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:32.745+0000 7fccd5dff700 1 -- 172.21.15.3:0/1665779241 wait complete. 2023-08-11T14:09:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 155.0 s, 912.9 tps, lat 109.173 ms stddev 191.168 2023-08-11T14:09:34.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:33 smithi003 ceph-mon[97195]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-08-11T14:09:34.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:33 smithi187 ceph-mon[130973]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-08-11T14:09:34.185 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:33 smithi103 ceph-mon[130434]: pgmap v516: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 28 MiB/s wr, 2.44k op/s 2023-08-11T14:09:36.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:35 smithi187 ceph-mon[130973]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 27 MiB/s wr, 2.03k op/s 2023-08-11T14:09:36.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:35 smithi003 ceph-mon[97195]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 27 MiB/s wr, 2.03k op/s 2023-08-11T14:09:36.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:35 smithi103 ceph-mon[130434]: pgmap v517: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 27 MiB/s wr, 2.03k op/s 2023-08-11T14:09:38.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:37 smithi187 ceph-mon[130973]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.83k op/s 2023-08-11T14:09:38.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:37 smithi003 ceph-mon[97195]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.83k op/s 2023-08-11T14:09:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 160.0 s, 995.3 tps, lat 101.577 ms stddev 285.746 2023-08-11T14:09:38.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:37 smithi103 ceph-mon[130434]: pgmap v518: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 25 MiB/s wr, 1.83k op/s 2023-08-11T14:09:39.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:38 smithi103 ceph-mon[130434]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.52k op/s 2023-08-11T14:09:39.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:38 smithi003 ceph-mon[97195]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.52k op/s 2023-08-11T14:09:39.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:38 smithi187 ceph-mon[130973]: pgmap v519: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 24 MiB/s wr, 1.52k op/s 2023-08-11T14:09:41.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:41 smithi003 ceph-mon[97195]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 25 MiB/s wr, 1.56k op/s 2023-08-11T14:09:41.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:41 smithi187 ceph-mon[130973]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 25 MiB/s wr, 1.56k op/s 2023-08-11T14:09:41.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:41 smithi103 ceph-mon[130434]: pgmap v520: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 25 MiB/s wr, 1.56k op/s 2023-08-11T14:09:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 165.0 s, 1045.6 tps, lat 95.617 ms stddev 208.998 2023-08-11T14:09:44.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:43 smithi003 ceph-mon[97195]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-08-11T14:09:44.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:43 smithi187 ceph-mon[130973]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-08-11T14:09:44.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:43 smithi103 ceph-mon[130434]: pgmap v521: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 28 MiB/s wr, 1.55k op/s 2023-08-11T14:09:46.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:45 smithi003 ceph-mon[97195]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 28 MiB/s wr, 1.73k op/s 2023-08-11T14:09:46.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:45 smithi187 ceph-mon[130973]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 28 MiB/s wr, 1.73k op/s 2023-08-11T14:09:46.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:45 smithi103 ceph-mon[130434]: pgmap v522: 129 pgs: 129 active+clean; 8.4 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 28 MiB/s wr, 1.73k op/s 2023-08-11T14:09:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:46 smithi103 ceph-mon[130434]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 25 MiB/s wr, 1.76k op/s 2023-08-11T14:09:47.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:46 smithi003 ceph-mon[97195]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 25 MiB/s wr, 1.76k op/s 2023-08-11T14:09:47.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:46 smithi187 ceph-mon[130973]: pgmap v523: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 25 MiB/s wr, 1.76k op/s 2023-08-11T14:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.672+0000 7fca49fff700 1 -- 172.21.15.3:0/1607739105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca4410a290 msgr2=0x7fca4410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.672+0000 7fca49fff700 1 --2- 172.21.15.3:0/1607739105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca4410a290 0x7fca4410c720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca3401a9a0 tx=0x7fca3401acb0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.673+0000 7fca49fff700 1 -- 172.21.15.3:0/1607739105 shutdown_connections 2023-08-11T14:09:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.673+0000 7fca49fff700 1 --2- 172.21.15.3:0/1607739105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 0x7fca4410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.673+0000 7fca49fff700 1 --2- 172.21.15.3:0/1607739105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca44107920 0x7fca44109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.673+0000 7fca49fff700 1 --2- 172.21.15.3:0/1607739105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca4410a290 0x7fca4410c720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.674+0000 7fca49fff700 1 -- 172.21.15.3:0/1607739105 >> 172.21.15.3:0/1607739105 conn(0x7fca4406d750 msgr2=0x7fca4406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.675+0000 7fca49fff700 1 -- 172.21.15.3:0/1607739105 shutdown_connections 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.678+0000 7fca49fff700 1 -- 172.21.15.3:0/1607739105 wait complete. 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.680+0000 7fca49fff700 1 Processor -- start 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.680+0000 7fca49fff700 1 -- start start 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca44107920 0x7fca4419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 0x7fca4419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419e810 con 0x7fca4410a290 2023-08-11T14:09:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419e980 con 0x7fca4410cc60 2023-08-11T14:09:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.681+0000 7fca49fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419eb40 con 0x7fca44107920 2023-08-11T14:09:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34762/0 (socket says 172.21.15.3:34762) 2023-08-11T14:09:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca437fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca44107920 0x7fca4419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 -- 172.21.15.3:0/4140066539 learned_addr learned my addr 172.21.15.3:0/4140066539 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca43fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 0x7fca4419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 -- 172.21.15.3:0/4140066539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca44107920 msgr2=0x7fca4419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca44107920 0x7fca4419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 -- 172.21.15.3:0/4140066539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 msgr2=0x7fca4419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 0x7fca4419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.682+0000 7fca42ffd700 1 -- 172.21.15.3:0/4140066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3401a600 con 0x7fca4410a290 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.683+0000 7fca42ffd700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fca34000c00 tx=0x7fca34003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.686+0000 7fca40ff9700 1 -- 172.21.15.3:0/4140066539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34004200 con 0x7fca4410a290 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.686+0000 7fca40ff9700 1 -- 172.21.15.3:0/4140066539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34004360 con 0x7fca4410a290 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.686+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4419edc0 con 0x7fca4410a290 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.686+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441c7540 con 0x7fca4410a290 2023-08-11T14:09:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.686+0000 7fca40ff9700 1 -- 172.21.15.3:0/4140066539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3401f7f0 con 0x7fca4410a290 2023-08-11T14:09:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.695+0000 7fca40ff9700 1 -- 172.21.15.3:0/4140066539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fca34018410 con 0x7fca4410a290 2023-08-11T14:09:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.697+0000 7fca40ff9700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 0x7fca2c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.697+0000 7fca40ff9700 1 -- 172.21.15.3:0/4140066539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fca340cdd70 con 0x7fca4410a290 2023-08-11T14:09:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.701+0000 7fca437fe700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 0x7fca2c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.718+0000 7fca437fe700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 0x7fca2c0a2610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fca4419db90 tx=0x7fca3c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 Processor -- start 2023-08-11T14:09:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 -- start start 2023-08-11T14:09:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf5a75d0 con 0x562ecf64f790 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf5a4ef0 con 0x562ecf64df40 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.720+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf5a07e0 con 0x562ecf64e8a0 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.721+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.721+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50714/0 (socket says 172.21.15.3:50714) 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.721+0000 7fca42ffd700 1 -- 172.21.15.3:0/3001504622 learned_addr learned my addr 172.21.15.3:0/3001504622 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca43fff700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 -- 172.21.15.3:0/3001504622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 msgr2=0x562ecf64f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 -- 172.21.15.3:0/3001504622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 msgr2=0x562ecf64fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.722+0000 7fca437fe700 1 -- 172.21.15.3:0/3001504622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ecf4205d0 con 0x562ecf64df40 2023-08-11T14:09:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.723+0000 7fca42ffd700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:09:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.723+0000 7fca43fff700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:09:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.723+0000 7fca437fe700 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64e360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d20 tx=0x7fca3c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f4be6abe302835 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.723+0000 7fca29ffb700 1 -- 172.21.15.3:0/3001504622 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c01c070 con 0x562ecf64df40 2023-08-11T14:09:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.726+0000 7fca29ffb700 1 -- 172.21.15.3:0/3001504622 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c017b50 con 0x562ecf64df40 2023-08-11T14:09:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.726+0000 7fca29ffb700 1 -- 172.21.15.3:0/3001504622 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c011070 con 0x562ecf64df40 2023-08-11T14:09:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.728+0000 7fca5da1eb80 1 -- 172.21.15.3:0/3001504622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 msgr2=0x562ecf64e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.728+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64e360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d20 tx=0x7fca3c011000 comp rx=0 tx=0).stop 2023-08-11T14:09:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.733+0000 7fca5da1eb80 1 -- 172.21.15.3:0/3001504622 shutdown_connections 2023-08-11T14:09:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.733+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.733+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64e360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.733+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/3001504622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf64f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.733+0000 7fca5da1eb80 1 -- 172.21.15.3:0/3001504622 >> 172.21.15.3:0/3001504622 conn(0x562ecf57b8f0 msgr2=0x562ecf64a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.737+0000 7fca5da1eb80 1 -- 172.21.15.3:0/3001504622 shutdown_connections 2023-08-11T14:09:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.737+0000 7fca5da1eb80 1 -- 172.21.15.3:0/3001504622 wait complete. 2023-08-11T14:09:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.738+0000 7fca5da1eb80 1 -- start start 2023-08-11T14:09:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.738+0000 7fca5da1eb80 1 Processor -- start 2023-08-11T14:09:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64e8a0 0x562ecf658180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64f790 0x562ecf65f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf53ea60 con 0x562ecf64e8a0 2023-08-11T14:09:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf53e760 con 0x562ecf64f790 2023-08-11T14:09:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf4228c0 con 0x562ecf64df40 2023-08-11T14:09:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.739+0000 7fca5da1eb80 10 client.0 _ll_register_callbacks cb 0x562ecf541a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.740+0000 7fca437fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.740+0000 7fca437fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50718/0 (socket says 172.21.15.3:50718) 2023-08-11T14:09:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.740+0000 7fca437fe700 1 -- 172.21.15.3:0/2048205764 learned_addr learned my addr 172.21.15.3:0/2048205764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.743+0000 7fca437fe700 1 -- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64f790 msgr2=0x562ecf65f1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:09:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.743+0000 7fca437fe700 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64f790 0x562ecf65f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.743+0000 7fca437fe700 1 -- 172.21.15.3:0/2048205764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64e8a0 msgr2=0x562ecf658180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.743+0000 7fca437fe700 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64e8a0 0x562ecf658180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.743+0000 7fca437fe700 1 -- 172.21.15.3:0/2048205764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441c7540 con 0x562ecf64df40 2023-08-11T14:09:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.744+0000 7fca437fe700 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca3c013040 tx=0x7fca3c022f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef903c728ea3d877 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.744+0000 7fca22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:09:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.745+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c031070 con 0x562ecf64df40 2023-08-11T14:09:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.745+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fca3c0227e0 con 0x562ecf64df40 2023-08-11T14:09:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.745+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ecf6417e0 con 0x562ecf64df40 2023-08-11T14:09:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.745+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c01c040 con 0x562ecf64df40 2023-08-11T14:09:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.746+0000 7fca5da1eb80 10 client.26227 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.746+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fca3c00cd60 con 0x562ecf64df40 2023-08-11T14:09:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.746+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ecf6417e0 con 0x562ecf64df40 2023-08-11T14:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.747+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca3c034070 con 0x562ecf64df40 2023-08-11T14:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.747+0000 7fca5da1eb80 10 client.26227 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.747+0000 7fca5da1eb80 10 client.26227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.747+0000 7fca5da1eb80 20 client.26227 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.748+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf663930 0x562ecf665d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.748+0000 7fca5da1eb80 4 client.26227 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.748+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ecf61d480 con 0x562ecf663930 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.748+0000 7fca42ffd700 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf663930 0x562ecf665d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.749+0000 7fca42ffd700 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf663930 0x562ecf665d50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabf81bdfac38e43 server_cookie=c0ef3ae4b04d49f1 in_seq=0 out_seq=0 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.749+0000 7fca22ffd700 10 client.26227 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.750+0000 7fca22ffd700 1 -- 172.21.15.3:0/2048205764 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562ecf61d480 con 0x562ecf663930 2023-08-11T14:09:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.750+0000 7fca22ffd700 10 client.26227 handle_command_reply: tid=0 2023-08-11T14:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.751+0000 7fca5da1eb80 1 client.26227 shutdown 2023-08-11T14:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.751+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 msgr2=0x562ecf657c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.751+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca3c013040 tx=0x7fca3c022f40 comp rx=0 tx=0).stop 2023-08-11T14:09:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 shutdown_connections 2023-08-11T14:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf663930 0x562ecf665d50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64f790 0x562ecf65f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64df40 0x562ecf657c40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2048205764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64e8a0 0x562ecf658180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 >> 172.21.15.3:0/2048205764 conn(0x562ecf57b8f0 msgr2=0x562ecf649710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca22ffd700 0 client.26227 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 shutdown_connections 2023-08-11T14:09:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2048205764 wait complete. 2023-08-11T14:09:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.752+0000 7fca5da1eb80 20 client.26227 trim_cache size 0 max 16384 2023-08-11T14:09:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.776+0000 7fca5da1eb80 1 Processor -- start 2023-08-11T14:09:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 -- start start 2023-08-11T14:09:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf6417c0 con 0x562ecf64f790 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf66ab50 con 0x562ecf64df40 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.777+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf656390 con 0x562ecf64e8a0 2023-08-11T14:09:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.778+0000 7fca437fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.778+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.778+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50734/0 (socket says 172.21.15.3:50734) 2023-08-11T14:09:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.778+0000 7fca42ffd700 1 -- 172.21.15.3:0/1969855927 learned_addr learned my addr 172.21.15.3:0/1969855927 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.779+0000 7fca42ffd700 1 -- 172.21.15.3:0/1969855927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 msgr2=0x562ecf64a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.779+0000 7fca42ffd700 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.779+0000 7fca42ffd700 1 -- 172.21.15.3:0/1969855927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 msgr2=0x562ecf64a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.779+0000 7fca42ffd700 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.779+0000 7fca42ffd700 1 -- 172.21.15.3:0/1969855927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ecf3ccd80 con 0x562ecf64e8a0 2023-08-11T14:09:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.780+0000 7fca42ffd700 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca3409b260 tx=0x7fca340cb450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9a2dae73f1dff5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.785+0000 7fca2a7fc700 1 -- 172.21.15.3:0/1969855927 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3409bc60 con 0x562ecf64e8a0 2023-08-11T14:09:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.785+0000 7fca2a7fc700 1 -- 172.21.15.3:0/1969855927 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3409b560 con 0x562ecf64e8a0 2023-08-11T14:09:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.785+0000 7fca2a7fc700 1 -- 172.21.15.3:0/1969855927 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340a10d0 con 0x562ecf64e8a0 2023-08-11T14:09:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.786+0000 7fca5da1eb80 1 -- 172.21.15.3:0/1969855927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 msgr2=0x562ecf65f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.786+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca3409b260 tx=0x7fca340cb450 comp rx=0 tx=0).stop 2023-08-11T14:09:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 -- 172.21.15.3:0/1969855927 shutdown_connections 2023-08-11T14:09:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf64a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf64a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/1969855927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65f150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 -- 172.21.15.3:0/1969855927 >> 172.21.15.3:0/1969855927 conn(0x562ecf57b8f0 msgr2=0x562ecf652610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 -- 172.21.15.3:0/1969855927 shutdown_connections 2023-08-11T14:09:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.787+0000 7fca5da1eb80 1 -- 172.21.15.3:0/1969855927 wait complete. 2023-08-11T14:09:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.788+0000 7fca5da1eb80 1 -- start start 2023-08-11T14:09:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.788+0000 7fca5da1eb80 1 Processor -- start 2023-08-11T14:09:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.788+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf65ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf65e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf65e6f0 con 0x562ecf64f790 2023-08-11T14:09:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf65e860 con 0x562ecf64df40 2023-08-11T14:09:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ecf65ea20 con 0x562ecf64e8a0 2023-08-11T14:09:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca5da1eb80 10 client.0 _ll_register_callbacks cb 0x7fca3401a0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:09:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:50746/0 (socket says 172.21.15.3:50746) 2023-08-11T14:09:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca42ffd700 1 -- 172.21.15.3:0/2382404619 learned_addr learned my addr 172.21.15.3:0/2382404619 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:09:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.789+0000 7fca437fe700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf65ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 -- 172.21.15.3:0/2382404619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 msgr2=0x562ecf65b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca43fff700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf65e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 -- 172.21.15.3:0/2382404619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 msgr2=0x562ecf65e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:09:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf65e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 -- 172.21.15.3:0/2382404619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3402c020 con 0x562ecf64df40 2023-08-11T14:09:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca43fff700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf65e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:09:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca437fe700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf65ac50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0174e0 tx=0x7fca3c0229f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55cb253440c3fd47 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:09:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.790+0000 7fca22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:09:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.793+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c031050 con 0x562ecf64df40 2023-08-11T14:09:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.793+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fca3c02c3e0 con 0x562ecf64df40 2023-08-11T14:09:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.793+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca3c017b50 con 0x562ecf64df40 2023-08-11T14:09:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.794+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c01c040 con 0x562ecf64df40 2023-08-11T14:09:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.795+0000 7fca5da1eb80 10 client.26246 fetch_fsmap learned FSMap version 33 2023-08-11T14:09:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.795+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562ecf3b5c00 con 0x562ecf64df40 2023-08-11T14:09:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c017b50 con 0x562ecf64df40 2023-08-11T14:09:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fca3c0177f0 con 0x562ecf64df40 2023-08-11T14:09:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 10 client.26246 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:09:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:09:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 20 client.26246 populate_metadata read hostname 'smithi003' 2023-08-11T14:09:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf6b7fa0 0x562ecf6ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:09:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 4 client.26246 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:09:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.796+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ecf5997c0 con 0x562ecf6b7fa0 2023-08-11T14:09:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.797+0000 7fca42ffd700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf6b7fa0 0x562ecf6ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:09:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.798+0000 7fca42ffd700 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf6b7fa0 0x562ecf6ba3c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f006885eba2793 server_cookie=51098abf4ec6c91 in_seq=0 out_seq=0 2023-08-11T14:09:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.798+0000 7fca22ffd700 10 client.26246 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.972+0000 7fca22ffd700 1 -- 172.21.15.3:0/2382404619 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ecf5997c0 con 0x562ecf6b7fa0 2023-08-11T14:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.973+0000 7fca22ffd700 10 client.26246 handle_command_reply: tid=0 2023-08-11T14:09:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.978+0000 7fca5da1eb80 1 client.26246 shutdown 2023-08-11T14:09:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.993+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 msgr2=0x562ecf65ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.993+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf65ac50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0174e0 tx=0x7fca3c0229f0 comp rx=0 tx=0).stop 2023-08-11T14:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 shutdown_connections 2023-08-11T14:09:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ecf64f790 0x562ecf65e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562ecf6b7fa0 0x562ecf6ba3c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ecf64df40 0x562ecf65ac50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 --2- 172.21.15.3:0/2382404619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ecf64e8a0 0x562ecf65b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 >> 172.21.15.3:0/2382404619 conn(0x562ecf57b8f0 msgr2=0x562ecf652610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca22ffd700 0 client.26246 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:09:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.998+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 shutdown_connections 2023-08-11T14:09:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:47.999+0000 7fca5da1eb80 1 -- 172.21.15.3:0/2382404619 wait complete. 2023-08-11T14:09:48.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.002+0000 7fca5da1eb80 20 client.26246 trim_cache size 0 max 16384 2023-08-11T14:09:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:09:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:09:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:09:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:09:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 msgr2=0x7fca2c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 0x7fca2c0a2610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fca4419db90 tx=0x7fca3c0094d0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 msgr2=0x7fca4419d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:09:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fca34000c00 tx=0x7fca34003bd0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 shutdown_connections 2023-08-11T14:09:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fca2c0a0150 0x7fca2c0a2610 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca4410cc60 0x7fca4419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca44107920 0x7fca4419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 --2- 172.21.15.3:0/4140066539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca4410a290 0x7fca4419d2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:09:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.008+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 >> 172.21.15.3:0/4140066539 conn(0x7fca4406d750 msgr2=0x7fca4410ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:09:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.010+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 shutdown_connections 2023-08-11T14:09:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:09:48.012+0000 7fca49fff700 1 -- 172.21.15.3:0/4140066539 wait complete. 2023-08-11T14:09:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 170.0 s, 1102.4 tps, lat 90.546 ms stddev 202.344 2023-08-11T14:09:50.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:49 smithi003 ceph-mon[97195]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-08-11T14:09:50.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:49 smithi187 ceph-mon[130973]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-08-11T14:09:50.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:49 smithi103 ceph-mon[130434]: pgmap v524: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 27 MiB/s wr, 1.80k op/s 2023-08-11T14:09:52.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:51 smithi003 ceph-mon[97195]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-08-11T14:09:52.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:51 smithi187 ceph-mon[130973]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-08-11T14:09:52.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:51 smithi103 ceph-mon[130434]: pgmap v525: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 1.89k op/s 2023-08-11T14:09:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 175.0 s, 835.2 tps, lat 119.143 ms stddev 246.324 2023-08-11T14:09:54.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:53 smithi103 ceph-mon[130434]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 30 MiB/s wr, 2.08k op/s 2023-08-11T14:09:54.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:53 smithi003 ceph-mon[97195]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 30 MiB/s wr, 2.08k op/s 2023-08-11T14:09:54.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:53 smithi187 ceph-mon[130973]: pgmap v526: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 30 MiB/s wr, 2.08k op/s 2023-08-11T14:09:55.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:54 smithi103 ceph-mon[130434]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.11k op/s 2023-08-11T14:09:55.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:54 smithi003 ceph-mon[97195]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.11k op/s 2023-08-11T14:09:55.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:54 smithi187 ceph-mon[130973]: pgmap v527: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 2.11k op/s 2023-08-11T14:09:57.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:57 smithi103 ceph-mon[130434]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-08-11T14:09:58.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:57 smithi003 ceph-mon[97195]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-08-11T14:09:58.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:57 smithi187 ceph-mon[130973]: pgmap v528: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 24 MiB/s wr, 1.83k op/s 2023-08-11T14:09:58.089 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 180.0 s, 561.5 tps, lat 178.640 ms stddev 347.155 2023-08-11T14:10:00.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:09:59 smithi103 ceph-mon[130434]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 1.99k op/s 2023-08-11T14:10:00.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:09:59 smithi003 ceph-mon[97195]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 1.99k op/s 2023-08-11T14:10:00.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:10:00.000+0000 7f7a41abb700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:10:00.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:10:00.000+0000 7f7a41abb700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-08-11T14:10:00.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:10:00.000+0000 7f7a41abb700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2023-08-11T14:10:00.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:09:59 smithi187 ceph-mon[130973]: pgmap v529: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 26 MiB/s wr, 1.99k op/s 2023-08-11T14:10:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:00 smithi103 ceph-mon[130434]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:10:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:00 smithi103 ceph-mon[130434]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-08-11T14:10:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:00 smithi103 ceph-mon[130434]: mds.i(mds.1): Metadata damage detected 2023-08-11T14:10:01.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-mon[97195]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:10:01.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-mon[97195]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-08-11T14:10:01.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:00 smithi003 ceph-mon[97195]: mds.i(mds.1): Metadata damage detected 2023-08-11T14:10:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:00 smithi187 ceph-mon[130973]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:10:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:00 smithi187 ceph-mon[130973]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-08-11T14:10:01.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:00 smithi187 ceph-mon[130973]: mds.i(mds.1): Metadata damage detected 2023-08-11T14:10:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.616+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3104744160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 msgr2=0x7fd818067d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.616+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3104744160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818067d10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b0a0 tx=0x7fd81000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.617+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3104744160 shutdown_connections 2023-08-11T14:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.617+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3104744160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818067d10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.617+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3104744160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8180ff820 0x7fd8180ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.617+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3104744160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd818100a10 0x7fd818100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.617+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3104744160 >> 172.21.15.3:0/3104744160 conn(0x7fd8180fad70 msgr2=0x7fd8180fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.631+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3104744160 shutdown_connections 2023-08-11T14:10:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.636+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3104744160 wait complete. 2023-08-11T14:10:01.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.638+0000 7fd81c9ff700 1 Processor -- start 2023-08-11T14:10:01.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 -- start start 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8180ff820 0x7fd81810f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd818100a10 0x7fd81810fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818116610 con 0x7fd8180ff820 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818116780 con 0x7fd8181013b0 2023-08-11T14:10:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.639+0000 7fd81c9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818116940 con 0x7fd818100a10 2023-08-11T14:10:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.640+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.640+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41376/0 (socket says 172.21.15.3:41376) 2023-08-11T14:10:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.640+0000 7fd816d9d700 1 -- 172.21.15.3:0/3948657964 learned_addr learned my addr 172.21.15.3:0/3948657964 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.641+0000 7fd816d9d700 1 -- 172.21.15.3:0/3948657964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd818100a10 msgr2=0x7fd81810fe80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.641+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd818100a10 0x7fd81810fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.641+0000 7fd816d9d700 1 -- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8180ff820 msgr2=0x7fd81810f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.641+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8180ff820 0x7fd81810f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.641+0000 7fd816d9d700 1 -- 172.21.15.3:0/3948657964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810009d00 con 0x7fd8181013b0 2023-08-11T14:10:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.642+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd81000bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.643+0000 7fd8077fe700 1 -- 172.21.15.3:0/3948657964 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000e040 con 0x7fd8181013b0 2023-08-11T14:10:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.643+0000 7fd8077fe700 1 -- 172.21.15.3:0/3948657964 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd810003e10 con 0x7fd8181013b0 2023-08-11T14:10:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.643+0000 7fd8077fe700 1 -- 172.21.15.3:0/3948657964 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81001b820 con 0x7fd8181013b0 2023-08-11T14:10:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.644+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd818110e90 con 0x7fd8181013b0 2023-08-11T14:10:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.644+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd818111340 con 0x7fd8181013b0 2023-08-11T14:10:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.652+0000 7fd8302f6b80 1 Processor -- start 2023-08-11T14:10:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.655+0000 7fd8302f6b80 1 -- start start 2023-08-11T14:10:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.655+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.655+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd78a0 0x55d1eebd81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.656+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.656+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eeb305d0 con 0x55d1eebd8790 2023-08-11T14:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.656+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eeb2def0 con 0x55d1eebd78a0 2023-08-11T14:10:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.656+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eeb297e0 con 0x55d1eebd6f40 2023-08-11T14:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.656+0000 7fd8077fe700 1 -- 172.21.15.3:0/3948657964 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd81001da30 con 0x7fd8181013b0 2023-08-11T14:10:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.657+0000 7fd815d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.657+0000 7fd815d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51092/0 (socket says 172.21.15.3:51092) 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.657+0000 7fd815d9b700 1 -- 172.21.15.3:0/216546899 learned_addr learned my addr 172.21.15.3:0/216546899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.657+0000 7fd815d9b700 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 -- 172.21.15.3:0/216546899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd78a0 msgr2=0x55d1eebd81b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd78a0 0x55d1eebd81b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 -- 172.21.15.3:0/216546899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 msgr2=0x55d1eebd8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebd8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 -- 172.21.15.3:0/216546899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ee9a95d0 con 0x55d1eebd6f40 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd815d9b700 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd808001b20 tx=0x7fd8080011e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6f06ceda940b3c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd8077fe700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 0x7fd8000a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.658+0000 7fd804ff9700 1 -- 172.21.15.3:0/216546899 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808004a70 con 0x55d1eebd6f40 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.659+0000 7fd8077fe700 1 -- 172.21.15.3:0/3948657964 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd8100cc180 con 0x7fd8181013b0 2023-08-11T14:10:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.659+0000 7fd804ff9700 1 -- 172.21.15.3:0/216546899 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd808004bd0 con 0x55d1eebd6f40 2023-08-11T14:10:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.659+0000 7fd804ff9700 1 -- 172.21.15.3:0/216546899 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808009020 con 0x55d1eebd6f40 2023-08-11T14:10:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.660+0000 7fd8302f6b80 1 -- 172.21.15.3:0/216546899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 msgr2=0x55d1eebd7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.660+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd808001b20 tx=0x7fd8080011e0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.669+0000 7fd81659c700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 0x7fd8000a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.670+0000 7fd81659c700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 0x7fd8000a25e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005950 tx=0x7fd80c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.674+0000 7fd8302f6b80 1 -- 172.21.15.3:0/216546899 shutdown_connections 2023-08-11T14:10:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.674+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebd8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.674+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd6f40 0x55d1eebd7360 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.674+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/216546899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd78a0 0x55d1eebd81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.674+0000 7fd8302f6b80 1 -- 172.21.15.3:0/216546899 >> 172.21.15.3:0/216546899 conn(0x55d1eeb048f0 msgr2=0x55d1eebd3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.689+0000 7fd8302f6b80 1 -- 172.21.15.3:0/216546899 shutdown_connections 2023-08-11T14:10:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.695+0000 7fd8302f6b80 1 -- 172.21.15.3:0/216546899 wait complete. 2023-08-11T14:10:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.696+0000 7fd8302f6b80 1 -- start start 2023-08-11T14:10:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.696+0000 7fd8302f6b80 1 Processor -- start 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eeac7a60 con 0x55d1eebd8790 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eeac7760 con 0x55d1eebd6f40 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1ee9ab8c0 con 0x55d1eebd78a0 2023-08-11T14:10:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.699+0000 7fd8302f6b80 10 client.0 _ll_register_callbacks cb 0x55d1eeacaa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.700+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.700+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47216/0 (socket says 172.21.15.3:47216) 2023-08-11T14:10:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.700+0000 7fd816d9d700 1 -- 172.21.15.3:0/3888568489 learned_addr learned my addr 172.21.15.3:0/3888568489 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.700+0000 7fd815d9b700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.702+0000 7fd816d9d700 1 -- 172.21.15.3:0/3888568489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 msgr2=0x55d1eebe1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.702+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.702+0000 7fd816d9d700 1 -- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 msgr2=0x55d1eebe0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.702+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.702+0000 7fd816d9d700 1 -- 172.21.15.3:0/3888568489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd808011040 con 0x55d1eebd8790 2023-08-11T14:10:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.703+0000 7fd816d9d700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd810096350 tx=0x7fd8100994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b382ef8d64716cba server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.709+0000 7fd7fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:10:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.709+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810099d40 con 0x55d1eebd8790 2023-08-11T14:10:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.709+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd8100998e0 con 0x55d1eebd8790 2023-08-11T14:10:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.711+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1eebca7e0 con 0x55d1eebd8790 2023-08-11T14:10:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.711+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81009d710 con 0x55d1eebd8790 2023-08-11T14:10:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.716+0000 7fd8302f6b80 10 client.16143 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.716+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d1ee955d80 con 0x55d1eebd8790 2023-08-11T14:10:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.718+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1eebca7e0 con 0x55d1eebd8790 2023-08-11T14:10:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.718+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8100a1210 con 0x55d1eebd8790 2023-08-11T14:10:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 20 client.16143 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec0ea30 0x55d1eec10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 4 client.16143 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:10:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.719+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1eeba6480 con 0x55d1eec0ea30 2023-08-11T14:10:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.726+0000 7fd81659c700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec0ea30 0x55d1eec10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.734+0000 7fd81659c700 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec0ea30 0x55d1eec10e50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9a5cf9b834954c server_cookie=c8c88744a73285be in_seq=0 out_seq=0 2023-08-11T14:10:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.734+0000 7fd7fdffb700 10 client.16143 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.737+0000 7fd7fdffb700 1 -- 172.21.15.3:0/3888568489 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d1eeba6480 con 0x55d1eec0ea30 2023-08-11T14:10:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.737+0000 7fd7fdffb700 10 client.16143 handle_command_reply: tid=0 2023-08-11T14:10:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.737+0000 7fd8302f6b80 1 client.16143 shutdown 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.737+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 msgr2=0x55d1eebe8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.737+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd810096350 tx=0x7fd8100994a0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 shutdown_connections 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd8790 0x55d1eebe8240 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec0ea30 0x55d1eec10e50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3888568489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 >> 172.21.15.3:0/3888568489 conn(0x55d1eeb048f0 msgr2=0x55d1eebd2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd7fdffb700 0 client.16143 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.738+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 shutdown_connections 2023-08-11T14:10:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.739+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3888568489 wait complete. 2023-08-11T14:10:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.741+0000 7fd8302f6b80 20 client.16143 trim_cache size 0 max 16384 2023-08-11T14:10:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 Processor -- start 2023-08-11T14:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 -- start start 2023-08-11T14:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd8790 0x55d1eebdb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eebca7c0 con 0x55d1eebd78a0 2023-08-11T14:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eebccea0 con 0x55d1eebd6f40 2023-08-11T14:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.773+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eec0e800 con 0x55d1eebd8790 2023-08-11T14:10:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47220/0 (socket says 172.21.15.3:47220) 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 -- 172.21.15.3:0/3739805310 learned_addr learned my addr 172.21.15.3:0/3739805310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 -- 172.21.15.3:0/3739805310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd8790 msgr2=0x55d1eebdb6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd8790 0x55d1eebdb6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 -- 172.21.15.3:0/3739805310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 msgr2=0x55d1eebe8a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe8a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.775+0000 7fd815d9b700 1 -- 172.21.15.3:0/3739805310 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1ee93ec00 con 0x55d1eebd78a0 2023-08-11T14:10:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.776+0000 7fd815d9b700 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd808007860 tx=0x7fd808000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83980594d4ceecf3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.780+0000 7fd8057fa700 1 -- 172.21.15.3:0/3739805310 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80801b040 con 0x55d1eebd78a0 2023-08-11T14:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.780+0000 7fd8057fa700 1 -- 172.21.15.3:0/3739805310 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd808007170 con 0x55d1eebd78a0 2023-08-11T14:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.780+0000 7fd8057fa700 1 -- 172.21.15.3:0/3739805310 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808009280 con 0x55d1eebd78a0 2023-08-11T14:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.781+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3739805310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 msgr2=0x55d1eebe8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.781+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd808007860 tx=0x7fd808000f80 comp rx=0 tx=0).stop 2023-08-11T14:10:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.784+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3739805310 shutdown_connections 2023-08-11T14:10:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.784+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd8790 0x55d1eebdb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.784+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd6f40 0x55d1eebe8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.784+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/3739805310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd78a0 0x55d1eebe8200 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.785+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3739805310 >> 172.21.15.3:0/3739805310 conn(0x55d1eeb048f0 msgr2=0x55d1eebe0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.797+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3739805310 shutdown_connections 2023-08-11T14:10:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.797+0000 7fd8302f6b80 1 -- 172.21.15.3:0/3739805310 wait complete. 2023-08-11T14:10:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.798+0000 7fd8302f6b80 1 -- start start 2023-08-11T14:10:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.798+0000 7fd8302f6b80 1 Processor -- start 2023-08-11T14:10:01.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd6f40 0x55d1eebe6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eebe7bb0 con 0x55d1eebd6f40 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eebe7d20 con 0x55d1eebd8790 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1eebe7ee0 con 0x55d1eebd78a0 2023-08-11T14:10:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.801+0000 7fd8302f6b80 10 client.0 _ll_register_callbacks cb 0x55d1eeacaa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41410/0 (socket says 172.21.15.3:41410) 2023-08-11T14:10:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 -- 172.21.15.3:0/582950942 learned_addr learned my addr 172.21.15.3:0/582950942 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 -- 172.21.15.3:0/582950942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 msgr2=0x55d1eebe6660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:10:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe6660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 -- 172.21.15.3:0/582950942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd6f40 msgr2=0x55d1eebe6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd6f40 0x55d1eebe6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.802+0000 7fd816d9d700 1 -- 172.21.15.3:0/582950942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1ee9a95d0 con 0x55d1eebd8790 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.803+0000 7fd816d9d700 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd810099870 tx=0x7fd810099090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad045205b93b44c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.803+0000 7fd7fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.804+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100998e0 con 0x55d1eebd8790 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.804+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd81009d710 con 0x55d1eebd8790 2023-08-11T14:10:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.804+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd808004bd0 con 0x55d1eebd8790 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.804+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd810095840 con 0x55d1eebd8790 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.805+0000 7fd8302f6b80 10 client.26255 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.805+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d1ee97c1f0 con 0x55d1eebd8790 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd808004bd0 con 0x55d1eebd8790 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd8100b7420 con 0x55d1eebd8790 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd8302f6b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd8302f6b80 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd8302f6b80 20 client.26255 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.806+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec63250 0x55d1eec65670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.807+0000 7fd8302f6b80 4 client.26255 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.807+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d1eeb227c0 con 0x55d1eec63250 2023-08-11T14:10:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.808+0000 7fd81659c700 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec63250 0x55d1eec65670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.809+0000 7fd81659c700 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec63250 0x55d1eec65670 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2371b5a56f04db8 server_cookie=2494424ff0cac3f2 in_seq=0 out_seq=0 2023-08-11T14:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.811+0000 7fd7fdffb700 10 client.26255 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.836+0000 7fd7fdffb700 1 -- 172.21.15.3:0/582950942 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d1eeb227c0 con 0x55d1eec63250 2023-08-11T14:10:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.836+0000 7fd7fdffb700 10 client.26255 handle_command_reply: tid=0 2023-08-11T14:10:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.837+0000 7fd8302f6b80 1 client.26255 shutdown 2023-08-11T14:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.837+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 msgr2=0x55d1eebe7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.837+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd810099870 tx=0x7fd810099090 comp rx=0 tx=0).stop 2023-08-11T14:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 shutdown_connections 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d1eec63250 0x55d1eec65670 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1eebd8790 0x55d1eebe7670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d1eebd6f40 0x55d1eebe6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 --2- 172.21.15.3:0/582950942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1eebd78a0 0x55d1eebe6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 >> 172.21.15.3:0/582950942 conn(0x55d1eeb048f0 msgr2=0x55d1eebe8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd7fdffb700 0 client.26255 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 shutdown_connections 2023-08-11T14:10:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.838+0000 7fd8302f6b80 1 -- 172.21.15.3:0/582950942 wait complete. 2023-08-11T14:10:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.839+0000 7fd8302f6b80 20 client.26255 trim_cache size 0 max 16384 2023-08-11T14:10:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.841+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 msgr2=0x7fd8000a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.841+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 0x7fd8000a25e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005950 tx=0x7fd80c0058e0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.841+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 msgr2=0x7fd818115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.841+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd810009fd0 tx=0x7fd81000bd90 comp rx=0 tx=0).stop 2023-08-11T14:10:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 shutdown_connections 2023-08-11T14:10:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8000a0120 0x7fd8000a25e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8181013b0 0x7fd818115fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8180ff820 0x7fd81810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 --2- 172.21.15.3:0/3948657964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd818100a10 0x7fd81810fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 >> 172.21.15.3:0/3948657964 conn(0x7fd8180fad70 msgr2=0x7fd8180fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 shutdown_connections 2023-08-11T14:10:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:01.842+0000 7fd81c9ff700 1 -- 172.21.15.3:0/3948657964 wait complete. 2023-08-11T14:10:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:01 smithi103 ceph-mon[130434]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-08-11T14:10:02.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:01 smithi003 ceph-mon[97195]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-08-11T14:10:02.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:01 smithi187 ceph-mon[130973]: pgmap v530: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 23 MiB/s wr, 1.92k op/s 2023-08-11T14:10:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 185.0 s, 1074.6 tps, lat 92.427 ms stddev 123.020 2023-08-11T14:10:04.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:03 smithi103 ceph-mon[130434]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 23 MiB/s wr, 1.93k op/s 2023-08-11T14:10:04.225 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:03 smithi187 ceph-mon[130973]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 23 MiB/s wr, 1.93k op/s 2023-08-11T14:10:04.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:03 smithi003 ceph-mon[97195]: pgmap v531: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 23 MiB/s wr, 1.93k op/s 2023-08-11T14:10:05.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:04 smithi103 ceph-mon[130434]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 23 MiB/s wr, 1.74k op/s 2023-08-11T14:10:05.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:04 smithi003 ceph-mon[97195]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 23 MiB/s wr, 1.74k op/s 2023-08-11T14:10:05.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:04 smithi187 ceph-mon[130973]: pgmap v532: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 23 MiB/s wr, 1.74k op/s 2023-08-11T14:10:07.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:07 smithi103 ceph-mon[130434]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-08-11T14:10:08.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:07 smithi003 ceph-mon[97195]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-08-11T14:10:08.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:07 smithi187 ceph-mon[130973]: pgmap v533: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 21 MiB/s wr, 1.61k op/s 2023-08-11T14:10:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 190.0 s, 789.6 tps, lat 124.696 ms stddev 345.407 2023-08-11T14:10:08.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:08 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:10:09.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:08 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:10:09.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:08 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:10:10.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:09 smithi003 ceph-mon[97195]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 1.71k op/s 2023-08-11T14:10:10.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:09 smithi187 ceph-mon[130973]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 1.71k op/s 2023-08-11T14:10:10.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:09 smithi103 ceph-mon[130434]: pgmap v534: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 23 MiB/s wr, 1.71k op/s 2023-08-11T14:10:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:10:12.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:11 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:10:12.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:12.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:10:12.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:11 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:10:12.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: pgmap v535: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:10:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:11 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:10:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 195.0 s, 829.0 tps, lat 123.242 ms stddev 242.970 2023-08-11T14:10:13.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:12 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:10:13.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:12 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:10:13.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:10:13.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:12 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:10:13.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:12 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:10:13.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:10:13.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:12 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:10:13.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:12 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:10:13.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:10:14.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:13 smithi103 ceph-mon[130434]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-08-11T14:10:14.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:13 smithi003 ceph-mon[97195]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-08-11T14:10:14.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:13 smithi187 ceph-mon[130973]: pgmap v536: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 1.83k op/s 2023-08-11T14:10:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 -- 172.21.15.3:0/3799430605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74941044d0 msgr2=0x7f7494104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 --2- 172.21.15.3:0/3799430605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74941044d0 0x7f7494104950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f748c009f90 tx=0x7f748c00b330 comp rx=0 tx=0).stop 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 -- 172.21.15.3:0/3799430605 shutdown_connections 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 --2- 172.21.15.3:0/3799430605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74941044d0 0x7f7494104950 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 --2- 172.21.15.3:0/3799430605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74940ff190 0x7f74941015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 --2- 172.21.15.3:0/3799430605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7494101b00 0x7f7494103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.103+0000 7f749b5ff700 1 -- 172.21.15.3:0/3799430605 >> 172.21.15.3:0/3799430605 conn(0x7f74940fad70 msgr2=0x7f74940fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.104+0000 7f749b5ff700 1 -- 172.21.15.3:0/3799430605 shutdown_connections 2023-08-11T14:10:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.104+0000 7f749b5ff700 1 -- 172.21.15.3:0/3799430605 wait complete. 2023-08-11T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.105+0000 7f749b5ff700 1 Processor -- start 2023-08-11T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.105+0000 7f749b5ff700 1 -- start start 2023-08-11T14:10:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74940ff190 0x7f74941944a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7494101b00 0x7f74941949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494195ec0 con 0x7f7494101b00 2023-08-11T14:10:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494196030 con 0x7f74941044d0 2023-08-11T14:10:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749b5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941961f0 con 0x7f74940ff190 2023-08-11T14:10:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f7499b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f7499b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43560/0 (socket says 172.21.15.3:43560) 2023-08-11T14:10:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f7499b9c700 1 -- 172.21.15.3:0/342432948 learned_addr learned my addr 172.21.15.3:0/342432948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f749939b700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74940ff190 0x7f74941944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.106+0000 7f7498b9a700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7494101b00 0x7f74941949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 -- 172.21.15.3:0/342432948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74940ff190 msgr2=0x7f74941944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74940ff190 0x7f74941944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 -- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7494101b00 msgr2=0x7f74941949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7494101b00 0x7f74941949e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 -- 172.21.15.3:0/342432948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c009c40 con 0x7f74941044d0 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f7499b9c700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b6c0 tx=0x7f748c004ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f748a7fc700 1 -- 172.21.15.3:0/342432948 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00e040 con 0x7f74941044d0 2023-08-11T14:10:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f748a7fc700 1 -- 172.21.15.3:0/342432948 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c007900 con 0x7f74941044d0 2023-08-11T14:10:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.107+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7494196470 con 0x7f74941044d0 2023-08-11T14:10:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.108+0000 7f748a7fc700 1 -- 172.21.15.3:0/342432948 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c01b6c0 con 0x7f74941044d0 2023-08-11T14:10:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.108+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749419cc90 con 0x7f74941044d0 2023-08-11T14:10:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.116+0000 7f74aee66b80 1 Processor -- start 2023-08-11T14:10:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.117+0000 7f74aee66b80 1 -- start start 2023-08-11T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 0x558c1e515360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e5161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 0x558c1e516c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e46e5d0 con 0x558c1e514f40 2023-08-11T14:10:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e46bef0 con 0x558c1e516790 2023-08-11T14:10:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f74aee66b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e4677e0 con 0x558c1e5158a0 2023-08-11T14:10:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f7498b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 0x558c1e515360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f749939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e5161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f7498b9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 0x558c1e515360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58054/0 (socket says 172.21.15.3:58054) 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f7498b9a700 1 -- 172.21.15.3:0/3443258544 learned_addr learned my addr 172.21.15.3:0/3443258544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.119+0000 7f7498b9a700 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 0x558c1e516c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 -- 172.21.15.3:0/3443258544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 msgr2=0x558c1e5161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e5161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 -- 172.21.15.3:0/3443258544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 msgr2=0x558c1e515360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 0x558c1e515360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 -- 172.21.15.3:0/3443258544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1e2e75d0 con 0x558c1e516790 2023-08-11T14:10:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.120+0000 7f7498b9a700 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 0x558c1e516c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7484024820 tx=0x7f74840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2346dc33752aa8df server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.121+0000 7f747f7fe700 1 -- 172.21.15.3:0/3443258544 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484024110 con 0x558c1e516790 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.121+0000 7f747f7fe700 1 -- 172.21.15.3:0/3443258544 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7484024270 con 0x558c1e516790 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.121+0000 7f747f7fe700 1 -- 172.21.15.3:0/3443258544 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484022600 con 0x558c1e516790 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.121+0000 7f74aee66b80 1 -- 172.21.15.3:0/3443258544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 msgr2=0x558c1e516c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.121+0000 7f74aee66b80 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 0x558c1e516c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7484024820 tx=0x7f74840047c0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f748a7fc700 1 -- 172.21.15.3:0/342432948 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f748c01d470 con 0x7f74941044d0 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 -- 172.21.15.3:0/3443258544 shutdown_connections 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e516790 0x558c1e516c10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e514f40 0x558c1e515360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 --2- 172.21.15.3:0/3443258544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e5161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 -- 172.21.15.3:0/3443258544 >> 172.21.15.3:0/3443258544 conn(0x558c1e4428f0 msgr2=0x558c1e511920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 -- 172.21.15.3:0/3443258544 shutdown_connections 2023-08-11T14:10:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.122+0000 7f74aee66b80 1 -- 172.21.15.3:0/3443258544 wait complete. 2023-08-11T14:10:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.123+0000 7f74aee66b80 1 -- start start 2023-08-11T14:10:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.123+0000 7f74aee66b80 1 Processor -- start 2023-08-11T14:10:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e514f40 0x558c1e51ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e526200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e405a60 con 0x558c1e516790 2023-08-11T14:10:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e405760 con 0x558c1e514f40 2023-08-11T14:10:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e2e98c0 con 0x558c1e5158a0 2023-08-11T14:10:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f74aee66b80 10 client.0 _ll_register_callbacks cb 0x558c1e408a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f749939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f749939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57268/0 (socket says 172.21.15.3:57268) 2023-08-11T14:10:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.129+0000 7f749939b700 1 -- 172.21.15.3:0/2112839954 learned_addr learned my addr 172.21.15.3:0/2112839954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 -- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e514f40 msgr2=0x558c1e51ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e514f40 0x558c1e51ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 -- 172.21.15.3:0/2112839954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 msgr2=0x558c1e526200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e526200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 -- 172.21.15.3:0/2112839954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748402c040 con 0x558c1e5158a0 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f748a7fc700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 0x7f74800a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f749939b700 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74900127f0 tx=0x7f7490012bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31209a85627fb0c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.130+0000 7f748a7fc700 1 -- 172.21.15.3:0/342432948 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f748c0cc770 con 0x7f74941044d0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f747cff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490017070 con 0x558c1e5158a0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f749000f640 con 0x558c1e5158a0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c1e5087e0 con 0x558c1e5158a0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f7498b9a700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 0x7f74800a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f749000f800 con 0x558c1e5158a0 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f74aee66b80 10 client.26254 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.131+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558c1e293d80 con 0x558c1e5158a0 2023-08-11T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c1e5087e0 con 0x558c1e5158a0 2023-08-11T14:10:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7490010950 con 0x558c1e5158a0 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 20 client.26254 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e52ab60 0x558c1e52cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 4 client.26254 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c1e4e4480 con 0x558c1e52ab60 2023-08-11T14:10:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.132+0000 7f7498b9a700 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e52ab60 0x558c1e52cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.133+0000 7f7498b9a700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 0x7f74800a23f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f748401a5d0 tx=0x7f7484004750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.134+0000 7f7498b9a700 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e52ab60 0x558c1e52cf80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b733329427b16589 server_cookie=1c551c17bdfe9e7 in_seq=0 out_seq=0 2023-08-11T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.134+0000 7f747cff9700 10 client.26254 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.134+0000 7f747cff9700 1 -- 172.21.15.3:0/2112839954 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558c1e4e4480 con 0x558c1e52ab60 2023-08-11T14:10:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.134+0000 7f747cff9700 10 client.26254 handle_command_reply: tid=0 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.135+0000 7f74aee66b80 1 client.26254 shutdown 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 msgr2=0x558c1e51f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74900127f0 tx=0x7f7490012bb0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 shutdown_connections 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e526200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e52ab60 0x558c1e52cf80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e514f40 0x558c1e51ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 --2- 172.21.15.3:0/2112839954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e5158a0 0x558c1e51f1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 >> 172.21.15.3:0/2112839954 conn(0x558c1e4428f0 msgr2=0x558c1e510710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.136+0000 7f747cff9700 0 client.26254 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.141+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 shutdown_connections 2023-08-11T14:10:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.149+0000 7f74aee66b80 1 -- 172.21.15.3:0/2112839954 wait complete. 2023-08-11T14:10:15.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.149+0000 7f74aee66b80 20 client.26254 trim_cache size 0 max 16384 2023-08-11T14:10:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.187+0000 7f74aee66b80 1 Processor -- start 2023-08-11T14:10:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 -- start start 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e52cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e51ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e50aee0 con 0x558c1e516790 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e508570 con 0x558c1e5158a0 2023-08-11T14:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f74aee66b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e51f900 con 0x558c1e514f40 2023-08-11T14:10:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f7498b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f7498b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43590/0 (socket says 172.21.15.3:43590) 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.192+0000 7f7498b9a700 1 -- 172.21.15.3:0/1692932191 learned_addr learned my addr 172.21.15.3:0/1692932191 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 -- 172.21.15.3:0/1692932191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 msgr2=0x558c1e52cb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e52cb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 -- 172.21.15.3:0/1692932191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 msgr2=0x558c1e51ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e51ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 -- 172.21.15.3:0/1692932191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1e50ca10 con 0x558c1e5158a0 2023-08-11T14:10:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.193+0000 7f7498b9a700 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f748401a7c0 tx=0x7f7484023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5c45b180574db5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.195+0000 7f747ffff700 1 -- 172.21.15.3:0/1692932191 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484042480 con 0x558c1e5158a0 2023-08-11T14:10:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.195+0000 7f747ffff700 1 -- 172.21.15.3:0/1692932191 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748401a050 con 0x558c1e5158a0 2023-08-11T14:10:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.195+0000 7f747ffff700 1 -- 172.21.15.3:0/1692932191 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748401fdc0 con 0x558c1e5158a0 2023-08-11T14:10:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.196+0000 7f74aee66b80 1 -- 172.21.15.3:0/1692932191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 msgr2=0x558c1e526120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.196+0000 7f74aee66b80 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f748401a7c0 tx=0x7f7484023000 comp rx=0 tx=0).stop 2023-08-11T14:10:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.197+0000 7f74aee66b80 1 -- 172.21.15.3:0/1692932191 shutdown_connections 2023-08-11T14:10:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.197+0000 7f74aee66b80 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e51ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.197+0000 7f74aee66b80 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e52cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.197+0000 7f74aee66b80 1 --2- 172.21.15.3:0/1692932191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e5158a0 0x558c1e526120 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f748401a7c0 tx=0x7f7484023000 comp rx=0 tx=0).stop 2023-08-11T14:10:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.197+0000 7f74aee66b80 1 -- 172.21.15.3:0/1692932191 >> 172.21.15.3:0/1692932191 conn(0x558c1e4428f0 msgr2=0x558c1e526a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.201+0000 7f74aee66b80 1 -- 172.21.15.3:0/1692932191 shutdown_connections 2023-08-11T14:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.201+0000 7f74aee66b80 1 -- 172.21.15.3:0/1692932191 wait complete. 2023-08-11T14:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.202+0000 7f74aee66b80 1 -- start start 2023-08-11T14:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.202+0000 7f74aee66b80 1 Processor -- start 2023-08-11T14:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e5239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e523ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e5258c0 con 0x558c1e516790 2023-08-11T14:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e525a30 con 0x558c1e524f00 2023-08-11T14:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1e525bf0 con 0x558c1e514f40 2023-08-11T14:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f74aee66b80 10 client.0 _ll_register_callbacks cb 0x7f7490009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f7499b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.203+0000 7f7499b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43596/0 (socket says 172.21.15.3:43596) 2023-08-11T14:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 -- 172.21.15.3:0/987125938 learned_addr learned my addr 172.21.15.3:0/987125938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f749939b700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e5239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7498b9a700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e523ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 -- 172.21.15.3:0/987125938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 msgr2=0x558c1e5239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e5239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 -- 172.21.15.3:0/987125938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 msgr2=0x558c1e523ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e523ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7499b9c700 1 -- 172.21.15.3:0/987125938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c1e2e75d0 con 0x558c1e524f00 2023-08-11T14:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.204+0000 7f7498b9a700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e523ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.205+0000 7f7499b9c700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f748c0963d0 tx=0x7f748c099a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5730731aff2264d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.205+0000 7f747cff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.205+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0b9270 con 0x558c1e524f00 2023-08-11T14:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.205+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f748c099ed0 con 0x558c1e524f00 2023-08-11T14:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.205+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7484024270 con 0x558c1e524f00 2023-08-11T14:10:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.209+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f748c09dd00 con 0x558c1e524f00 2023-08-11T14:10:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.209+0000 7f74aee66b80 10 client.26270 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.209+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558c1e27cc00 con 0x558c1e524f00 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7484024270 con 0x558c1e524f00 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f748c0b5030 con 0x558c1e524f00 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 10 client.26270 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 20 client.26270 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e57f4c0 0x558c1e5818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 4 client.26270 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.212+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558c1e4607c0 con 0x558c1e57f4c0 2023-08-11T14:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.215+0000 7f749939b700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e57f4c0 0x558c1e5818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.218+0000 7f749939b700 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e57f4c0 0x558c1e5818e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ef84c6e631f798 server_cookie=46352d400188231d in_seq=0 out_seq=0 2023-08-11T14:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.218+0000 7f747cff9700 10 client.26270 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:15.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.285+0000 7f747cff9700 1 -- 172.21.15.3:0/987125938 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558c1e4607c0 con 0x558c1e57f4c0 2023-08-11T14:10:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.285+0000 7f747cff9700 10 client.26270 handle_command_reply: tid=0 2023-08-11T14:10:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.285+0000 7f74aee66b80 1 client.26270 shutdown 2023-08-11T14:10:15.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 msgr2=0x558c1e525380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f748c0963d0 tx=0x7f748c099a10 comp rx=0 tx=0).stop 2023-08-11T14:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 shutdown_connections 2023-08-11T14:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558c1e57f4c0 0x558c1e5818e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c1e524f00 0x558c1e525380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558c1e514f40 0x558c1e5239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 --2- 172.21.15.3:0/987125938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558c1e516790 0x558c1e523ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.286+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 >> 172.21.15.3:0/987125938 conn(0x558c1e4428f0 msgr2=0x558c1e526a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.287+0000 7f747cff9700 0 client.26270 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:15.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.287+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 shutdown_connections 2023-08-11T14:10:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.287+0000 7f74aee66b80 1 -- 172.21.15.3:0/987125938 wait complete. 2023-08-11T14:10:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:10:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:10:15.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:10:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:10:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.288+0000 7f74aee66b80 20 client.26270 trim_cache size 0 max 16384 2023-08-11T14:10:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.288+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 msgr2=0x7f74800a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.288+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 0x7f74800a23f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f748401a5d0 tx=0x7f7484004750 comp rx=0 tx=0).stop 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.288+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 msgr2=0x7f7494195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.288+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b6c0 tx=0x7f748c004ca0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 shutdown_connections 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f748009ff30 0x7f74800a23f0 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f748401a5d0 tx=0x7f7484004750 comp rx=0 tx=0).stop 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74941044d0 0x7f7494195860 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74940ff190 0x7f74941944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 --2- 172.21.15.3:0/342432948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7494101b00 0x7f74941949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.289+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 >> 172.21.15.3:0/342432948 conn(0x7f74940fad70 msgr2=0x7f74940fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:15.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:14 smithi003 ceph-mon[97195]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-08-11T14:10:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.290+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 shutdown_connections 2023-08-11T14:10:15.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:15.290+0000 7f749b5ff700 1 -- 172.21.15.3:0/342432948 wait complete. 2023-08-11T14:10:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:14 smithi187 ceph-mon[130973]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-08-11T14:10:15.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:14 smithi103 ceph-mon[130434]: pgmap v537: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 24 MiB/s wr, 1.78k op/s 2023-08-11T14:10:17.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:17 smithi103 ceph-mon[130434]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-08-11T14:10:18.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:17 smithi003 ceph-mon[97195]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-08-11T14:10:18.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:17 smithi187 ceph-mon[130973]: pgmap v538: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 20 MiB/s wr, 1.59k op/s 2023-08-11T14:10:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 200.0 s, 1287.8 tps, lat 62.132 ms stddev 46.556 2023-08-11T14:10:20.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:19 smithi187 ceph-mon[130973]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-08-11T14:10:20.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:19 smithi003 ceph-mon[97195]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-08-11T14:10:20.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:19 smithi103 ceph-mon[130434]: pgmap v539: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.47k op/s 2023-08-11T14:10:22.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:21 smithi103 ceph-mon[130434]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:22.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:21 smithi003 ceph-mon[97195]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:22.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:21 smithi187 ceph-mon[130973]: pgmap v540: 129 pgs: 129 active+clean; 8.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 24 MiB/s wr, 1.84k op/s 2023-08-11T14:10:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 205.0 s, 889.6 tps, lat 134.452 ms stddev 436.051 2023-08-11T14:10:24.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:23 smithi103 ceph-mon[130434]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-08-11T14:10:24.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:23 smithi003 ceph-mon[97195]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-08-11T14:10:24.304 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:23 smithi187 ceph-mon[130973]: pgmap v541: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 23 MiB/s wr, 1.78k op/s 2023-08-11T14:10:25.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:24 smithi003 ceph-mon[97195]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:10:25.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:24 smithi187 ceph-mon[130973]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:10:25.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:24 smithi103 ceph-mon[130434]: pgmap v542: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:10:27.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:27 smithi103 ceph-mon[130434]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 26 MiB/s wr, 2.14k op/s 2023-08-11T14:10:28.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:27 smithi187 ceph-mon[130973]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 26 MiB/s wr, 2.14k op/s 2023-08-11T14:10:28.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:27 smithi003 ceph-mon[97195]: pgmap v543: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 26 MiB/s wr, 2.14k op/s 2023-08-11T14:10:28.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 210.0 s, 524.4 tps, lat 176.209 ms stddev 457.545 2023-08-11T14:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.722+0000 7fccdddff700 1 -- 172.21.15.3:0/4196769903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810a290 msgr2=0x7fccd810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.722+0000 7fccdddff700 1 --2- 172.21.15.3:0/4196769903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810a290 0x7fccd810c720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009990 tx=0x7fccc8009ca0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.738+0000 7fccdddff700 1 -- 172.21.15.3:0/4196769903 shutdown_connections 2023-08-11T14:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.738+0000 7fccdddff700 1 --2- 172.21.15.3:0/4196769903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd810cc60 0x7fccd810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.738+0000 7fccdddff700 1 --2- 172.21.15.3:0/4196769903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd8107920 0x7fccd8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.738+0000 7fccdddff700 1 --2- 172.21.15.3:0/4196769903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810a290 0x7fccd810c720 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.738+0000 7fccdddff700 1 -- 172.21.15.3:0/4196769903 >> 172.21.15.3:0/4196769903 conn(0x7fccd806d750 msgr2=0x7fccd806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.749+0000 7fccdddff700 1 -- 172.21.15.3:0/4196769903 shutdown_connections 2023-08-11T14:10:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.756+0000 7fccdddff700 1 -- 172.21.15.3:0/4196769903 wait complete. 2023-08-11T14:10:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.757+0000 7fccdddff700 1 Processor -- start 2023-08-11T14:10:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 -- start start 2023-08-11T14:10:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 0x7fccd81a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd810a290 0x7fccd81a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 0x7fccd81a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a6f30 con 0x7fccd810a290 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a70a0 con 0x7fccd810cc60 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.758+0000 7fccdddff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a7260 con 0x7fccd8107920 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.760+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 0x7fccd81a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.760+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 0x7fccd81a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40674/0 (socket says 172.21.15.3:40674) 2023-08-11T14:10:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.760+0000 7fccd77fe700 1 -- 172.21.15.3:0/2458392985 learned_addr learned my addr 172.21.15.3:0/2458392985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.760+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 0x7fccd81a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.769+0000 7fccd7fff700 1 -- 172.21.15.3:0/2458392985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 msgr2=0x7fccd81a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.769+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 0x7fccd81a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.769+0000 7fccd7fff700 1 -- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd810a290 msgr2=0x7fccd81a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.769+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd810a290 0x7fccd81a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.769+0000 7fccd7fff700 1 -- 172.21.15.3:0/2458392985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc80095f0 con 0x7fccd810cc60 2023-08-11T14:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.773+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 0x7fccd81a68a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fccd000efc0 tx=0x7fccd000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.781+0000 7fccd57fa700 1 -- 172.21.15.3:0/2458392985 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0018070 con 0x7fccd810cc60 2023-08-11T14:10:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.781+0000 7fccd57fa700 1 -- 172.21.15.3:0/2458392985 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd000f040 con 0x7fccd810cc60 2023-08-11T14:10:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.781+0000 7fccd57fa700 1 -- 172.21.15.3:0/2458392985 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000cd90 con 0x7fccd810cc60 2023-08-11T14:10:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.781+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd81a74e0 con 0x7fccd810cc60 2023-08-11T14:10:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.781+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd8078d60 con 0x7fccd810cc60 2023-08-11T14:10:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.788+0000 7fccf16e9b80 1 Processor -- start 2023-08-11T14:10:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 -- start start 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d4f40 0x55e5963d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e5963d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59632e5d0 con 0x55e5963d4f40 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59632bef0 con 0x55e5963d6790 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.799+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5963277e0 con 0x55e5963d58a0 2023-08-11T14:10:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.800+0000 7fccd57fa700 1 -- 172.21.15.3:0/2458392985 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fccd0003a00 con 0x7fccd810cc60 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.804+0000 7fccd57fa700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 0x7fccc40a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.804+0000 7fccd57fa700 1 -- 172.21.15.3:0/2458392985 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccd00cb000 con 0x7fccd810cc60 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.810+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.810+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40682/0 (socket says 172.21.15.3:40682) 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.810+0000 7fccd77fe700 1 -- 172.21.15.3:0/3949896069 learned_addr learned my addr 172.21.15.3:0/3949896069 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.813+0000 7fccceffd700 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d4f40 0x55e5963d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.813+0000 7fccceffd700 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e5963d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.814+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 0x7fccc40a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.819+0000 7fccd77fe700 1 -- 172.21.15.3:0/3949896069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 msgr2=0x55e5963d6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.819+0000 7fccd77fe700 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e5963d6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.819+0000 7fccd77fe700 1 -- 172.21.15.3:0/3949896069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d4f40 msgr2=0x55e5963d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.819+0000 7fccd77fe700 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d4f40 0x55e5963d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.819+0000 7fccd77fe700 1 -- 172.21.15.3:0/3949896069 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5961a75d0 con 0x55e5963d58a0 2023-08-11T14:10:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.820+0000 7fccd77fe700 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fccc000be60 tx=0x7fccc00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=310f7f429c9a8658 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.830+0000 7fcccdffb700 1 -- 172.21.15.3:0/3949896069 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0011440 con 0x55e5963d58a0 2023-08-11T14:10:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.830+0000 7fcccdffb700 1 -- 172.21.15.3:0/3949896069 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc0015070 con 0x55e5963d58a0 2023-08-11T14:10:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.830+0000 7fcccdffb700 1 -- 172.21.15.3:0/3949896069 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0019890 con 0x55e5963d58a0 2023-08-11T14:10:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.831+0000 7fccf16e9b80 1 -- 172.21.15.3:0/3949896069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 msgr2=0x55e5963d61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.831+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fccc000be60 tx=0x7fccc00106b0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccf16e9b80 1 -- 172.21.15.3:0/3949896069 shutdown_connections 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e5963d6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d4f40 0x55e5963d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/3949896069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963d61b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccf16e9b80 1 -- 172.21.15.3:0/3949896069 >> 172.21.15.3:0/3949896069 conn(0x55e5963028f0 msgr2=0x55e5963d1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.836+0000 7fccd7fff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 0x7fccc40a2610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fccd0004de0 tx=0x7fccd0004d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.837+0000 7fccf16e9b80 1 -- 172.21.15.3:0/3949896069 shutdown_connections 2023-08-11T14:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.837+0000 7fccf16e9b80 1 -- 172.21.15.3:0/3949896069 wait complete. 2023-08-11T14:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.838+0000 7fccf16e9b80 1 -- start start 2023-08-11T14:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.838+0000 7fccf16e9b80 1 Processor -- start 2023-08-11T14:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d4f40 0x55e5963dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5962c5a60 con 0x55e5963d6790 2023-08-11T14:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5962c5760 con 0x55e5963d4f40 2023-08-11T14:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5961a98c0 con 0x55e5963d58a0 2023-08-11T14:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.839+0000 7fccf16e9b80 10 client.0 _ll_register_callbacks cb 0x55e5962c8a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.841+0000 7fccceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.841+0000 7fccceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37200/0 (socket says 172.21.15.3:37200) 2023-08-11T14:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.841+0000 7fccceffd700 1 -- 172.21.15.3:0/1115747387 learned_addr learned my addr 172.21.15.3:0/1115747387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.841+0000 7fccceffd700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d4f40 0x55e5963dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.842+0000 7fccceffd700 1 -- 172.21.15.3:0/1115747387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 msgr2=0x55e5963df1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.842+0000 7fccceffd700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963df1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.842+0000 7fccceffd700 1 -- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d4f40 msgr2=0x55e5963dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.842+0000 7fccceffd700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d4f40 0x55e5963dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.842+0000 7fccceffd700 1 -- 172.21.15.3:0/1115747387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc000ba90 con 0x55e5963d6790 2023-08-11T14:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.843+0000 7fccceffd700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fccc8015000 tx=0x7fccc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dacdd3f494ba5d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.843+0000 7fccb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:10:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.854+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800fa70 con 0x55e5963d6790 2023-08-11T14:10:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.854+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccc800fbd0 con 0x55e5963d6790 2023-08-11T14:10:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.854+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5963c87e0 con 0x55e5963d6790 2023-08-11T14:10:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.854+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc800fd90 con 0x55e5963d6790 2023-08-11T14:10:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.862+0000 7fccf16e9b80 10 client.16164 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.862+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e596153d80 con 0x55e5963d6790 2023-08-11T14:10:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5963c87e0 con 0x55e5963d6790 2023-08-11T14:10:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc80273c0 con 0x55e5963d6790 2023-08-11T14:10:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccf16e9b80 10 client.16164 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccf16e9b80 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccf16e9b80 20 client.16164 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.875+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e5963eaaa0 0x55e5963ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.876+0000 7fccf16e9b80 4 client.16164 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.876+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5963a4480 con 0x55e5963eaaa0 2023-08-11T14:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.884+0000 7fccd77fe700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e5963eaaa0 0x55e5963ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.916+0000 7fccd77fe700 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e5963eaaa0 0x55e5963ecec0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df6c00f0d9cea60 server_cookie=8cb655b47a927f8e in_seq=0 out_seq=0 2023-08-11T14:10:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.916+0000 7fccb6ffd700 10 client.16164 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.921+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1115747387 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e5963a4480 con 0x55e5963eaaa0 2023-08-11T14:10:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.921+0000 7fccb6ffd700 10 client.16164 handle_command_reply: tid=0 2023-08-11T14:10:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.921+0000 7fccf16e9b80 1 client.16164 shutdown 2023-08-11T14:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.935+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 msgr2=0x55e5963e6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.935+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fccc8015000 tx=0x7fccc8000bc0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 shutdown_connections 2023-08-11T14:10:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963e6200 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e5963eaaa0 0x55e5963ecec0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d4f40 0x55e5963dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1115747387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d58a0 0x55e5963df1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.947+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 >> 172.21.15.3:0/1115747387 conn(0x55e5963028f0 msgr2=0x55e5963d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.949+0000 7fccb6ffd700 0 client.16164 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.963+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 shutdown_connections 2023-08-11T14:10:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.974+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1115747387 wait complete. 2023-08-11T14:10:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:28.991+0000 7fccf16e9b80 20 client.16164 trim_cache size 0 max 16384 2023-08-11T14:10:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.020+0000 7fccf16e9b80 1 Processor -- start 2023-08-11T14:10:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 -- start start 2023-08-11T14:10:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d58a0 0x55e5963e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5963ddfe0 con 0x55e5963d6790 2023-08-11T14:10:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5963caea0 con 0x55e5963d58a0 2023-08-11T14:10:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.027+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5963d5c80 con 0x55e5963d4f40 2023-08-11T14:10:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.032+0000 7fccceffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.032+0000 7fccceffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40706/0 (socket says 172.21.15.3:40706) 2023-08-11T14:10:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.032+0000 7fccceffd700 1 -- 172.21.15.3:0/2285727524 learned_addr learned my addr 172.21.15.3:0/2285727524 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.038+0000 7fccceffd700 1 -- 172.21.15.3:0/2285727524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d58a0 msgr2=0x55e5963e4180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:10:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.038+0000 7fccceffd700 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d58a0 0x55e5963e4180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.038+0000 7fccceffd700 1 -- 172.21.15.3:0/2285727524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 msgr2=0x55e5963d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.038+0000 7fccceffd700 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.038+0000 7fccceffd700 1 -- 172.21.15.3:0/2285727524 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59613cc00 con 0x55e5963d4f40 2023-08-11T14:10:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.045+0000 7fccceffd700 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b720 tx=0x7fccc80098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89e89dcfe62bb72 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.048+0000 7fccce7fc700 1 -- 172.21.15.3:0/2285727524 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80058e0 con 0x55e5963d4f40 2023-08-11T14:10:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.048+0000 7fccce7fc700 1 -- 172.21.15.3:0/2285727524 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc801c070 con 0x55e5963d4f40 2023-08-11T14:10:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.048+0000 7fccce7fc700 1 -- 172.21.15.3:0/2285727524 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8028410 con 0x55e5963d4f40 2023-08-11T14:10:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.050+0000 7fccf16e9b80 1 -- 172.21.15.3:0/2285727524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 msgr2=0x55e5963d5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.050+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b720 tx=0x7fccc80098e0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.060+0000 7fccf16e9b80 1 -- 172.21.15.3:0/2285727524 shutdown_connections 2023-08-11T14:10:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.060+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d6790 0x55e5963d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.060+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963d5320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.060+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/2285727524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d58a0 0x55e5963e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.060+0000 7fccf16e9b80 1 -- 172.21.15.3:0/2285727524 >> 172.21.15.3:0/2285727524 conn(0x55e5963028f0 msgr2=0x55e5963dcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.062+0000 7fccf16e9b80 1 -- 172.21.15.3:0/2285727524 shutdown_connections 2023-08-11T14:10:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.063+0000 7fccf16e9b80 1 -- 172.21.15.3:0/2285727524 wait complete. 2023-08-11T14:10:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.063+0000 7fccf16e9b80 1 -- start start 2023-08-11T14:10:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.063+0000 7fccf16e9b80 1 Processor -- start 2023-08-11T14:10:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 0x55e5963e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e59643cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59643d080 con 0x55e5963d58a0 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59643d1c0 con 0x55e5963d6790 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59643d350 con 0x55e5963d4f40 2023-08-11T14:10:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.064+0000 7fccf16e9b80 10 client.0 _ll_register_callbacks cb 0x7fccd00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.069+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 0x55e5963e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.069+0000 7fccd77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 0x55e5963e3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37222/0 (socket says 172.21.15.3:37222) 2023-08-11T14:10:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.069+0000 7fccd77fe700 1 -- 172.21.15.3:0/1955105633 learned_addr learned my addr 172.21.15.3:0/1955105633 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.069+0000 7fccceffd700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e59643cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.069+0000 7fccceffd700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.070+0000 7fccceffd700 1 -- 172.21.15.3:0/1955105633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 msgr2=0x55e5963e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.070+0000 7fccceffd700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.070+0000 7fccceffd700 1 -- 172.21.15.3:0/1955105633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 msgr2=0x55e5963e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.070+0000 7fccceffd700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 0x55e5963e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.070+0000 7fccceffd700 1 -- 172.21.15.3:0/1955105633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e596153d80 con 0x55e5963d6790 2023-08-11T14:10:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.078+0000 7fccceffd700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e59643cb40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccc8027750 tx=0x7fccc800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf7e2e8f8c7a160 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.078+0000 7fccb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:10:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.091+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800f7f0 con 0x55e5963d6790 2023-08-11T14:10:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.091+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fccc800f950 con 0x55e5963d6790 2023-08-11T14:10:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.091+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc0015070 con 0x55e5963d6790 2023-08-11T14:10:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.096+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc800fb10 con 0x55e5963d6790 2023-08-11T14:10:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.097+0000 7fccf16e9b80 10 client.26282 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.097+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e59617a1f0 con 0x55e5963d6790 2023-08-11T14:10:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.105+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc0015070 con 0x55e5963d6790 2023-08-11T14:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.105+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fccc8028410 con 0x55e5963d6790 2023-08-11T14:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 10 client.26282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 20 client.26282 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e59643e850 0x55e596440c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 4 client.26282 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e5963207c0 con 0x55e59643e850 2023-08-11T14:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.106+0000 7fccd77fe700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e59643e850 0x55e596440c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.119+0000 7fccd77fe700 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e59643e850 0x55e596440c40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecdec22675ea9f8 server_cookie=3900f3f3771cdca8 in_seq=0 out_seq=0 2023-08-11T14:10:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:29.119+0000 7fccb6ffd700 10 client.26282 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:30.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:29 smithi103 ceph-mon[130434]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.23k op/s 2023-08-11T14:10:30.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:29 smithi003 ceph-mon[97195]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.23k op/s 2023-08-11T14:10:30.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:29 smithi187 ceph-mon[130973]: pgmap v544: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.23k op/s 2023-08-11T14:10:32.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:31 smithi103 ceph-mon[130434]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-08-11T14:10:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.246+0000 7fccb6ffd700 1 -- 172.21.15.3:0/1955105633 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e5963207c0 con 0x55e59643e850 2023-08-11T14:10:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.246+0000 7fccb6ffd700 10 client.26282 handle_command_reply: tid=0 2023-08-11T14:10:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.246+0000 7fccf16e9b80 1 client.26282 shutdown 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 msgr2=0x55e59643cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e59643cb40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccc8027750 tx=0x7fccc800b610 comp rx=0 tx=0).stop 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 shutdown_connections 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e59643e850 0x55e596440c40 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5963d6790 0x55e59643cb40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5963d4f40 0x55e5963e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 --2- 172.21.15.3:0/1955105633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e5963d58a0 0x55e5963e3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 >> 172.21.15.3:0/1955105633 conn(0x55e5963028f0 msgr2=0x55e5963d96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccb6ffd700 0 client.26282 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 shutdown_connections 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 1 -- 172.21.15.3:0/1955105633 wait complete. 2023-08-11T14:10:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.247+0000 7fccf16e9b80 20 client.26282 trim_cache size 0 max 16384 2023-08-11T14:10:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:10:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:10:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:10:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:10:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.248+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 msgr2=0x7fccc40a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.248+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 0x7fccc40a2610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fccd0004de0 tx=0x7fccd0004d30 comp rx=0 tx=0).stop 2023-08-11T14:10:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.248+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 msgr2=0x7fccd81a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.248+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 0x7fccd81a68a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fccd000efc0 tx=0x7fccd000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 shutdown_connections 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fccc40a0150 0x7fccc40a2610 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd810cc60 0x7fccd81a68a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccd8107920 0x7fccd81a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 --2- 172.21.15.3:0/2458392985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd810a290 0x7fccd81a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 >> 172.21.15.3:0/2458392985 conn(0x7fccd806d750 msgr2=0x7fccd8109180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 shutdown_connections 2023-08-11T14:10:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:32.249+0000 7fccdddff700 1 -- 172.21.15.3:0/2458392985 wait complete. 2023-08-11T14:10:32.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:31 smithi003 ceph-mon[97195]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-08-11T14:10:32.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:31 smithi187 ceph-mon[130973]: pgmap v545: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 29 MiB/s wr, 2.44k op/s 2023-08-11T14:10:33.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 215.0 s, 462.3 tps, lat 231.338 ms stddev 311.058 2023-08-11T14:10:34.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:33 smithi103 ceph-mon[130434]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 25 MiB/s wr, 2.26k op/s 2023-08-11T14:10:34.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:33 smithi003 ceph-mon[97195]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 25 MiB/s wr, 2.26k op/s 2023-08-11T14:10:34.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:33 smithi187 ceph-mon[130973]: pgmap v546: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 25 MiB/s wr, 2.26k op/s 2023-08-11T14:10:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:34 smithi003 ceph-mon[97195]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2023-08-11T14:10:35.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:34 smithi187 ceph-mon[130973]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2023-08-11T14:10:35.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:34 smithi103 ceph-mon[130434]: pgmap v547: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 20 MiB/s wr, 1.86k op/s 2023-08-11T14:10:37.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:37 smithi103 ceph-mon[130434]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 2.32k op/s 2023-08-11T14:10:38.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:37 smithi003 ceph-mon[97195]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 2.32k op/s 2023-08-11T14:10:38.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:37 smithi187 ceph-mon[130973]: pgmap v548: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 2.32k op/s 2023-08-11T14:10:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 220.0 s, 915.9 tps, lat 109.602 ms stddev 181.426 2023-08-11T14:10:40.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:39 smithi103 ceph-mon[130434]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.55k op/s 2023-08-11T14:10:40.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:39 smithi003 ceph-mon[97195]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.55k op/s 2023-08-11T14:10:40.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:39 smithi187 ceph-mon[130973]: pgmap v549: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 1.55k op/s 2023-08-11T14:10:42.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:41 smithi103 ceph-mon[130434]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.69k op/s 2023-08-11T14:10:42.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:41 smithi003 ceph-mon[97195]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.69k op/s 2023-08-11T14:10:42.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:41 smithi187 ceph-mon[130973]: pgmap v550: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 1.69k op/s 2023-08-11T14:10:43.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 225.0 s, 977.8 tps, lat 103.066 ms stddev 180.176 2023-08-11T14:10:44.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:43 smithi103 ceph-mon[130434]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-08-11T14:10:44.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:43 smithi003 ceph-mon[97195]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-08-11T14:10:44.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:43 smithi187 ceph-mon[130973]: pgmap v551: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 21 MiB/s wr, 1.79k op/s 2023-08-11T14:10:45.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:45 smithi187 ceph-mon[130973]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-08-11T14:10:45.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:45 smithi003 ceph-mon[97195]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-08-11T14:10:45.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:45 smithi103 ceph-mon[130434]: pgmap v552: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.30k op/s 2023-08-11T14:10:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.567+0000 7fa0a3dff700 1 -- 172.21.15.3:0/3240644376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 msgr2=0x7fa09c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.567+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/3240644376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c109d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa0980098f0 tx=0x7fa098009c00 comp rx=0 tx=0).stop 2023-08-11T14:10:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.568+0000 7fa0a3dff700 1 -- 172.21.15.3:0/3240644376 shutdown_connections 2023-08-11T14:10:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.568+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/3240644376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10cc60 0x7fa09c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.568+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/3240644376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c109d50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.568+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/3240644376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10a290 0x7fa09c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.568+0000 7fa0a3dff700 1 -- 172.21.15.3:0/3240644376 >> 172.21.15.3:0/3240644376 conn(0x7fa09c06d710 msgr2=0x7fa09c06fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.572+0000 7fa0a3dff700 1 -- 172.21.15.3:0/3240644376 shutdown_connections 2023-08-11T14:10:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.572+0000 7fa0a3dff700 1 -- 172.21.15.3:0/3240644376 wait complete. 2023-08-11T14:10:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.574+0000 7fa0a3dff700 1 Processor -- start 2023-08-11T14:10:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.576+0000 7fa0a3dff700 1 -- start start 2023-08-11T14:10:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c1a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10cc60 0x7fa09c1aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c1ad010 con 0x7fa09c10cc60 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c10f050 con 0x7fa09c107920 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a3dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c10f1e0 con 0x7fa09c10a290 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a1b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c1a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.577+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.578+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60480/0 (socket says 172.21.15.3:60480) 2023-08-11T14:10:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.578+0000 7fa0a139a700 1 -- 172.21.15.3:0/1101492836 learned_addr learned my addr 172.21.15.3:0/1101492836 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.583+0000 7fa0a239c700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10cc60 0x7fa09c1aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 -- 172.21.15.3:0/1101492836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 msgr2=0x7fa09c1a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c1a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 -- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10cc60 msgr2=0x7fa09c1aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10cc60 0x7fa09c1aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 -- 172.21.15.3:0/1101492836 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009520 con 0x7fa09c10a290 2023-08-11T14:10:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.584+0000 7fa0a139a700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fa0 tx=0x7fa09400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.585+0000 7fa092ffd700 1 -- 172.21.15.3:0/1101492836 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400dcf0 con 0x7fa09c10a290 2023-08-11T14:10:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.585+0000 7fa092ffd700 1 -- 172.21.15.3:0/1101492836 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa09400f040 con 0x7fa09c10a290 2023-08-11T14:10:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.585+0000 7fa092ffd700 1 -- 172.21.15.3:0/1101492836 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094014600 con 0x7fa09c10a290 2023-08-11T14:10:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.588+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c10f4c0 con 0x7fa09c10a290 2023-08-11T14:10:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.588+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c10f9d0 con 0x7fa09c10a290 2023-08-11T14:10:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.594+0000 7fa0b779bb80 1 Processor -- start 2023-08-11T14:10:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 -- start start 2023-08-11T14:10:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b23172360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 0x561b23173c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b230cb5d0 con 0x561b231728a0 2023-08-11T14:10:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b230c8ef0 con 0x561b23173790 2023-08-11T14:10:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.595+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b230c47e0 con 0x561b23171f40 2023-08-11T14:10:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.596+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 0x561b23173c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.596+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 0x561b23173c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43760/0 (socket says 172.21.15.3:43760) 2023-08-11T14:10:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.596+0000 7fa0a139a700 1 -- 172.21.15.3:0/2177932335 learned_addr learned my addr 172.21.15.3:0/2177932335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.597+0000 7fa0a239c700 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b23172360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 -- 172.21.15.3:0/2177932335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 msgr2=0x561b23173c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 0x561b23173c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 -- 172.21.15.3:0/2177932335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 msgr2=0x561b231731b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:10:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231731b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 -- 172.21.15.3:0/2177932335 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b22f445d0 con 0x561b23171f40 2023-08-11T14:10:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.601+0000 7fa0a239c700 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b23172360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009990 tx=0x7fa08c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7b79d19509c7ba server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.602+0000 7fa087fff700 1 -- 172.21.15.3:0/2177932335 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00e710 con 0x561b23171f40 2023-08-11T14:10:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.602+0000 7fa087fff700 1 -- 172.21.15.3:0/2177932335 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08c00ed30 con 0x561b23171f40 2023-08-11T14:10:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.602+0000 7fa087fff700 1 -- 172.21.15.3:0/2177932335 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0123d0 con 0x561b23171f40 2023-08-11T14:10:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 -- 172.21.15.3:0/2177932335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 msgr2=0x561b23172360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b23172360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009990 tx=0x7fa08c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 -- 172.21.15.3:0/2177932335 shutdown_connections 2023-08-11T14:10:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23173790 0x561b23173c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b23172360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/2177932335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.603+0000 7fa0b779bb80 1 -- 172.21.15.3:0/2177932335 >> 172.21.15.3:0/2177932335 conn(0x561b2309f8f0 msgr2=0x561b2316e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.622+0000 7fa0b779bb80 1 -- 172.21.15.3:0/2177932335 shutdown_connections 2023-08-11T14:10:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.622+0000 7fa092ffd700 1 -- 172.21.15.3:0/1101492836 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa094004ad0 con 0x7fa09c10a290 2023-08-11T14:10:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.624+0000 7fa092ffd700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 0x7fa0880a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.624+0000 7fa092ffd700 1 -- 172.21.15.3:0/1101492836 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa0940d1bc0 con 0x7fa09c10a290 2023-08-11T14:10:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.628+0000 7fa0b779bb80 1 -- 172.21.15.3:0/2177932335 wait complete. 2023-08-11T14:10:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.628+0000 7fa0b779bb80 1 -- start start 2023-08-11T14:10:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.629+0000 7fa0b779bb80 1 Processor -- start 2023-08-11T14:10:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.634+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 0x7fa0880a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.637+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 0x7fa0880a2520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa098005950 tx=0x7fa098019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23171f40 0x561b2317bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23181200 0x561b231835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b23062a60 con 0x561b23171f40 2023-08-11T14:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b23062760 con 0x561b231728a0 2023-08-11T14:10:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b22f468c0 con 0x561b23181200 2023-08-11T14:10:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.640+0000 7fa0b779bb80 10 client.0 _ll_register_callbacks cb 0x561b23065a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.651+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43768/0 (socket says 172.21.15.3:43768) 2023-08-11T14:10:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 -- 172.21.15.3:0/412277453 learned_addr learned my addr 172.21.15.3:0/412277453 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a239c700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23181200 0x561b231835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 -- 172.21.15.3:0/412277453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23181200 msgr2=0x561b231835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23181200 0x561b231835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 -- 172.21.15.3:0/412277453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23171f40 msgr2=0x561b2317bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23171f40 0x561b2317bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.653+0000 7fa0a139a700 1 -- 172.21.15.3:0/412277453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c0095c0 con 0x561b231728a0 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.654+0000 7fa0a139a700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa09409f220 tx=0x7fa0940094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f80d177c6988c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.654+0000 7fa0857fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:10:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.654+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09409ebf0 con 0x561b231728a0 2023-08-11T14:10:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.654+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b231657e0 con 0x561b231728a0 2023-08-11T14:10:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.655+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa09409f990 con 0x561b231728a0 2023-08-11T14:10:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.655+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09409d470 con 0x561b231728a0 2023-08-11T14:10:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.655+0000 7fa0b779bb80 10 client.26294 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.655+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561b22ef0d80 con 0x561b231728a0 2023-08-11T14:10:45.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.660+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0940bb080 con 0x561b231728a0 2023-08-11T14:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.660+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa0940a6c10 con 0x561b231728a0 2023-08-11T14:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 10 client.26294 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 20 client.26294 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b23187f00 0x561b2318a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 4 client.26294 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.661+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b23141480 con 0x561b23187f00 2023-08-11T14:10:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.667+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b23187f00 0x561b2318a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.668+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b23187f00 0x561b2318a340 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec695b98c0574837 server_cookie=b49113896ef1c8a7 in_seq=0 out_seq=0 2023-08-11T14:10:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.674+0000 7fa0857fa700 10 client.26294 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.675+0000 7fa0857fa700 1 -- 172.21.15.3:0/412277453 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561b23141480 con 0x561b23187f00 2023-08-11T14:10:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.675+0000 7fa0857fa700 10 client.26294 handle_command_reply: tid=0 2023-08-11T14:10:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.675+0000 7fa0b779bb80 1 client.26294 shutdown 2023-08-11T14:10:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.675+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 msgr2=0x561b2317c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.675+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa09409f220 tx=0x7fa0940094b0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 shutdown_connections 2023-08-11T14:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23181200 0x561b231835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23171f40 0x561b2317bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b23187f00 0x561b2318a340 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/412277453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b231728a0 0x561b2317c1d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 >> 172.21.15.3:0/412277453 conn(0x561b2309f8f0 msgr2=0x561b2316d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.684+0000 7fa0857fa700 0 client.26294 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.702+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 shutdown_connections 2023-08-11T14:10:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.702+0000 7fa0b779bb80 1 -- 172.21.15.3:0/412277453 wait complete. 2023-08-11T14:10:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.710+0000 7fa0b779bb80 20 client.26294 trim_cache size 0 max 16384 2023-08-11T14:10:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.740+0000 7fa0b779bb80 1 Processor -- start 2023-08-11T14:10:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.741+0000 7fa0b779bb80 1 -- start start 2023-08-11T14:10:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.741+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b2318ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 0x561b2317aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b23165630 con 0x561b231728a0 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2317a340 con 0x561b23181200 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b23172c80 con 0x561b23171f40 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.742+0000 7fa0a1b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b2318ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.743+0000 7fa0a1b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b2318ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60504/0 (socket says 172.21.15.3:60504) 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.743+0000 7fa0a1b9b700 1 -- 172.21.15.3:0/4047885309 learned_addr learned my addr 172.21.15.3:0/4047885309 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.743+0000 7fa0a239c700 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 0x561b2317aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 -- 172.21.15.3:0/4047885309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 msgr2=0x561b2318ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b2318ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 -- 172.21.15.3:0/4047885309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 msgr2=0x561b231979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 -- 172.21.15.3:0/4047885309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b22ed9c00 con 0x561b23181200 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.744+0000 7fa0a239c700 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 0x561b2317aa50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c013d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742f26a8366b25a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.745+0000 7fa090bff700 1 -- 172.21.15.3:0/4047885309 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00eea0 con 0x561b23181200 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.745+0000 7fa090bff700 1 -- 172.21.15.3:0/4047885309 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08c013e90 con 0x561b23181200 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.745+0000 7fa090bff700 1 -- 172.21.15.3:0/4047885309 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c004580 con 0x561b23181200 2023-08-11T14:10:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 -- 172.21.15.3:0/4047885309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 msgr2=0x561b2317aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 0x561b2317aa50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c013d20 comp rx=0 tx=0).stop 2023-08-11T14:10:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 -- 172.21.15.3:0/4047885309 shutdown_connections 2023-08-11T14:10:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23181200 0x561b2317aa50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b23171f40 0x561b2318ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/4047885309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b231728a0 0x561b231979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.746+0000 7fa0b779bb80 1 -- 172.21.15.3:0/4047885309 >> 172.21.15.3:0/4047885309 conn(0x561b2309f8f0 msgr2=0x561b23169740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.760+0000 7fa0b779bb80 1 -- 172.21.15.3:0/4047885309 shutdown_connections 2023-08-11T14:10:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.763+0000 7fa0b779bb80 1 -- 172.21.15.3:0/4047885309 wait complete. 2023-08-11T14:10:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.763+0000 7fa0b779bb80 1 -- start start 2023-08-11T14:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.763+0000 7fa0b779bb80 1 Processor -- start 2023-08-11T14:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 0x561b231d6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 0x561b231d87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 0x561b231dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b231db230 con 0x561b23181200 2023-08-11T14:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b231db370 con 0x561b23171f40 2023-08-11T14:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b231db4b0 con 0x561b231728a0 2023-08-11T14:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.774+0000 7fa0b779bb80 10 client.0 _ll_register_callbacks cb 0x561b23065a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.775+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 0x561b231d87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.775+0000 7fa0a139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 0x561b231d87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60506/0 (socket says 172.21.15.3:60506) 2023-08-11T14:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.775+0000 7fa0a139a700 1 -- 172.21.15.3:0/1238658597 learned_addr learned my addr 172.21.15.3:0/1238658597 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.775+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 0x561b231d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.776+0000 7fa0a1b9b700 1 -- 172.21.15.3:0/1238658597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 msgr2=0x561b231d87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.776+0000 7fa0a239c700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 0x561b231dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.776+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 0x561b231d87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.776+0000 7fa0a1b9b700 1 -- 172.21.15.3:0/1238658597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 msgr2=0x561b231dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.777+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 0x561b231dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.777+0000 7fa0a1b9b700 1 -- 172.21.15.3:0/1238658597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b22f445d0 con 0x561b23171f40 2023-08-11T14:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.777+0000 7fa0a239c700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 0x561b231dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:10:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.781+0000 7fa0a1b9b700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 0x561b231d6250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x561b2317cbb0 tx=0x7fa0980058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f6ed3076c41c77 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.781+0000 7fa0857fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:10:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.781+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09802a040 con 0x561b23171f40 2023-08-11T14:10:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.781+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa098005220 con 0x561b23171f40 2023-08-11T14:10:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.781+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa08c00ed30 con 0x561b23171f40 2023-08-11T14:10:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.782+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0980053e0 con 0x561b23171f40 2023-08-11T14:10:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.786+0000 7fa0b779bb80 10 client.26300 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.786+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561b22f171f0 con 0x561b23171f40 2023-08-11T14:10:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.786+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08c00ed30 con 0x561b23171f40 2023-08-11T14:10:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.786+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa09800b9b0 con 0x561b23171f40 2023-08-11T14:10:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 10 client.26300 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 20 client.26300 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b231dcee0 0x561b231df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 4 client.26300 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b230bd7c0 con 0x561b231dcee0 2023-08-11T14:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.787+0000 7fa0a139a700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b231dcee0 0x561b231df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.788+0000 7fa0a139a700 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b231dcee0 0x561b231df300 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5fea12d481cf9c server_cookie=9ab30ed6d85cb960 in_seq=0 out_seq=0 2023-08-11T14:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.788+0000 7fa0857fa700 10 client.26300 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0857fa700 1 -- 172.21.15.3:0/1238658597 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b230bd7c0 con 0x561b231dcee0 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0857fa700 10 client.26300 handle_command_reply: tid=0 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 client.26300 shutdown 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 msgr2=0x561b231d6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 0x561b231d6250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x561b2317cbb0 tx=0x7fa0980058e0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 shutdown_connections 2023-08-11T14:10:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561b23181200 0x561b231dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561b231dcee0 0x561b231df300 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b23171f40 0x561b231d6250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 --2- 172.21.15.3:0/1238658597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561b231728a0 0x561b231d87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 >> 172.21.15.3:0/1238658597 conn(0x561b2309f8f0 msgr2=0x561b2317aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.904+0000 7fa0857fa700 0 client.26300 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.905+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 shutdown_connections 2023-08-11T14:10:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.905+0000 7fa0b779bb80 1 -- 172.21.15.3:0/1238658597 wait complete. 2023-08-11T14:10:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.905+0000 7fa0b779bb80 20 client.26300 trim_cache size 0 max 16384 2023-08-11T14:10:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:10:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:10:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:10:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:10:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.906+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 msgr2=0x7fa0880a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.906+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 0x7fa0880a2520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa098005950 tx=0x7fa098019040 comp rx=0 tx=0).stop 2023-08-11T14:10:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.906+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 msgr2=0x7fa09c1a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.906+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa094007fa0 tx=0x7fa09400d3b0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 shutdown_connections 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa09c10cc60 0x7fa09c1aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa0880a0060 0x7fa0880a2520 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c107920 0x7fa09c1a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 --2- 172.21.15.3:0/1101492836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa09c10a290 0x7fa09c1a59d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.907+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 >> 172.21.15.3:0/1101492836 conn(0x7fa09c06d710 msgr2=0x7fa09c10bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.908+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 shutdown_connections 2023-08-11T14:10:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:45.908+0000 7fa0a3dff700 1 -- 172.21.15.3:0/1101492836 wait complete. 2023-08-11T14:10:47.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:47 smithi103 ceph-mon[130434]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-08-11T14:10:48.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:47 smithi187 ceph-mon[130973]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-08-11T14:10:48.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:47 smithi003 ceph-mon[97195]: pgmap v553: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 25 MiB/s wr, 2.00k op/s 2023-08-11T14:10:48.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 230.0 s, 476.8 tps, lat 207.815 ms stddev 448.698 2023-08-11T14:10:50.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:49 smithi103 ceph-mon[130434]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-08-11T14:10:50.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:49 smithi187 ceph-mon[130973]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-08-11T14:10:50.308 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:49 smithi003 ceph-mon[97195]: pgmap v554: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 21 MiB/s wr, 1.68k op/s 2023-08-11T14:10:52.185 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:51 smithi103 ceph-mon[130434]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-08-11T14:10:52.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:51 smithi187 ceph-mon[130973]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-08-11T14:10:52.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:51 smithi003 ceph-mon[97195]: pgmap v555: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.82k op/s 2023-08-11T14:10:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 235.0 s, 1260.3 tps, lat 80.176 ms stddev 83.639 2023-08-11T14:10:54.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:53 smithi103 ceph-mon[130434]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-08-11T14:10:54.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:53 smithi003 ceph-mon[97195]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-08-11T14:10:54.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:53 smithi187 ceph-mon[130973]: pgmap v556: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 23 MiB/s wr, 1.79k op/s 2023-08-11T14:10:56.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:55 smithi003 ceph-mon[97195]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:10:56.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:55 smithi187 ceph-mon[130973]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:10:56.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:55 smithi103 ceph-mon[130434]: pgmap v557: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:10:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 240.0 s, 858.5 tps, lat 114.320 ms stddev 328.148 2023-08-11T14:10:58.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:10:57 smithi003 ceph-mon[97195]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.10k op/s 2023-08-11T14:10:58.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:10:57 smithi187 ceph-mon[130973]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.10k op/s 2023-08-11T14:10:58.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:10:57 smithi103 ceph-mon[130434]: pgmap v558: 129 pgs: 129 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 27 MiB/s wr, 2.10k op/s 2023-08-11T14:10:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.212+0000 7f4641bff700 1 -- 172.21.15.3:0/4269055363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10a290 msgr2=0x7f463c10c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.212+0000 7f4641bff700 1 --2- 172.21.15.3:0/4269055363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10a290 0x7f463c10c720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4634009f90 tx=0x7f463400b330 comp rx=0 tx=0).stop 2023-08-11T14:10:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.218+0000 7f4641bff700 1 -- 172.21.15.3:0/4269055363 shutdown_connections 2023-08-11T14:10:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.218+0000 7f4641bff700 1 --2- 172.21.15.3:0/4269055363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c10cc60 0x7f463c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.219+0000 7f4641bff700 1 --2- 172.21.15.3:0/4269055363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c107920 0x7f463c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.219+0000 7f4641bff700 1 --2- 172.21.15.3:0/4269055363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10a290 0x7f463c10c720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.219+0000 7f4641bff700 1 -- 172.21.15.3:0/4269055363 >> 172.21.15.3:0/4269055363 conn(0x7f463c06d750 msgr2=0x7f463c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.238+0000 7f4641bff700 1 -- 172.21.15.3:0/4269055363 shutdown_connections 2023-08-11T14:10:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.247+0000 7f4641bff700 1 -- 172.21.15.3:0/4269055363 wait complete. 2023-08-11T14:10:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.248+0000 7f4641bff700 1 Processor -- start 2023-08-11T14:10:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 -- start start 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 0x7f463c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 0x7f463c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10cc60 0x7f463c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c19e6a0 con 0x7f463c107920 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c19e810 con 0x7f463c10cc60 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.256+0000 7f4641bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c19e9d0 con 0x7f463c10a290 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.260+0000 7f463b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 0x7f463c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.260+0000 7f463b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 0x7f463c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54752/0 (socket says 172.21.15.3:54752) 2023-08-11T14:10:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.260+0000 7f463b7fe700 1 -- 172.21.15.3:0/3637772239 learned_addr learned my addr 172.21.15.3:0/3637772239 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.261+0000 7f463affd700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 0x7f463c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.272+0000 7f463bfff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10cc60 0x7f463c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.277+0000 7f463affd700 1 -- 172.21.15.3:0/3637772239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10cc60 msgr2=0x7f463c19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.277+0000 7f463affd700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10cc60 0x7f463c19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.277+0000 7f463affd700 1 -- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 msgr2=0x7f463c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.277+0000 7f463affd700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 0x7f463c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.278+0000 7f463affd700 1 -- 172.21.15.3:0/3637772239 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009c40 con 0x7f463c10a290 2023-08-11T14:10:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.284+0000 7f463affd700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 0x7f463c19d260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4634000f80 tx=0x7f463400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.305+0000 7f4638ff9700 1 -- 172.21.15.3:0/3637772239 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400e040 con 0x7f463c10a290 2023-08-11T14:10:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.305+0000 7f4638ff9700 1 -- 172.21.15.3:0/3637772239 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4634004e60 con 0x7f463c10a290 2023-08-11T14:10:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.305+0000 7f4638ff9700 1 -- 172.21.15.3:0/3637772239 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463401b7c0 con 0x7f463c10a290 2023-08-11T14:10:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.305+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c19ec50 con 0x7f463c10a290 2023-08-11T14:10:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.305+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c1a53d0 con 0x7f463c10a290 2023-08-11T14:10:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.312+0000 7f4655604b80 1 Processor -- start 2023-08-11T14:10:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 -- start start 2023-08-11T14:10:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 0x562bc37c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37c08a0 0x562bc37c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37195d0 con 0x562bc37c08a0 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc3716ef0 con 0x562bc37c1790 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.323+0000 7f4655604b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37127e0 con 0x562bc37bff40 2023-08-11T14:10:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.324+0000 7f463bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 0x562bc37c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.330+0000 7f463affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.330+0000 7f4638ff9700 1 -- 172.21.15.3:0/3637772239 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f463401dac0 con 0x7f463c10a290 2023-08-11T14:10:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.332+0000 7f4638ff9700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 0x7f46240a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.333+0000 7f4638ff9700 1 -- 172.21.15.3:0/3637772239 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f46340cc180 con 0x7f463c10a290 2023-08-11T14:10:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.333+0000 7f463affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46142/0 (socket says 172.21.15.3:46142) 2023-08-11T14:10:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.333+0000 7f463affd700 1 -- 172.21.15.3:0/1236043653 learned_addr learned my addr 172.21.15.3:0/1236043653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.345+0000 7f463bfff700 1 -- 172.21.15.3:0/1236043653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 msgr2=0x562bc37c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.345+0000 7f463bfff700 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.345+0000 7f463bfff700 1 -- 172.21.15.3:0/1236043653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37c08a0 msgr2=0x562bc37c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.345+0000 7f463bfff700 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37c08a0 0x562bc37c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.345+0000 7f463bfff700 1 -- 172.21.15.3:0/1236043653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc35925d0 con 0x562bc37bff40 2023-08-11T14:10:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.347+0000 7f463bfff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 0x7f46240a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.348+0000 7f463bfff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 0x7f46240a2610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cfd0 tx=0x7f462c013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.356+0000 7f463bfff700 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 0x562bc37c0360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cc30 tx=0x7f462c00cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a0ccd8554f0a67 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.370+0000 7f4621ffb700 1 -- 172.21.15.3:0/1236043653 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c010bf0 con 0x562bc37bff40 2023-08-11T14:10:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.370+0000 7f4621ffb700 1 -- 172.21.15.3:0/1236043653 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f462c010e50 con 0x562bc37bff40 2023-08-11T14:10:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.370+0000 7f4621ffb700 1 -- 172.21.15.3:0/1236043653 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005580 con 0x562bc37bff40 2023-08-11T14:10:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.371+0000 7f4655604b80 1 -- 172.21.15.3:0/1236043653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 msgr2=0x562bc37c0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.371+0000 7f4655604b80 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 0x562bc37c0360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cc30 tx=0x7f462c00cf40 comp rx=0 tx=0).stop 2023-08-11T14:10:59.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.382+0000 7f4655604b80 1 -- 172.21.15.3:0/1236043653 shutdown_connections 2023-08-11T14:10:59.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.382+0000 7f4655604b80 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.382+0000 7f4655604b80 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37bff40 0x562bc37c0360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.382+0000 7f4655604b80 1 --2- 172.21.15.3:0/1236043653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37c08a0 0x562bc37c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.382+0000 7f4655604b80 1 -- 172.21.15.3:0/1236043653 >> 172.21.15.3:0/1236043653 conn(0x562bc36ed8f0 msgr2=0x562bc37bc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.393+0000 7f4655604b80 1 -- 172.21.15.3:0/1236043653 shutdown_connections 2023-08-11T14:10:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.401+0000 7f4655604b80 1 -- 172.21.15.3:0/1236043653 wait complete. 2023-08-11T14:10:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.402+0000 7f4655604b80 1 -- start start 2023-08-11T14:10:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.402+0000 7f4655604b80 1 Processor -- start 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c08a0 0x562bc37ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc36b0a60 con 0x562bc37bff40 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc36b0760 con 0x562bc37c1790 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc35948c0 con 0x562bc37c08a0 2023-08-11T14:10:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.403+0000 7f4655604b80 10 client.0 _ll_register_callbacks cb 0x562bc36b3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.404+0000 7f463affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c08a0 0x562bc37ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.404+0000 7f463b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.404+0000 7f463b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54770/0 (socket says 172.21.15.3:54770) 2023-08-11T14:10:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.404+0000 7f463b7fe700 1 -- 172.21.15.3:0/2873571532 learned_addr learned my addr 172.21.15.3:0/2873571532 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.405+0000 7f463b7fe700 1 -- 172.21.15.3:0/2873571532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c08a0 msgr2=0x562bc37ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.405+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c08a0 0x562bc37ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.405+0000 7f463b7fe700 1 -- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 msgr2=0x562bc37d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.405+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.405+0000 7f463b7fe700 1 -- 172.21.15.3:0/2873571532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c00c860 con 0x562bc37bff40 2023-08-11T14:10:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.406+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4630021a00 tx=0x7f4630021dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aab1332a06b1ad server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.412+0000 7f461affd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:10:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.412+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463001f7d0 con 0x562bc37bff40 2023-08-11T14:10:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.412+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f463001f930 con 0x562bc37bff40 2023-08-11T14:10:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.412+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bc37b37e0 con 0x562bc37bff40 2023-08-11T14:10:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.432+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f463001faf0 con 0x562bc37bff40 2023-08-11T14:10:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.433+0000 7f4655604b80 10 client.16179 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.433+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562bc353ed80 con 0x562bc37bff40 2023-08-11T14:10:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.444+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bc37b37e0 con 0x562bc37bff40 2023-08-11T14:10:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.444+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4630022450 con 0x562bc37bff40 2023-08-11T14:10:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 20 client.16179 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc37d5af0 0x562bc37d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 4 client.16179 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:10:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.446+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc378f480 con 0x562bc37d5af0 2023-08-11T14:10:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.449+0000 7f463affd700 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc37d5af0 0x562bc37d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.456+0000 7f463affd700 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc37d5af0 0x562bc37d7f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634488dac3d90cba server_cookie=b2d82d3bd1a58bfc in_seq=0 out_seq=0 2023-08-11T14:10:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.468+0000 7f461affd700 10 client.16179 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.468+0000 7f461affd700 1 -- 172.21.15.3:0/2873571532 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562bc378f480 con 0x562bc37d5af0 2023-08-11T14:10:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.468+0000 7f461affd700 10 client.16179 handle_command_reply: tid=0 2023-08-11T14:10:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.469+0000 7f4655604b80 1 client.16179 shutdown 2023-08-11T14:10:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.495+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 msgr2=0x562bc37c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.495+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4630021a00 tx=0x7f4630021dc0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 shutdown_connections 2023-08-11T14:10:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc37d5af0 0x562bc37d7f10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c1790 0x562bc37d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37c9c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 --2- 172.21.15.3:0/2873571532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c08a0 0x562bc37ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.501+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 >> 172.21.15.3:0/2873571532 conn(0x562bc36ed8f0 msgr2=0x562bc37bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.503+0000 7f461affd700 0 client.16179 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:10:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.532+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 shutdown_connections 2023-08-11T14:10:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.551+0000 7f4655604b80 1 -- 172.21.15.3:0/2873571532 wait complete. 2023-08-11T14:10:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.553+0000 7f4655604b80 20 client.16179 trim_cache size 0 max 16384 2023-08-11T14:10:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.595+0000 7f4655604b80 1 Processor -- start 2023-08-11T14:10:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 -- start start 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37bc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37c91e0 con 0x562bc37bff40 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37b37c0 con 0x562bc37c08a0 2023-08-11T14:10:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.596+0000 7f4655604b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37b5ef0 con 0x562bc37c1790 2023-08-11T14:10:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.605+0000 7f463affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.605+0000 7f463affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46162/0 (socket says 172.21.15.3:46162) 2023-08-11T14:10:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.605+0000 7f463affd700 1 -- 172.21.15.3:0/520749015 learned_addr learned my addr 172.21.15.3:0/520749015 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.605+0000 7f463bfff700 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.610+0000 7f463b7fe700 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37bc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.612+0000 7f463bfff700 1 -- 172.21.15.3:0/520749015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 msgr2=0x562bc37d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.612+0000 7f463bfff700 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.612+0000 7f463bfff700 1 -- 172.21.15.3:0/520749015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 msgr2=0x562bc37bc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.612+0000 7f463bfff700 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37bc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.613+0000 7f463bfff700 1 -- 172.21.15.3:0/520749015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc3527c00 con 0x562bc37c1790 2023-08-11T14:10:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.641+0000 7f463bfff700 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37bc550 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c830 tx=0x7f462c010bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751fa9a60ef8bc5e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.653+0000 7f46227fc700 1 -- 172.21.15.3:0/520749015 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c012d50 con 0x562bc37c1790 2023-08-11T14:10:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.653+0000 7f46227fc700 1 -- 172.21.15.3:0/520749015 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f462c005690 con 0x562bc37c1790 2023-08-11T14:10:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.653+0000 7f46227fc700 1 -- 172.21.15.3:0/520749015 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c004f80 con 0x562bc37c1790 2023-08-11T14:10:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.654+0000 7f4655604b80 1 -- 172.21.15.3:0/520749015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 msgr2=0x562bc37bc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:10:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.654+0000 7f4655604b80 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37bc550 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c830 tx=0x7f462c010bf0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.655+0000 7f4655604b80 1 -- 172.21.15.3:0/520749015 shutdown_connections 2023-08-11T14:10:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.655+0000 7f4655604b80 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37bc550 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.655+0000 7f4655604b80 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37bc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.655+0000 7f4655604b80 1 --2- 172.21.15.3:0/520749015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.655+0000 7f4655604b80 1 -- 172.21.15.3:0/520749015 >> 172.21.15.3:0/520749015 conn(0x562bc36ed8f0 msgr2=0x562bc37c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:10:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.673+0000 7f4655604b80 1 -- 172.21.15.3:0/520749015 shutdown_connections 2023-08-11T14:10:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.676+0000 7f4655604b80 1 -- 172.21.15.3:0/520749015 wait complete. 2023-08-11T14:10:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.676+0000 7f4655604b80 1 -- start start 2023-08-11T14:10:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.677+0000 7f4655604b80 1 Processor -- start 2023-08-11T14:10:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.677+0000 7f4655604b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37d0930 con 0x562bc37bff40 2023-08-11T14:10:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37d0aa0 con 0x562bc37c08a0 2023-08-11T14:10:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc37d0c60 con 0x562bc37c1790 2023-08-11T14:10:59.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.678+0000 7f4655604b80 10 client.0 _ll_register_callbacks cb 0x562bc36b3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:10:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.682+0000 7f463bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.682+0000 7f463bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38660/0 (socket says 172.21.15.3:38660) 2023-08-11T14:10:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.682+0000 7f463bfff700 1 -- 172.21.15.3:0/2235796329 learned_addr learned my addr 172.21.15.3:0/2235796329 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:10:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.685+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.687+0000 7f463bfff700 1 -- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 msgr2=0x562bc37cf3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:10:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.687+0000 7f463bfff700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37cf3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.687+0000 7f463bfff700 1 -- 172.21.15.3:0/2235796329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 msgr2=0x562bc37ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:10:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.687+0000 7f463bfff700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:10:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.687+0000 7f463bfff700 1 -- 172.21.15.3:0/2235796329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bc35925d0 con 0x562bc37c1790 2023-08-11T14:10:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.692+0000 7f463bfff700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f462c012f80 tx=0x7f462c004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78ce3b6a74067072 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:10:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.692+0000 7f461affd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:10:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.700+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005050 con 0x562bc37c1790 2023-08-11T14:10:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.700+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f462c018040 con 0x562bc37c1790 2023-08-11T14:10:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.700+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c010e50 con 0x562bc37c1790 2023-08-11T14:10:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.706+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462c0051b0 con 0x562bc37c1790 2023-08-11T14:10:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.707+0000 7f4655604b80 10 client.26302 fetch_fsmap learned FSMap version 33 2023-08-11T14:10:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.707+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562bc35651f0 con 0x562bc37c1790 2023-08-11T14:10:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.719+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f462c010e50 con 0x562bc37c1790 2023-08-11T14:10:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.719+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f462c025460 con 0x562bc37c1790 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 10 client.26302 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 10 client.26302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 20 client.26302 populate_metadata read hostname 'smithi003' 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc382a200 0x562bc382c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 4 client.26302 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:10:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.729+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562bc370b7c0 con 0x562bc382a200 2023-08-11T14:10:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.740+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc382a200 0x562bc382c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:10:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.758+0000 7f463b7fe700 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc382a200 0x562bc382c620 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cca503439ad57 server_cookie=37881fc1b59a3057 in_seq=0 out_seq=0 2023-08-11T14:10:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:10:59.758+0000 7f461affd700 10 client.26302 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:00.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:00 smithi187 ceph-mon[130973]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:00.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:00 smithi003 ceph-mon[97195]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:00.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:00 smithi103 ceph-mon[130434]: pgmap v559: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:01.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:01 smithi103 ceph-mon[130434]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:01.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:01 smithi003 ceph-mon[97195]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:01.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:01 smithi187 ceph-mon[130973]: pgmap v560: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.309+0000 7f461affd700 1 -- 172.21.15.3:0/2235796329 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562bc370b7c0 con 0x562bc382a200 2023-08-11T14:11:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.309+0000 7f461affd700 10 client.26302 handle_command_reply: tid=0 2023-08-11T14:11:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.309+0000 7f4655604b80 1 client.26302 shutdown 2023-08-11T14:11:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.309+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 msgr2=0x562bc37d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:02.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.309+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f462c012f80 tx=0x7f462c004a00 comp rx=0 tx=0).stop 2023-08-11T14:11:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 shutdown_connections 2023-08-11T14:11:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bc37c1790 0x562bc37d03f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562bc37bff40 0x562bc37ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x562bc382a200 0x562bc382c620 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 --2- 172.21.15.3:0/2235796329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562bc37c08a0 0x562bc37cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.310+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 >> 172.21.15.3:0/2235796329 conn(0x562bc36ed8f0 msgr2=0x562bc37bc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.311+0000 7f461affd700 0 client.26302 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.311+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 shutdown_connections 2023-08-11T14:11:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.311+0000 7f4655604b80 1 -- 172.21.15.3:0/2235796329 wait complete. 2023-08-11T14:11:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.312+0000 7f4655604b80 20 client.26302 trim_cache size 0 max 16384 2023-08-11T14:11:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:11:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:11:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:11:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:11:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.316+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 msgr2=0x7f46240a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.316+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 0x7f46240a2610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cfd0 tx=0x7f462c013000 comp rx=0 tx=0).stop 2023-08-11T14:11:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.316+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 msgr2=0x7f463c19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.316+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 0x7f463c19d260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4634000f80 tx=0x7f463400bcd0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 shutdown_connections 2023-08-11T14:11:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f46240a0150 0x7f46240a2610 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463c10cc60 0x7f463c19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c107920 0x7f463c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 --2- 172.21.15.3:0/3637772239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f463c10a290 0x7f463c19d260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 >> 172.21.15.3:0/3637772239 conn(0x7f463c06d750 msgr2=0x7f463c10ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 shutdown_connections 2023-08-11T14:11:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:02.318+0000 7f4641bff700 1 -- 172.21.15.3:0/3637772239 wait complete. 2023-08-11T14:11:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 245.0 s, 562.8 tps, lat 181.410 ms stddev 461.057 2023-08-11T14:11:04.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:03 smithi103 ceph-mon[130434]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-08-11T14:11:04.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:03 smithi003 ceph-mon[97195]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-08-11T14:11:04.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:03 smithi187 ceph-mon[130973]: pgmap v561: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 26 MiB/s wr, 2.16k op/s 2023-08-11T14:11:06.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:05 smithi103 ceph-mon[130434]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.89k op/s 2023-08-11T14:11:06.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:05 smithi003 ceph-mon[97195]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.89k op/s 2023-08-11T14:11:06.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:05 smithi187 ceph-mon[130973]: pgmap v562: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 22 MiB/s wr, 1.89k op/s 2023-08-11T14:11:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 250.0 s, 713.6 tps, lat 135.197 ms stddev 243.280 2023-08-11T14:11:08.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:07 smithi103 ceph-mon[130434]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-08-11T14:11:08.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:07 smithi187 ceph-mon[130973]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-08-11T14:11:08.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:07 smithi003 ceph-mon[97195]: pgmap v563: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 25 MiB/s wr, 2.16k op/s 2023-08-11T14:11:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:09 smithi103 ceph-mon[130434]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:09.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:09 smithi187 ceph-mon[130973]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:09.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:09 smithi003 ceph-mon[97195]: pgmap v564: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:11.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:11 smithi103 ceph-mon[130434]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:12.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:11 smithi003 ceph-mon[97195]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:12.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:11 smithi187 ceph-mon[130973]: pgmap v565: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 21 MiB/s wr, 1.76k op/s 2023-08-11T14:11:13.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:12 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:11:13.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:12 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:11:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 255.0 s, 410.4 tps, lat 247.322 ms stddev 436.034 2023-08-11T14:11:13.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:12 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:11:14.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.95k op/s 2023-08-11T14:11:14.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:13 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.95k op/s 2023-08-11T14:11:14.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:13 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: pgmap v566: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 23 MiB/s wr, 1.95k op/s 2023-08-11T14:11:14.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:14.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:13 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:11:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:11:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:11:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:11:15.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:11:15.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:11:15.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:11:15.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:14 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:11:15.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:11:15.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:11:15.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:11:15.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:11:15.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:11:15.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:11:15.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:14 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:11:15.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:11:15.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:11:15.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:11:15.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:11:15.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:11:15.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:11:15.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:14 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:11:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.605+0000 7f43fefff700 1 -- 172.21.15.3:0/3276287829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810a290 msgr2=0x7f43f810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.605+0000 7f43fefff700 1 --2- 172.21.15.3:0/3276287829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810a290 0x7f43f810c720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43e001a9a0 tx=0x7f43e001acb0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.609+0000 7f43fefff700 1 -- 172.21.15.3:0/3276287829 shutdown_connections 2023-08-11T14:11:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.609+0000 7f43fefff700 1 --2- 172.21.15.3:0/3276287829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43f810cc60 0x7f43f810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.609+0000 7f43fefff700 1 --2- 172.21.15.3:0/3276287829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f8107920 0x7f43f8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.609+0000 7f43fefff700 1 --2- 172.21.15.3:0/3276287829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810a290 0x7f43f810c720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.609+0000 7f43fefff700 1 -- 172.21.15.3:0/3276287829 >> 172.21.15.3:0/3276287829 conn(0x7f43f806d750 msgr2=0x7f43f806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.619+0000 7f43fefff700 1 -- 172.21.15.3:0/3276287829 shutdown_connections 2023-08-11T14:11:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.624+0000 7f43fefff700 1 -- 172.21.15.3:0/3276287829 wait complete. 2023-08-11T14:11:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.625+0000 7f43fefff700 1 Processor -- start 2023-08-11T14:11:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.629+0000 7f43fefff700 1 -- start start 2023-08-11T14:11:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43f8107920 0x7f43f819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810cc60 0x7f43f819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819e6f0 con 0x7f43f8107920 2023-08-11T14:11:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819e860 con 0x7f43f810a290 2023-08-11T14:11:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.630+0000 7f43fefff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f819ea20 con 0x7f43f810cc60 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.631+0000 7f43effff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.631+0000 7f43effff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47914/0 (socket says 172.21.15.3:47914) 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.631+0000 7f43effff700 1 -- 172.21.15.3:0/646674101 learned_addr learned my addr 172.21.15.3:0/646674101 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.632+0000 7f43effff700 1 -- 172.21.15.3:0/646674101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810cc60 msgr2=0x7f43f819dff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.632+0000 7f43effff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810cc60 0x7f43f819dff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.632+0000 7f43effff700 1 -- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43f8107920 msgr2=0x7f43f819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.632+0000 7f43effff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43f8107920 0x7f43f819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.632+0000 7f43effff700 1 -- 172.21.15.3:0/646674101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e001a600 con 0x7f43f810a290 2023-08-11T14:11:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.633+0000 7f43effff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f43e0003790 tx=0x7f43e0028a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.636+0000 7f43f67fc700 1 -- 172.21.15.3:0/646674101 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e002d070 con 0x7f43f810a290 2023-08-11T14:11:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.636+0000 7f43f67fc700 1 -- 172.21.15.3:0/646674101 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e0028e30 con 0x7f43f810a290 2023-08-11T14:11:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.636+0000 7f43f67fc700 1 -- 172.21.15.3:0/646674101 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e00056b0 con 0x7f43f810a290 2023-08-11T14:11:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.636+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f819eca0 con 0x7f43f810a290 2023-08-11T14:11:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.636+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f81a5420 con 0x7f43f810a290 2023-08-11T14:11:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.644+0000 7f4412906b80 1 Processor -- start 2023-08-11T14:11:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.644+0000 7f4412906b80 1 -- start start 2023-08-11T14:11:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2c790 0x563e34e2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34d845d0 con 0x563e34e2af40 2023-08-11T14:11:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34d81ef0 con 0x563e34e2c790 2023-08-11T14:11:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.645+0000 7f4412906b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34d7d7e0 con 0x563e34e2b8a0 2023-08-11T14:11:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.651+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.651+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57376/0 (socket says 172.21.15.3:57376) 2023-08-11T14:11:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.651+0000 7f43fcd9b700 1 -- 172.21.15.3:0/4171591804 learned_addr learned my addr 172.21.15.3:0/4171591804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43f67fc700 1 -- 172.21.15.3:0/646674101 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f43e00050d0 con 0x7f43f810a290 2023-08-11T14:11:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43fcd9b700 1 -- 172.21.15.3:0/4171591804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2c790 msgr2=0x563e34e2cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2c790 0x563e34e2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43fcd9b700 1 -- 172.21.15.3:0/4171591804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 msgr2=0x563e34e2b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.655+0000 7f43fcd9b700 1 -- 172.21.15.3:0/4171591804 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e34bfd5d0 con 0x563e34e2b8a0 2023-08-11T14:11:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.657+0000 7f43f67fc700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 0x7f43e40a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.657+0000 7f43f67fc700 1 -- 172.21.15.3:0/646674101 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f43e00dd020 con 0x7f43f810a290 2023-08-11T14:11:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.659+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e400 tx=0x7f43e800e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6c807d131ed191 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.666+0000 7f43eeffd700 1 -- 172.21.15.3:0/4171591804 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018070 con 0x563e34e2b8a0 2023-08-11T14:11:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.666+0000 7f43eeffd700 1 -- 172.21.15.3:0/4171591804 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e800f040 con 0x563e34e2b8a0 2023-08-11T14:11:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.666+0000 7f43eeffd700 1 -- 172.21.15.3:0/4171591804 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8004280 con 0x563e34e2b8a0 2023-08-11T14:11:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.667+0000 7f4412906b80 1 -- 172.21.15.3:0/4171591804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 msgr2=0x563e34e2c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.667+0000 7f4412906b80 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e400 tx=0x7f43e800e710 comp rx=0 tx=0).stop 2023-08-11T14:11:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.668+0000 7f43fd59c700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 0x7f43e40a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.671+0000 7f4412906b80 1 -- 172.21.15.3:0/4171591804 shutdown_connections 2023-08-11T14:11:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.671+0000 7f4412906b80 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2c790 0x563e34e2cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.671+0000 7f4412906b80 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e2b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.671+0000 7f4412906b80 1 --2- 172.21.15.3:0/4171591804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e2c1b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.671+0000 7f4412906b80 1 -- 172.21.15.3:0/4171591804 >> 172.21.15.3:0/4171591804 conn(0x563e34d588f0 msgr2=0x563e34e27920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.689+0000 7f4412906b80 1 -- 172.21.15.3:0/4171591804 shutdown_connections 2023-08-11T14:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.700+0000 7f4412906b80 1 -- 172.21.15.3:0/4171591804 wait complete. 2023-08-11T14:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.700+0000 7f4412906b80 1 -- start start 2023-08-11T14:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.701+0000 7f4412906b80 1 Processor -- start 2023-08-11T14:11:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 0x563e34e3c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34d1ba60 con 0x563e34e2c790 2023-08-11T14:11:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34d1b760 con 0x563e34e2af40 2023-08-11T14:11:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.711+0000 7f4412906b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34bff8c0 con 0x563e34e2b8a0 2023-08-11T14:11:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.712+0000 7f4412906b80 10 client.0 _ll_register_callbacks cb 0x563e34d1ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.718+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.718+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47940/0 (socket says 172.21.15.3:47940) 2023-08-11T14:11:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.718+0000 7f43fcd9b700 1 -- 172.21.15.3:0/2228862003 learned_addr learned my addr 172.21.15.3:0/2228862003 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.723+0000 7f43fd59c700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 0x7f43e40a2610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009a20 tx=0x7f43f0005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.723+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 0x563e34e3c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.723+0000 7f43effff700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.728+0000 7f43fd59c700 1 -- 172.21.15.3:0/2228862003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 msgr2=0x563e34e351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.728+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.728+0000 7f43fd59c700 1 -- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 msgr2=0x563e34e34c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.728+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.728+0000 7f43fd59c700 1 -- 172.21.15.3:0/2228862003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e800e140 con 0x563e34e2c790 2023-08-11T14:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.730+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 0x563e34e3c1e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43f000d5c0 tx=0x7f43f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e44948657d529ac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.730+0000 7f43d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:11:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.731+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00109a0 con 0x563e34e2c790 2023-08-11T14:11:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.731+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f43f0010b00 con 0x563e34e2c790 2023-08-11T14:11:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.731+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e34e1e7e0 con 0x563e34e2c790 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.732+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f0010cc0 con 0x563e34e2c790 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.732+0000 7f4412906b80 10 client.16191 fetch_fsmap learned FSMap version 33 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.732+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563e34ba9d80 con 0x563e34e2c790 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f002a030 con 0x563e34e2c790 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43f000f4b0 con 0x563e34e2c790 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:11:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 20 client.16191 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e40b70 0x563e34e42f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 4 client.16191 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:11:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.733+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e34dfa480 con 0x563e34e40b70 2023-08-11T14:11:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.741+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e40b70 0x563e34e42f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.750+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e40b70 0x563e34e42f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb37369b3eefbc9 server_cookie=8614b7b3dc004976 in_seq=0 out_seq=0 2023-08-11T14:11:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.750+0000 7f43d7fff700 10 client.16191 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.751+0000 7f43d7fff700 1 -- 172.21.15.3:0/2228862003 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563e34dfa480 con 0x563e34e40b70 2023-08-11T14:11:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.751+0000 7f43d7fff700 10 client.16191 handle_command_reply: tid=0 2023-08-11T14:11:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.751+0000 7f4412906b80 1 client.16191 shutdown 2023-08-11T14:11:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.766+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 msgr2=0x563e34e3c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.766+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 0x563e34e3c1e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43f000d5c0 tx=0x7f43f0004750 comp rx=0 tx=0).stop 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 shutdown_connections 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2c790 0x563e34e3c1e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e40b70 0x563e34e42f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 --2- 172.21.15.3:0/2228862003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2b8a0 0x563e34e351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.767+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 >> 172.21.15.3:0/2228862003 conn(0x563e34d588f0 msgr2=0x563e34e26710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.769+0000 7f43d7fff700 0 client.16191 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.769+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 shutdown_connections 2023-08-11T14:11:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.776+0000 7f4412906b80 1 -- 172.21.15.3:0/2228862003 wait complete. 2023-08-11T14:11:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.779+0000 7f4412906b80 20 client.16191 trim_cache size 0 max 16384 2023-08-11T14:11:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.810+0000 7f4412906b80 1 Processor -- start 2023-08-11T14:11:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.811+0000 7f4412906b80 1 -- start start 2023-08-11T14:11:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.811+0000 7f4412906b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.811+0000 7f4412906b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 0x563e34e3c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.811+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.811+0000 7f43fcd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47956/0 (socket says 172.21.15.3:47956) 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.812+0000 7f43effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 0x563e34e3c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.812+0000 7f43effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 0x563e34e3c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55906/0 (socket says 172.21.15.3:55906) 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.816+0000 7f4412906b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.816+0000 7f4412906b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e2bfa0 con 0x563e34e2b8a0 2023-08-11T14:11:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.816+0000 7f4412906b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e329d0 con 0x563e34e2af40 2023-08-11T14:11:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.816+0000 7f4412906b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e302c0 con 0x563e34e2c790 2023-08-11T14:11:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.819+0000 7f43fcd9b700 1 -- 172.21.15.3:0/1274367420 learned_addr learned my addr 172.21.15.3:0/1274367420 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 -- 172.21.15.3:0/1274367420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 msgr2=0x563e34e34db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e34db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 -- 172.21.15.3:0/1274367420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 msgr2=0x563e34e3c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 0x563e34e3c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 -- 172.21.15.3:0/1274367420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e34b92c00 con 0x563e34e2af40 2023-08-11T14:11:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.821+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f43e800eb10 tx=0x7f43e8004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98338d44e0338c9c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.822+0000 7f43ef7fe700 1 -- 172.21.15.3:0/1274367420 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e802a070 con 0x563e34e2af40 2023-08-11T14:11:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.822+0000 7f43ef7fe700 1 -- 172.21.15.3:0/1274367420 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e8009190 con 0x563e34e2af40 2023-08-11T14:11:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.822+0000 7f43ef7fe700 1 -- 172.21.15.3:0/1274367420 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018040 con 0x563e34e2af40 2023-08-11T14:11:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 -- 172.21.15.3:0/1274367420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 msgr2=0x563e34e34870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f43e800eb10 tx=0x7f43e8004f60 comp rx=0 tx=0).stop 2023-08-11T14:11:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 -- 172.21.15.3:0/1274367420 shutdown_connections 2023-08-11T14:11:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e34db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2af40 0x563e34e34870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 --2- 172.21.15.3:0/1274367420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2b8a0 0x563e34e3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.823+0000 7f4412906b80 1 -- 172.21.15.3:0/1274367420 >> 172.21.15.3:0/1274367420 conn(0x563e34d588f0 msgr2=0x563e34e2f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.824+0000 7f4412906b80 1 -- 172.21.15.3:0/1274367420 shutdown_connections 2023-08-11T14:11:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.824+0000 7f4412906b80 1 -- 172.21.15.3:0/1274367420 wait complete. 2023-08-11T14:11:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.824+0000 7f4412906b80 1 -- start start 2023-08-11T14:11:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 Processor -- start 2023-08-11T14:11:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2b8a0 0x563e34e3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e3b8c0 con 0x563e34e2af40 2023-08-11T14:11:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e3ba30 con 0x563e34e2b8a0 2023-08-11T14:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.825+0000 7f4412906b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e34e3bbf0 con 0x563e34e2c790 2023-08-11T14:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.826+0000 7f4412906b80 10 client.0 _ll_register_callbacks cb 0x7f43f000d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.826+0000 7f43fd59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.826+0000 7f43fd59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57398/0 (socket says 172.21.15.3:57398) 2023-08-11T14:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.826+0000 7f43fd59c700 1 -- 172.21.15.3:0/2589707697 learned_addr learned my addr 172.21.15.3:0/2589707697 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 -- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2b8a0 msgr2=0x563e34e3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2b8a0 0x563e34e3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 -- 172.21.15.3:0/2589707697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 msgr2=0x563e34e39e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 -- 172.21.15.3:0/2589707697 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e34bfd5d0 con 0x563e34e2c790 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.827+0000 7f43fd59c700 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43f00107f0 tx=0x7f43f00233e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e75478bd186024 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.828+0000 7f43d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.828+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0010b30 con 0x563e34e2c790 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.828+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f43f0010c90 con 0x563e34e2c790 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.828+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43e800f040 con 0x563e34e2c790 2023-08-11T14:11:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.829+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f0010e50 con 0x563e34e2c790 2023-08-11T14:11:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.829+0000 7f4412906b80 10 client.26311 fetch_fsmap learned FSMap version 33 2023-08-11T14:11:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.829+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563e34bd01f0 con 0x563e34e2c790 2023-08-11T14:11:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e800f040 con 0x563e34e2c790 2023-08-11T14:11:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f43f0005490 con 0x563e34e2c790 2023-08-11T14:11:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 10 client.26311 fetch_fsmap finished waiting for FSMap version 33 2023-08-11T14:11:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 10 client.26311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 20 client.26311 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e955c0 0x563e34e979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 4 client.26311 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:11:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e34d767c0 con 0x563e34e955c0 2023-08-11T14:11:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.830+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e955c0 0x563e34e979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.832+0000 7f43fcd9b700 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e955c0 0x563e34e979e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87409a0d27b47ff server_cookie=734f31ac5b515fe3 in_seq=0 out_seq=0 2023-08-11T14:11:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.832+0000 7f43d7fff700 10 client.26311 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.958+0000 7f43d7fff700 1 -- 172.21.15.3:0/2589707697 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e34d767c0 con 0x563e34e955c0 2023-08-11T14:11:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.958+0000 7f43d7fff700 10 client.26311 handle_command_reply: tid=0 2023-08-11T14:11:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.958+0000 7f4412906b80 1 client.26311 shutdown 2023-08-11T14:11:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.968+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 msgr2=0x563e34e3b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.968+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43f00107f0 tx=0x7f43f00233e0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 shutdown_connections 2023-08-11T14:11:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e34e2c790 0x563e34e3b380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563e34e2af40 0x563e34e39e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563e34e955c0 0x563e34e979e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 --2- 172.21.15.3:0/2589707697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e34e2b8a0 0x563e34e3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 >> 172.21.15.3:0/2589707697 conn(0x563e34d588f0 msgr2=0x563e34e26f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.969+0000 7f43d7fff700 0 client.26311 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.970+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 shutdown_connections 2023-08-11T14:11:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.971+0000 7f4412906b80 1 -- 172.21.15.3:0/2589707697 wait complete. 2023-08-11T14:11:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.974+0000 7f4412906b80 20 client.26311 trim_cache size 0 max 16384 2023-08-11T14:11:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:11:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:11:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:11:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:11:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 msgr2=0x7f43e40a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 0x7f43e40a2610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009a20 tx=0x7f43f0005f70 comp rx=0 tx=0).stop 2023-08-11T14:11:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 msgr2=0x7f43f819d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f43e0003790 tx=0x7f43e0028a40 comp rx=0 tx=0).stop 2023-08-11T14:11:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 shutdown_connections 2023-08-11T14:11:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43f810cc60 0x7f43f819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43f8107920 0x7f43f819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f43e40a0150 0x7f43e40a2610 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 --2- 172.21.15.3:0/646674101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f810a290 0x7f43f819d2b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.980+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 >> 172.21.15.3:0/646674101 conn(0x7f43f806d750 msgr2=0x7f43f810ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.983+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 shutdown_connections 2023-08-11T14:11:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:15.983+0000 7f43fefff700 1 -- 172.21.15.3:0/646674101 wait complete. 2023-08-11T14:11:16.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:15 smithi003 ceph-mon[97195]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.26k op/s 2023-08-11T14:11:16.301 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:15 smithi187 ceph-mon[130973]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.26k op/s 2023-08-11T14:11:16.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:15 smithi103 ceph-mon[130434]: pgmap v567: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.26k op/s 2023-08-11T14:11:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 260.0 s, 648.9 tps, lat 154.838 ms stddev 282.178 2023-08-11T14:11:18.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:18 smithi103 ceph-mon[130434]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-08-11T14:11:18.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:18 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:18.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:18 smithi003 ceph-mon[97195]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-08-11T14:11:18.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:18 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:18.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:18 smithi187 ceph-mon[130973]: pgmap v568: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 18 MiB/s wr, 1.50k op/s 2023-08-11T14:11:18.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:18 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:19.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:19 smithi103 ceph-mon[130434]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-08-11T14:11:19.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:19 smithi003 ceph-mon[97195]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-08-11T14:11:19.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:19 smithi187 ceph-mon[130973]: pgmap v569: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-08-11T14:11:22.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:21 smithi003 ceph-mon[97195]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:11:22.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:21 smithi187 ceph-mon[130973]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:11:22.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:21 smithi103 ceph-mon[130434]: pgmap v570: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:11:23.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:22 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:23.057 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:22 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 265.0 s, 1061.4 tps, lat 95.006 ms stddev 166.778 2023-08-11T14:11:23.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:22 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:24.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:23 smithi103 ceph-mon[130434]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:11:24.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:23 smithi003 ceph-mon[97195]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:11:24.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:23 smithi187 ceph-mon[130973]: pgmap v571: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:11:26.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:25 smithi187 ceph-mon[130973]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:11:26.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:25 smithi003 ceph-mon[97195]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:11:26.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:25 smithi103 ceph-mon[130434]: pgmap v572: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:11:28.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 270.0 s, 618.4 tps, lat 152.406 ms stddev 348.902 2023-08-11T14:11:28.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:28 smithi187 ceph-mon[130973]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:11:28.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:28 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:28.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:28 smithi003 ceph-mon[97195]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:11:28.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:28 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:28 smithi103 ceph-mon[130434]: pgmap v573: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:11:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:28 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:11:30.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:30 smithi103 ceph-mon[130434]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.45k op/s 2023-08-11T14:11:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.444+0000 7fa7d1fff700 1 -- 172.21.15.3:0/1955197214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc10a290 msgr2=0x7fa7cc10c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.444+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/1955197214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc10a290 0x7fa7cc10c720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40098f0 tx=0x7fa7c4009c00 comp rx=0 tx=0).stop 2023-08-11T14:11:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.447+0000 7fa7d1fff700 1 -- 172.21.15.3:0/1955197214 shutdown_connections 2023-08-11T14:11:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.447+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/1955197214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.447+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/1955197214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc107920 0x7fa7cc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.447+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/1955197214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc10a290 0x7fa7cc10c720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.447+0000 7fa7d1fff700 1 -- 172.21.15.3:0/1955197214 >> 172.21.15.3:0/1955197214 conn(0x7fa7cc06d750 msgr2=0x7fa7cc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.458+0000 7fa7d1fff700 1 -- 172.21.15.3:0/1955197214 shutdown_connections 2023-08-11T14:11:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.459+0000 7fa7d1fff700 1 -- 172.21.15.3:0/1955197214 wait complete. 2023-08-11T14:11:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.460+0000 7fa7d1fff700 1 Processor -- start 2023-08-11T14:11:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.460+0000 7fa7d1fff700 1 -- start start 2023-08-11T14:11:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc107920 0x7fa7cc1183c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7cc10a290 0x7fa7cc118920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc119280 con 0x7fa7cc107920 2023-08-11T14:11:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc117270 con 0x7fa7cc10cc60 2023-08-11T14:11:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7d1fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc117430 con 0x7fa7cc10a290 2023-08-11T14:11:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46404/0 (socket says 172.21.15.3:46404) 2023-08-11T14:11:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.461+0000 7fa7cbfff700 1 -- 172.21.15.3:0/2366841392 learned_addr learned my addr 172.21.15.3:0/2366841392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.465+0000 7fa7cbfff700 1 -- 172.21.15.3:0/2366841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7cc10a290 msgr2=0x7fa7cc118920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:11:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.465+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7cc10a290 0x7fa7cc118920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.465+0000 7fa7cbfff700 1 -- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc107920 msgr2=0x7fa7cc1183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.465+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc107920 0x7fa7cc1183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.465+0000 7fa7cbfff700 1 -- 172.21.15.3:0/2366841392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4009520 con 0x7fa7cc10cc60 2023-08-11T14:11:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.471+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0007fd0 tx=0x7fa7c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.471+0000 7fa7c8ff9700 1 -- 172.21.15.3:0/2366841392 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017070 con 0x7fa7cc10cc60 2023-08-11T14:11:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.472+0000 7fa7c8ff9700 1 -- 172.21.15.3:0/2366841392 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7c000f040 con 0x7fa7cc10cc60 2023-08-11T14:11:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.472+0000 7fa7c8ff9700 1 -- 172.21.15.3:0/2366841392 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00057d0 con 0x7fa7cc10cc60 2023-08-11T14:11:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.472+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc117710 con 0x7fa7cc10cc60 2023-08-11T14:11:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.472+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc078b80 con 0x7fa7cc10cc60 2023-08-11T14:11:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.479+0000 7fa7e58fcb80 1 Processor -- start 2023-08-11T14:11:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 -- start start 2023-08-11T14:11:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577ade87360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577adde05d0 con 0x5577ade86f40 2023-08-11T14:11:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577addddef0 con 0x5577ade878a0 2023-08-11T14:11:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.480+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577addd97e0 con 0x5577ade88790 2023-08-11T14:11:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.483+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.483+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577ade87360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.483+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577ade87360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44190/0 (socket says 172.21.15.3:44190) 2023-08-11T14:11:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.483+0000 7fa7caffd700 1 -- 172.21.15.3:0/3836642530 learned_addr learned my addr 172.21.15.3:0/3836642530 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.484+0000 7fa7c8ff9700 1 -- 172.21.15.3:0/2366841392 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa7c0005930 con 0x7fa7cc10cc60 2023-08-11T14:11:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 -- 172.21.15.3:0/3836642530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 msgr2=0x5577ade881b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:11:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade881b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 -- 172.21.15.3:0/3836642530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 msgr2=0x5577ade87360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577ade87360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 -- 172.21.15.3:0/3836642530 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577adc595d0 con 0x5577ade88790 2023-08-11T14:11:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.485+0000 7fa7caffd700 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade88c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4016000 tx=0x7fa7c4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54edfd12554cfc90 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.486+0000 7fa7b1ffb700 1 -- 172.21.15.3:0/3836642530 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400f960 con 0x5577ade88790 2023-08-11T14:11:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.486+0000 7fa7b1ffb700 1 -- 172.21.15.3:0/3836642530 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7c40054e0 con 0x5577ade88790 2023-08-11T14:11:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.486+0000 7fa7b1ffb700 1 -- 172.21.15.3:0/3836642530 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004910 con 0x5577ade88790 2023-08-11T14:11:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.487+0000 7fa7c8ff9700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 0x7fa7b40a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.487+0000 7fa7c8ff9700 1 -- 172.21.15.3:0/2366841392 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa7c00cbc50 con 0x7fa7cc10cc60 2023-08-11T14:11:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.487+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/3836642530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 msgr2=0x5577ade88c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.487+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade88c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4016000 tx=0x7fa7c4009c00 comp rx=0 tx=0).stop 2023-08-11T14:11:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.488+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 0x7fa7b40a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.490+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 0x7fa7b40a2610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc119620 tx=0x7fa7bc006cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.497+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/3836642530 shutdown_connections 2023-08-11T14:11:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.497+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade88c10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.497+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577ade87360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.497+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/3836642530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.497+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/3836642530 >> 172.21.15.3:0/3836642530 conn(0x5577addb48f0 msgr2=0x5577ade83920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.504+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/3836642530 shutdown_connections 2023-08-11T14:11:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.511+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/3836642530 wait complete. 2023-08-11T14:11:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.512+0000 7fa7e58fcb80 1 -- start start 2023-08-11T14:11:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 Processor -- start 2023-08-11T14:11:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade88790 0x5577ade98190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577add77a60 con 0x5577ade88790 2023-08-11T14:11:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577add77760 con 0x5577ade86f40 2023-08-11T14:11:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.513+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577adc5b8c0 con 0x5577ade878a0 2023-08-11T14:11:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.514+0000 7fa7e58fcb80 10 client.0 _ll_register_callbacks cb 0x5577add7aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.514+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.514+0000 7fa7caffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40366/0 (socket says 172.21.15.3:40366) 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.514+0000 7fa7caffd700 1 -- 172.21.15.3:0/1489790164 learned_addr learned my addr 172.21.15.3:0/1489790164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.515+0000 7fa7caffd700 1 -- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 msgr2=0x5577ade90c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.515+0000 7fa7caffd700 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade90c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.515+0000 7fa7caffd700 1 -- 172.21.15.3:0/1489790164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade88790 msgr2=0x5577ade98190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.515+0000 7fa7caffd700 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade88790 0x5577ade98190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.515+0000 7fa7caffd700 1 -- 172.21.15.3:0/1489790164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc10dc60 con 0x5577ade878a0 2023-08-11T14:11:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.516+0000 7fa7caffd700 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c4005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4284c8b02d155db4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.516+0000 7fa7af7f6700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.516+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4005230 con 0x5577ade878a0 2023-08-11T14:11:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.516+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa7c4003680 con 0x5577ade878a0 2023-08-11T14:11:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.517+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577ade7a7e0 con 0x5577ade878a0 2023-08-11T14:11:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.517+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c4003840 con 0x5577ade878a0 2023-08-11T14:11:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.517+0000 7fa7e58fcb80 10 client.26320 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.517+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5577adc05d80 con 0x5577ade878a0 2023-08-11T14:11:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577ade7a7e0 con 0x5577ade878a0 2023-08-11T14:11:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa7c4015020 con 0x5577ade878a0 2023-08-11T14:11:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 10 client.26320 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 20 client.26320 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577ade9c660 0x5577ade9eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 4 client.26320 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:11:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.518+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577ade56480 con 0x5577ade9c660 2023-08-11T14:11:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.522+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577ade9c660 0x5577ade9eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.532+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577ade9c660 0x5577ade9eae0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695d86e3d1111c95 server_cookie=2ebc8d887eee7632 in_seq=0 out_seq=0 2023-08-11T14:11:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.540+0000 7fa7af7f6700 10 client.26320 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.540+0000 7fa7af7f6700 1 -- 172.21.15.3:0/1489790164 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5577ade56480 con 0x5577ade9c660 2023-08-11T14:11:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.540+0000 7fa7af7f6700 10 client.26320 handle_command_reply: tid=0 2023-08-11T14:11:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:30 smithi187 ceph-mon[130973]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.45k op/s 2023-08-11T14:11:30.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:30 smithi003 ceph-mon[97195]: pgmap v574: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 20 MiB/s wr, 1.45k op/s 2023-08-11T14:11:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.541+0000 7fa7e58fcb80 1 client.26320 shutdown 2023-08-11T14:11:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.542+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 msgr2=0x5577ade91150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.542+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c4005bb0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 shutdown_connections 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade88790 0x5577ade98190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577ade9c660 0x5577ade9eae0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/1489790164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade878a0 0x5577ade91150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 >> 172.21.15.3:0/1489790164 conn(0x5577addb48f0 msgr2=0x5577ade82710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7af7f6700 0 client.26320 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.545+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 shutdown_connections 2023-08-11T14:11:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.552+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/1489790164 wait complete. 2023-08-11T14:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.561+0000 7fa7e58fcb80 20 client.26320 trim_cache size 0 max 16384 2023-08-11T14:11:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.595+0000 7fa7e58fcb80 1 Processor -- start 2023-08-11T14:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 -- start start 2023-08-11T14:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577adea3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade8b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade83ed0 con 0x5577ade86f40 2023-08-11T14:11:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade7cea0 con 0x5577ade878a0 2023-08-11T14:11:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.596+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade90160 con 0x5577ade88790 2023-08-11T14:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.597+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade8b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.597+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade8b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40372/0 (socket says 172.21.15.3:40372) 2023-08-11T14:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.597+0000 7fa7cbfff700 1 -- 172.21.15.3:0/678944512 learned_addr learned my addr 172.21.15.3:0/678944512 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.602+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577adea3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.603+0000 7fa7cb7fe700 1 -- 172.21.15.3:0/678944512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 msgr2=0x5577ade8b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.603+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade8b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.603+0000 7fa7cb7fe700 1 -- 172.21.15.3:0/678944512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 msgr2=0x5577ade98150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.603+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.603+0000 7fa7cb7fe700 1 -- 172.21.15.3:0/678944512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577adbeec00 con 0x5577ade86f40 2023-08-11T14:11:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.611+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577adea3440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5577ade9f3c0 tx=0x7fa7bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcf4e617f09e10b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.622+0000 7fa7b27fc700 1 -- 172.21.15.3:0/678944512 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc020050 con 0x5577ade86f40 2023-08-11T14:11:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.622+0000 7fa7b27fc700 1 -- 172.21.15.3:0/678944512 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7bc019040 con 0x5577ade86f40 2023-08-11T14:11:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.622+0000 7fa7b27fc700 1 -- 172.21.15.3:0/678944512 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc012390 con 0x5577ade86f40 2023-08-11T14:11:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.623+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/678944512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 msgr2=0x5577adea3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.623+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577adea3440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5577ade9f3c0 tx=0x7fa7bc012000 comp rx=0 tx=0).stop 2023-08-11T14:11:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.633+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/678944512 shutdown_connections 2023-08-11T14:11:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.633+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade8b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.633+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade86f40 0x5577adea3440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.633+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/678944512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade878a0 0x5577ade98150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.633+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/678944512 >> 172.21.15.3:0/678944512 conn(0x5577addb48f0 msgr2=0x5577ade83320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.643+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/678944512 shutdown_connections 2023-08-11T14:11:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.643+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/678944512 wait complete. 2023-08-11T14:11:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.643+0000 7fa7e58fcb80 1 -- start start 2023-08-11T14:11:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.644+0000 7fa7e58fcb80 1 Processor -- start 2023-08-11T14:11:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade878a0 0x5577ade97f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade965a0 con 0x5577ade878a0 2023-08-11T14:11:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade96710 con 0x5577ade86f40 2023-08-11T14:11:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577ade968d0 con 0x5577ade88790 2023-08-11T14:11:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.645+0000 7fa7e58fcb80 10 client.0 _ll_register_callbacks cb 0x7fa7c4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40380/0 (socket says 172.21.15.3:40380) 2023-08-11T14:11:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 -- 172.21.15.3:0/189878984 learned_addr learned my addr 172.21.15.3:0/189878984 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 -- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 msgr2=0x5577ade97a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade97a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 -- 172.21.15.3:0/189878984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade878a0 msgr2=0x5577ade97f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade878a0 0x5577ade97f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.646+0000 7fa7cbfff700 1 -- 172.21.15.3:0/189878984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc020930 con 0x5577ade88790 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.647+0000 7fa7cbfff700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a0c00 tx=0x7fa7c0098d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbef2a7e75e47627 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.647+0000 7fa7af7f6700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.647+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0099320 con 0x5577ade88790 2023-08-11T14:11:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.647+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa7c009d150 con 0x5577ade88790 2023-08-11T14:11:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.647+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7c40054e0 con 0x5577ade88790 2023-08-11T14:11:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.658+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c00a1130 con 0x5577ade88790 2023-08-11T14:11:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.664+0000 7fa7e58fcb80 10 client.26326 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.664+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5577adc2c1f0 con 0x5577ade88790 2023-08-11T14:11:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7c40054e0 con 0x5577ade88790 2023-08-11T14:11:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa7c00a0ca0 con 0x5577ade88790 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 10 client.26326 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 20 client.26326 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577adef1660 0x5577adef3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 4 client.26326 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:11:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.669+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577addd27c0 con 0x5577adef1660 2023-08-11T14:11:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.677+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577adef1660 0x5577adef3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.685+0000 7fa7cb7fe700 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577adef1660 0x5577adef3a80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcd0fde8c6820df server_cookie=ed87565c6a541205 in_seq=0 out_seq=0 2023-08-11T14:11:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.687+0000 7fa7af7f6700 10 client.26326 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.751+0000 7fa7af7f6700 1 -- 172.21.15.3:0/189878984 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577addd27c0 con 0x5577adef1660 2023-08-11T14:11:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.751+0000 7fa7af7f6700 10 client.26326 handle_command_reply: tid=0 2023-08-11T14:11:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.751+0000 7fa7e58fcb80 1 client.26326 shutdown 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 msgr2=0x5577ade96020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00a0c00 tx=0x7fa7c0098d80 comp rx=0 tx=0).stop 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 shutdown_connections 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5577ade88790 0x5577ade96020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5577adef1660 0x5577adef3a80 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577ade86f40 0x5577ade97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 --2- 172.21.15.3:0/189878984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5577ade878a0 0x5577ade97f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 >> 172.21.15.3:0/189878984 conn(0x5577addb48f0 msgr2=0x5577ade8b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.757+0000 7fa7af7f6700 0 client.26326 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.758+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 shutdown_connections 2023-08-11T14:11:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.758+0000 7fa7e58fcb80 1 -- 172.21.15.3:0/189878984 wait complete. 2023-08-11T14:11:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.758+0000 7fa7e58fcb80 20 client.26326 trim_cache size 0 max 16384 2023-08-11T14:11:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:11:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:11:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:11:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:11:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.768+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 msgr2=0x7fa7b40a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.768+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 0x7fa7b40a2610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc119620 tx=0x7fa7bc006cb0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.768+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 msgr2=0x7fa7cc116d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.768+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0007fd0 tx=0x7fa7c000d3b0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 shutdown_connections 2023-08-11T14:11:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa7b40a0150 0x7fa7b40a2610 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7cc10cc60 0x7fa7cc116d00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7cc107920 0x7fa7cc1183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 --2- 172.21.15.3:0/2366841392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7cc10a290 0x7fa7cc118920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.769+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 >> 172.21.15.3:0/2366841392 conn(0x7fa7cc06d750 msgr2=0x7fa7cc10ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.770+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 shutdown_connections 2023-08-11T14:11:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:30.772+0000 7fa7d1fff700 1 -- 172.21.15.3:0/2366841392 wait complete. 2023-08-11T14:11:31.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:31 smithi103 ceph-mon[130434]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-08-11T14:11:31.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:31 smithi003 ceph-mon[97195]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-08-11T14:11:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:31 smithi187 ceph-mon[130973]: pgmap v575: 129 pgs: 129 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 1.37k op/s 2023-08-11T14:11:33.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 275.0 s, 685.3 tps, lat 152.912 ms stddev 310.698 2023-08-11T14:11:34.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:33 smithi103 ceph-mon[130434]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.62k op/s 2023-08-11T14:11:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:33 smithi003 ceph-mon[97195]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.62k op/s 2023-08-11T14:11:34.303 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:33 smithi187 ceph-mon[130973]: pgmap v576: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 21 MiB/s wr, 1.62k op/s 2023-08-11T14:11:36.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:35 smithi003 ceph-mon[97195]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.40k op/s 2023-08-11T14:11:36.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:35 smithi187 ceph-mon[130973]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.40k op/s 2023-08-11T14:11:36.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:35 smithi103 ceph-mon[130434]: pgmap v577: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 18 MiB/s wr, 1.40k op/s 2023-08-11T14:11:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 280.0 s, 793.1 tps, lat 125.097 ms stddev 324.397 2023-08-11T14:11:38.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:37 smithi187 ceph-mon[130973]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:38.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:37 smithi003 ceph-mon[97195]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:38.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:37 smithi103 ceph-mon[130434]: pgmap v578: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 21 MiB/s wr, 1.63k op/s 2023-08-11T14:11:40.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:40 smithi003 ceph-mon[97195]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-08-11T14:11:40.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:40 smithi187 ceph-mon[130973]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-08-11T14:11:40.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:40 smithi103 ceph-mon[130434]: pgmap v579: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 18 MiB/s wr, 1.44k op/s 2023-08-11T14:11:42.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:42 smithi187 ceph-mon[130973]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:11:42.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:42 smithi103 ceph-mon[130434]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:11:42.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:42 smithi003 ceph-mon[97195]: pgmap v580: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:11:43.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 285.0 s, 799.0 tps, lat 126.890 ms stddev 289.703 2023-08-11T14:11:44.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:44 smithi103 ceph-mon[130434]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-08-11T14:11:44.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:44 smithi187 ceph-mon[130973]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-08-11T14:11:44.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:44 smithi003 ceph-mon[97195]: pgmap v581: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-08-11T14:11:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 -- 172.21.15.3:0/1654172107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff380107920 msgr2=0x7ff380109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 --2- 172.21.15.3:0/1654172107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff380107920 0x7ff380109d50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff374009990 tx=0x7ff374009ca0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 -- 172.21.15.3:0/1654172107 shutdown_connections 2023-08-11T14:11:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 --2- 172.21.15.3:0/1654172107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38010cc60 0x7ff38010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 --2- 172.21.15.3:0/1654172107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff380107920 0x7ff380109d50 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff374009990 tx=0x7ff374009ca0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 --2- 172.21.15.3:0/1654172107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff38010a290 0x7ff38010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.786+0000 7ff386bff700 1 -- 172.21.15.3:0/1654172107 >> 172.21.15.3:0/1654172107 conn(0x7ff38006d750 msgr2=0x7ff38006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.807+0000 7ff386bff700 1 -- 172.21.15.3:0/1654172107 shutdown_connections 2023-08-11T14:11:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.815+0000 7ff386bff700 1 -- 172.21.15.3:0/1654172107 wait complete. 2023-08-11T14:11:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.816+0000 7ff386bff700 1 Processor -- start 2023-08-11T14:11:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.819+0000 7ff386bff700 1 -- start start 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38010a290 0x7ff380114170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff38010cc60 0x7ff3801146b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff380116030 con 0x7ff38010cc60 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3801161a0 con 0x7ff38010a290 2023-08-11T14:11:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff386bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff380116360 con 0x7ff380115580 2023-08-11T14:11:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff38519c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff38519c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38944/0 (socket says 172.21.15.3:38944) 2023-08-11T14:11:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.820+0000 7ff38519c700 1 -- 172.21.15.3:0/686590814 learned_addr learned my addr 172.21.15.3:0/686590814 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.821+0000 7ff38519c700 1 -- 172.21.15.3:0/686590814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38010a290 msgr2=0x7ff380114170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.821+0000 7ff38519c700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38010a290 0x7ff380114170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.821+0000 7ff38519c700 1 -- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff38010cc60 msgr2=0x7ff3801146b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.821+0000 7ff38519c700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff38010cc60 0x7ff3801146b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.821+0000 7ff38519c700 1 -- 172.21.15.3:0/686590814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740095f0 con 0x7ff380115580 2023-08-11T14:11:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.822+0000 7ff38519c700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff37800ada0 tx=0x7ff37800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.822+0000 7ff37dffb700 1 -- 172.21.15.3:0/686590814 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37800f070 con 0x7ff380115580 2023-08-11T14:11:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.822+0000 7ff37dffb700 1 -- 172.21.15.3:0/686590814 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff378007e60 con 0x7ff380115580 2023-08-11T14:11:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.822+0000 7ff37dffb700 1 -- 172.21.15.3:0/686590814 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780054e0 con 0x7ff380115580 2023-08-11T14:11:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.822+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff380116820 con 0x7ff380115580 2023-08-11T14:11:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.824+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff380116ce0 con 0x7ff380115580 2023-08-11T14:11:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.831+0000 7ff39a5d1b80 1 Processor -- start 2023-08-11T14:11:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.832+0000 7ff39a5d1b80 1 -- start start 2023-08-11T14:11:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 0x565496c4d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 0x565496c4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c4ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496ba65d0 con 0x565496c4e790 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496ba3ef0 con 0x565496c4d8a0 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496b9f7e0 con 0x565496c4cf40 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 0x565496c4d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 0x565496c4d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38946/0 (socket says 172.21.15.3:38946) 2023-08-11T14:11:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.833+0000 7ff37ffff700 1 -- 172.21.15.3:0/68582687 learned_addr learned my addr 172.21.15.3:0/68582687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.834+0000 7ff38499b700 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 0x565496c4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.837+0000 7ff38499b700 1 -- 172.21.15.3:0/68582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 msgr2=0x565496c4d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.837+0000 7ff38499b700 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 0x565496c4d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.837+0000 7ff38499b700 1 -- 172.21.15.3:0/68582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 msgr2=0x565496c4ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.837+0000 7ff38499b700 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c4ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.838+0000 7ff38499b700 1 -- 172.21.15.3:0/68582687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565496a1f5d0 con 0x565496c4d8a0 2023-08-11T14:11:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.838+0000 7ff37dffb700 1 -- 172.21.15.3:0/686590814 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ff378005640 con 0x7ff380115580 2023-08-11T14:11:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.840+0000 7ff37dffb700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 0x7ff3680a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.841+0000 7ff37dffb700 1 -- 172.21.15.3:0/686590814 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff3780cabf0 con 0x7ff380115580 2023-08-11T14:11:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.842+0000 7ff37ffff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 0x7ff3680a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.842+0000 7ff38499b700 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 0x565496c4e1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff374008000 tx=0x7ff374005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beffaa2909fd3d9b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.843+0000 7ff37ffff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 0x7ff3680a2610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x565496c4df70 tx=0x7ff37000a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.844+0000 7ff366ffd700 1 -- 172.21.15.3:0/68582687 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37401d070 con 0x565496c4d8a0 2023-08-11T14:11:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.845+0000 7ff366ffd700 1 -- 172.21.15.3:0/68582687 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37400be00 con 0x565496c4d8a0 2023-08-11T14:11:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.845+0000 7ff366ffd700 1 -- 172.21.15.3:0/68582687 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374005320 con 0x565496c4d8a0 2023-08-11T14:11:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.846+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/68582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 msgr2=0x565496c4e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.846+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 0x565496c4e1b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff374008000 tx=0x7ff374005da0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.858+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/68582687 shutdown_connections 2023-08-11T14:11:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.858+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c4ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.858+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4cf40 0x565496c4d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.858+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/68582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4d8a0 0x565496c4e1b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.858+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/68582687 >> 172.21.15.3:0/68582687 conn(0x565496b7a8f0 msgr2=0x565496c49920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.880+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/68582687 shutdown_connections 2023-08-11T14:11:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.881+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/68582687 wait complete. 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.881+0000 7ff39a5d1b80 1 -- start start 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 Processor -- start 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496b3da60 con 0x565496c4e790 2023-08-11T14:11:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496b3d760 con 0x565496c4cf40 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.882+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496a218c0 con 0x565496c4d8a0 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.883+0000 7ff39a5d1b80 10 client.0 _ll_register_callbacks cb 0x565496b40a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.883+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.883+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38954/0 (socket says 172.21.15.3:38954) 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.883+0000 7ff37ffff700 1 -- 172.21.15.3:0/3188512701 learned_addr learned my addr 172.21.15.3:0/3188512701 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.887+0000 7ff37ffff700 1 -- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 msgr2=0x565496c56b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.887+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c56b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.887+0000 7ff37ffff700 1 -- 172.21.15.3:0/3188512701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 msgr2=0x565496c5e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.887+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.887+0000 7ff37ffff700 1 -- 172.21.15.3:0/3188512701 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38010dc60 con 0x565496c4d8a0 2023-08-11T14:11:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.898+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff370010dd0 tx=0x7ff37000a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbcf7b61161fae96 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.898+0000 7ff35ffff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.899+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370004230 con 0x565496c4d8a0 2023-08-11T14:11:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.899+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff370017070 con 0x565496c4d8a0 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.899+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565496c407e0 con 0x565496c4d8a0 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.899+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff370010eb0 con 0x565496c4d8a0 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.902+0000 7ff39a5d1b80 10 client.26338 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.902+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5654969cbd80 con 0x565496c4d8a0 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565496c407e0 con 0x565496c4d8a0 2023-08-11T14:11:44.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff37001e620 con 0x565496c4d8a0 2023-08-11T14:11:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 10 client.26338 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 20 client.26338 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496c84a80 0x565496c86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 4 client.26338 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:11:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.904+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565496c1c480 con 0x565496c84a80 2023-08-11T14:11:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.910+0000 7ff38499b700 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496c84a80 0x565496c86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.937+0000 7ff38499b700 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496c84a80 0x565496c86ee0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e6259dc5ca85fd server_cookie=2cc813ff54123797 in_seq=0 out_seq=0 2023-08-11T14:11:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.938+0000 7ff35ffff700 10 client.26338 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.949+0000 7ff35ffff700 1 -- 172.21.15.3:0/3188512701 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x565496c1c480 con 0x565496c84a80 2023-08-11T14:11:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.949+0000 7ff35ffff700 10 client.26338 handle_command_reply: tid=0 2023-08-11T14:11:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.949+0000 7ff39a5d1b80 1 client.26338 shutdown 2023-08-11T14:11:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.958+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 msgr2=0x565496c570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.958+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff370010dd0 tx=0x7ff37000a510 comp rx=0 tx=0).stop 2023-08-11T14:11:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 shutdown_connections 2023-08-11T14:11:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496c84a80 0x565496c86ee0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3188512701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c570c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 >> 172.21.15.3:0/3188512701 conn(0x565496b7a8f0 msgr2=0x565496c48710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.972+0000 7ff35ffff700 0 client.26338 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:44.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.987+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 shutdown_connections 2023-08-11T14:11:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:44.993+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3188512701 wait complete. 2023-08-11T14:11:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.004+0000 7ff39a5d1b80 20 client.26338 trim_cache size 0 max 16384 2023-08-11T14:11:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.041+0000 7ff39a5d1b80 1 Processor -- start 2023-08-11T14:11:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.041+0000 7ff39a5d1b80 1 -- start start 2023-08-11T14:11:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.041+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c49320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.042+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.042+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.042+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c42ea0 con 0x565496c4e790 2023-08-11T14:11:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.042+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c4e610 con 0x565496c4cf40 2023-08-11T14:11:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.042+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c4e010 con 0x565496c4d8a0 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38960/0 (socket says 172.21.15.3:38960) 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 -- 172.21.15.3:0/3211013508 learned_addr learned my addr 172.21.15.3:0/3211013508 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff38499b700 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c49320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 -- 172.21.15.3:0/3211013508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 msgr2=0x565496c49320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c49320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 -- 172.21.15.3:0/3211013508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 msgr2=0x565496c5e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.043+0000 7ff37ffff700 1 -- 172.21.15.3:0/3211013508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654969b4c00 con 0x565496c4d8a0 2023-08-11T14:11:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.044+0000 7ff37ffff700 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff370005fa0 tx=0x7ff37000ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3cd1d33ef3ed2a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.047+0000 7ff3677fe700 1 -- 172.21.15.3:0/3211013508 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37001eb70 con 0x565496c4d8a0 2023-08-11T14:11:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.047+0000 7ff3677fe700 1 -- 172.21.15.3:0/3211013508 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff370015070 con 0x565496c4d8a0 2023-08-11T14:11:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.048+0000 7ff3677fe700 1 -- 172.21.15.3:0/3211013508 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3700038e0 con 0x565496c4d8a0 2023-08-11T14:11:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.049+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3211013508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 msgr2=0x565496c49860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.049+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff370005fa0 tx=0x7ff37000ac60 comp rx=0 tx=0).stop 2023-08-11T14:11:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.056+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3211013508 shutdown_connections 2023-08-11T14:11:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.056+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4e790 0x565496c5e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.056+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c49320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.056+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/3211013508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4d8a0 0x565496c49860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.056+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3211013508 >> 172.21.15.3:0/3211013508 conn(0x565496b7a8f0 msgr2=0x565496c54ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.074+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3211013508 shutdown_connections 2023-08-11T14:11:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.075+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/3211013508 wait complete. 2023-08-11T14:11:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.075+0000 7ff39a5d1b80 1 -- start start 2023-08-11T14:11:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 Processor -- start 2023-08-11T14:11:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4d8a0 0x565496c5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4e790 0x565496c5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c5c570 con 0x565496c4d8a0 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c5c6e0 con 0x565496c4cf40 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x565496c5c8a0 con 0x565496c4e790 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.076+0000 7ff39a5d1b80 10 client.0 _ll_register_callbacks cb 0x565496b40a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44966/0 (socket says 172.21.15.3:44966) 2023-08-11T14:11:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 -- 172.21.15.3:0/4189448704 learned_addr learned my addr 172.21.15.3:0/4189448704 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38519c700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4e790 0x565496c5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 -- 172.21.15.3:0/4189448704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4e790 msgr2=0x565496c5bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4e790 0x565496c5bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 -- 172.21.15.3:0/4189448704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4d8a0 msgr2=0x565496c5df20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4d8a0 0x565496c5df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.077+0000 7ff38499b700 1 -- 172.21.15.3:0/4189448704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654969cbd80 con 0x565496c4cf40 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.078+0000 7ff38499b700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff37400b5c0 tx=0x7ff374000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1170aa29b1a7d3e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.078+0000 7ff35ffff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.078+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374027070 con 0x565496c4cf40 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.078+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ff3740271d0 con 0x565496c4cf40 2023-08-11T14:11:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.079+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff37400be00 con 0x565496c4cf40 2023-08-11T14:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.090+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff37401d040 con 0x565496c4cf40 2023-08-11T14:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.090+0000 7ff39a5d1b80 10 client.26336 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.090+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5654969f21f0 con 0x565496c4cf40 2023-08-11T14:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37400be00 con 0x565496c4cf40 2023-08-11T14:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7ff3740276e0 con 0x565496c4cf40 2023-08-11T14:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 20 client.26336 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496cd9550 0x565496cdb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 4 client.26336 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.100+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565496b987c0 con 0x565496cd9550 2023-08-11T14:11:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.101+0000 7ff37ffff700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496cd9550 0x565496cdb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.109+0000 7ff37ffff700 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496cd9550 0x565496cdb970 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23633c6e029c6965 server_cookie=13b1bf21a632cde in_seq=0 out_seq=0 2023-08-11T14:11:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.109+0000 7ff35ffff700 10 client.26336 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:45.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:45 smithi003 ceph-mon[97195]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-08-11T14:11:45.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:45 smithi187 ceph-mon[130973]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-08-11T14:11:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:45 smithi103 ceph-mon[130434]: pgmap v582: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 18 MiB/s wr, 1.25k op/s 2023-08-11T14:11:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.834+0000 7ff35ffff700 1 -- 172.21.15.3:0/4189448704 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565496b987c0 con 0x565496cd9550 2023-08-11T14:11:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.834+0000 7ff35ffff700 10 client.26336 handle_command_reply: tid=0 2023-08-11T14:11:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.834+0000 7ff39a5d1b80 1 client.26336 shutdown 2023-08-11T14:11:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.834+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 msgr2=0x565496c5d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.834+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff37400b5c0 tx=0x7ff374000bc0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 shutdown_connections 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565496c4e790 0x565496c5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x565496cd9550 0x565496cdb970 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496c4cf40 0x565496c5d9e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 --2- 172.21.15.3:0/4189448704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565496c4d8a0 0x565496c5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 >> 172.21.15.3:0/4189448704 conn(0x565496b7a8f0 msgr2=0x565496c49320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff35ffff700 0 client.26336 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 shutdown_connections 2023-08-11T14:11:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 1 -- 172.21.15.3:0/4189448704 wait complete. 2023-08-11T14:11:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.835+0000 7ff39a5d1b80 20 client.26336 trim_cache size 0 max 16384 2023-08-11T14:11:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:11:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:11:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:11:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:11:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 msgr2=0x7ff3680a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 0x7ff3680a2610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x565496c4df70 tx=0x7ff37000a950 comp rx=0 tx=0).stop 2023-08-11T14:11:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 msgr2=0x7ff380115a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff37800ada0 tx=0x7ff37800c330 comp rx=0 tx=0).stop 2023-08-11T14:11:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 shutdown_connections 2023-08-11T14:11:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff380115580 0x7ff380115a30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ff3680a0150 0x7ff3680a2610 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38010a290 0x7ff380114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 --2- 172.21.15.3:0/686590814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff38010cc60 0x7ff3801146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.850+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 >> 172.21.15.3:0/686590814 conn(0x7ff38006d750 msgr2=0x7ff38010bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.868+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 shutdown_connections 2023-08-11T14:11:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:45.872+0000 7ff386bff700 1 -- 172.21.15.3:0/686590814 wait complete. 2023-08-11T14:11:48.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:47 smithi187 ceph-mon[130973]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-08-11T14:11:48.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:47 smithi003 ceph-mon[97195]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-08-11T14:11:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 290.0 s, 818.0 tps, lat 123.639 ms stddev 271.840 2023-08-11T14:11:48.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:47 smithi103 ceph-mon[130434]: pgmap v583: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 1.54k op/s 2023-08-11T14:11:50.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:49 smithi003 ceph-mon[97195]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-08-11T14:11:50.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:49 smithi187 ceph-mon[130973]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-08-11T14:11:50.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:49 smithi103 ceph-mon[130434]: pgmap v584: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 20 MiB/s wr, 1.56k op/s 2023-08-11T14:11:52.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:52 smithi003 ceph-mon[97195]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:11:52.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:52 smithi187 ceph-mon[130973]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:11:52.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:52 smithi103 ceph-mon[130434]: pgmap v585: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:11:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 295.0 s, 592.8 tps, lat 150.126 ms stddev 399.475 2023-08-11T14:11:54.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:54 smithi103 ceph-mon[130434]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.77k op/s 2023-08-11T14:11:54.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:54 smithi003 ceph-mon[97195]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.77k op/s 2023-08-11T14:11:54.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:54 smithi187 ceph-mon[130973]: pgmap v586: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 23 MiB/s wr, 1.77k op/s 2023-08-11T14:11:55.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:55 smithi003 ceph-mon[97195]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:55.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:55 smithi187 ceph-mon[130973]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:55.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:55 smithi103 ceph-mon[130434]: pgmap v587: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:11:58.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:11:57 smithi003 ceph-mon[97195]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.67k op/s 2023-08-11T14:11:58.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:11:57 smithi187 ceph-mon[130973]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.67k op/s 2023-08-11T14:11:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 300.0 s, 331.5 tps, lat 332.290 ms stddev 471.146 2023-08-11T14:11:58.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:11:57 smithi103 ceph-mon[130434]: pgmap v588: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 21 MiB/s wr, 1.67k op/s 2023-08-11T14:11:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.684+0000 7f6c57fff700 1 -- 172.21.15.3:0/1831063563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c50107920 msgr2=0x7f6c50109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.684+0000 7f6c57fff700 1 --2- 172.21.15.3:0/1831063563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c50107920 0x7f6c50109d50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c400098f0 tx=0x7f6c40009c00 comp rx=0 tx=0).stop 2023-08-11T14:11:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.699+0000 7f6c57fff700 1 -- 172.21.15.3:0/1831063563 shutdown_connections 2023-08-11T14:11:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.699+0000 7f6c57fff700 1 --2- 172.21.15.3:0/1831063563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5010cc60 0x7f6c5010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.699+0000 7f6c57fff700 1 --2- 172.21.15.3:0/1831063563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c50107920 0x7f6c50109d50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.699+0000 7f6c57fff700 1 --2- 172.21.15.3:0/1831063563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c5010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.699+0000 7f6c57fff700 1 -- 172.21.15.3:0/1831063563 >> 172.21.15.3:0/1831063563 conn(0x7f6c5006d750 msgr2=0x7f6c5006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.717+0000 7f6c57fff700 1 -- 172.21.15.3:0/1831063563 shutdown_connections 2023-08-11T14:11:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.729+0000 7f6c57fff700 1 -- 172.21.15.3:0/1831063563 wait complete. 2023-08-11T14:11:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.730+0000 7f6c57fff700 1 Processor -- start 2023-08-11T14:11:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 -- start start 2023-08-11T14:11:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c50107920 0x7f6c50113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c5010cc60 0x7f6c50115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501159c0 con 0x7f6c5010cc60 2023-08-11T14:11:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50115b30 con 0x7f6c50107920 2023-08-11T14:11:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.731+0000 7f6c57fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c50115cf0 con 0x7f6c5010a290 2023-08-11T14:11:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.732+0000 7f6c5559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.732+0000 7f6c5559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34998/0 (socket says 172.21.15.3:34998) 2023-08-11T14:11:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.732+0000 7f6c5559a700 1 -- 172.21.15.3:0/313791791 learned_addr learned my addr 172.21.15.3:0/313791791 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c50107920 0x7f6c50113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c5559a700 1 -- 172.21.15.3:0/313791791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c50107920 msgr2=0x7f6c50113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c5559a700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c50107920 0x7f6c50113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c5559a700 1 -- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c5010cc60 msgr2=0x7f6c50115360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c5559a700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c5010cc60 0x7f6c50115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.733+0000 7f6c5559a700 1 -- 172.21.15.3:0/313791791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c40009520 con 0x7f6c5010a290 2023-08-11T14:11:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.751+0000 7f6c5559a700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e9c0 tx=0x7f6c4c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.752+0000 7f6c46ffd700 1 -- 172.21.15.3:0/313791791 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00cbd0 con 0x7f6c5010a290 2023-08-11T14:11:59.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.752+0000 7f6c46ffd700 1 -- 172.21.15.3:0/313791791 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4c00cd30 con 0x7f6c5010a290 2023-08-11T14:11:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.752+0000 7f6c46ffd700 1 -- 172.21.15.3:0/313791791 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c005580 con 0x7f6c5010a290 2023-08-11T14:11:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.752+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c50115fd0 con 0x7f6c5010a290 2023-08-11T14:11:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.752+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c50116800 con 0x7f6c5010a290 2023-08-11T14:11:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.779+0000 7f6c6b915b80 1 Processor -- start 2023-08-11T14:11:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 -- start start 2023-08-11T14:11:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 0x5613a37a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a36fb5d0 con 0x5613a37a3790 2023-08-11T14:11:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a36f8ef0 con 0x5613a37a1f40 2023-08-11T14:11:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.790+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a36f47e0 con 0x5613a37a28a0 2023-08-11T14:11:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.791+0000 7f6c55d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 0x5613a37a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.791+0000 7f6c55d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 0x5613a37a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35010/0 (socket says 172.21.15.3:35010) 2023-08-11T14:11:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.791+0000 7f6c55d9b700 1 -- 172.21.15.3:0/3607782171 learned_addr learned my addr 172.21.15.3:0/3607782171 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:59.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.791+0000 7f6c5659c700 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.792+0000 7f6c5659c700 1 -- 172.21.15.3:0/3607782171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 msgr2=0x5613a37a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.792+0000 7f6c5659c700 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 0x5613a37a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.792+0000 7f6c5659c700 1 -- 172.21.15.3:0/3607782171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 msgr2=0x5613a37a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.792+0000 7f6c5659c700 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.792+0000 7f6c5659c700 1 -- 172.21.15.3:0/3607782171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613a35745d0 con 0x5613a37a1f40 2023-08-11T14:11:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.793+0000 7f6c46ffd700 1 -- 172.21.15.3:0/313791791 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6c4c004750 con 0x7f6c5010a290 2023-08-11T14:11:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.795+0000 7f6c46ffd700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 0x7f6c3c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.795+0000 7f6c46ffd700 1 -- 172.21.15.3:0/313791791 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c4c014070 con 0x7f6c5010a290 2023-08-11T14:11:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.796+0000 7f6c5659c700 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37a2360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009ce0 tx=0x7f6c4800b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94105790298a630b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.796+0000 7f6c3bfff700 1 -- 172.21.15.3:0/3607782171 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800bd20 con 0x5613a37a1f40 2023-08-11T14:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.796+0000 7f6c3bfff700 1 -- 172.21.15.3:0/3607782171 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4800be80 con 0x5613a37a1f40 2023-08-11T14:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.796+0000 7f6c3bfff700 1 -- 172.21.15.3:0/3607782171 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48012890 con 0x5613a37a1f40 2023-08-11T14:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.797+0000 7f6c6b915b80 1 -- 172.21.15.3:0/3607782171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 msgr2=0x5613a37a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.797+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37a2360 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009ce0 tx=0x7f6c4800b410 comp rx=0 tx=0).stop 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 -- 172.21.15.3:0/3607782171 shutdown_connections 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37a2360 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/3607782171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a28a0 0x5613a37a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 -- 172.21.15.3:0/3607782171 >> 172.21.15.3:0/3607782171 conn(0x5613a36cf8f0 msgr2=0x5613a379e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.798+0000 7f6c6b915b80 1 -- 172.21.15.3:0/3607782171 shutdown_connections 2023-08-11T14:11:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.799+0000 7f6c6b915b80 1 -- 172.21.15.3:0/3607782171 wait complete. 2023-08-11T14:11:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.799+0000 7f6c6b915b80 1 -- start start 2023-08-11T14:11:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 Processor -- start 2023-08-11T14:11:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a28a0 0x5613a37ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 0x5613a37b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a3692a60 con 0x5613a37a28a0 2023-08-11T14:11:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a3692760 con 0x5613a37a3790 2023-08-11T14:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a35768c0 con 0x5613a37a1f40 2023-08-11T14:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.800+0000 7f6c6b915b80 10 client.0 _ll_register_callbacks cb 0x5613a3695a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.801+0000 7f6c55d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.801+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 0x5613a37b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.801+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 0x5613a37b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50062/0 (socket says 172.21.15.3:50062) 2023-08-11T14:11:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.801+0000 7f6c5659c700 1 -- 172.21.15.3:0/942131594 learned_addr learned my addr 172.21.15.3:0/942131594 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 -- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 msgr2=0x5613a37b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 0x5613a37b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 -- 172.21.15.3:0/942131594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a28a0 msgr2=0x5613a37ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a28a0 0x5613a37ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 -- 172.21.15.3:0/942131594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48009910 con 0x5613a37a1f40 2023-08-11T14:11:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.804+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 0x7f6c3c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.805+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37abdb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000fad0 tx=0x7f6c4000f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612e33683baabffd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.805+0000 7f6c397fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.805+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4001c070 con 0x5613a37a1f40 2023-08-11T14:11:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.805+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c40005240 con 0x5613a37a1f40 2023-08-11T14:11:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.806+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613a3771480 con 0x5613a37a1f40 2023-08-11T14:11:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.809+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40005700 con 0x5613a37a1f40 2023-08-11T14:11:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.810+0000 7f6c6b915b80 10 client.26356 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.810+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5613a3520d80 con 0x5613a37a1f40 2023-08-11T14:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.811+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 0x7f6c3c0a2610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5613a37b3910 tx=0x7f6c4002c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.811+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613a3771480 con 0x5613a37a1f40 2023-08-11T14:11:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.811+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6c40039bd0 con 0x5613a37a1f40 2023-08-11T14:11:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 20 client.26356 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a37b7b10 0x5613a37b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 4 client.26356 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:11:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.813+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613a36ed7c0 con 0x5613a37b7b10 2023-08-11T14:11:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.821+0000 7f6c5559a700 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a37b7b10 0x5613a37b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.830+0000 7f6c5559a700 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a37b7b10 0x5613a37b9f30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9187fb4226e3c7e6 server_cookie=6d2937aca5691594 in_seq=0 out_seq=0 2023-08-11T14:11:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.830+0000 7f6c397fa700 10 client.26356 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.835+0000 7f6c397fa700 1 -- 172.21.15.3:0/942131594 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5613a36ed7c0 con 0x5613a37b7b10 2023-08-11T14:11:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.835+0000 7f6c397fa700 10 client.26356 handle_command_reply: tid=0 2023-08-11T14:11:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.840+0000 7f6c6b915b80 1 client.26356 shutdown 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.847+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 msgr2=0x5613a37abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.847+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37abdb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000fad0 tx=0x7f6c4000f9f0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 shutdown_connections 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a37b7b10 0x5613a37b9f30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a3790 0x5613a37b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37abdb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/942131594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a28a0 0x5613a37ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 >> 172.21.15.3:0/942131594 conn(0x5613a36cf8f0 msgr2=0x5613a379ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c397fa700 0 client.26356 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.848+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 shutdown_connections 2023-08-11T14:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.856+0000 7f6c6b915b80 1 -- 172.21.15.3:0/942131594 wait complete. 2023-08-11T14:11:59.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.866+0000 7f6c6b915b80 20 client.26356 trim_cache size 0 max 16384 2023-08-11T14:11:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.898+0000 7f6c6b915b80 1 Processor -- start 2023-08-11T14:11:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 -- start start 2023-08-11T14:11:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a28a0 0x5613a37b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a36ed7c0 con 0x5613a37a3790 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a3771480 con 0x5613a37a28a0 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.907+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a37a3040 con 0x5613a37a1f40 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40234/0 (socket says 172.21.15.3:40234) 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 -- 172.21.15.3:0/2315925857 learned_addr learned my addr 172.21.15.3:0/2315925857 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 -- 172.21.15.3:0/2315925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 msgr2=0x5613a37a2320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37a2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 -- 172.21.15.3:0/2315925857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a28a0 msgr2=0x5613a37b32c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a28a0 0x5613a37b32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.908+0000 7f6c5659c700 1 -- 172.21.15.3:0/2315925857 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613a3509c00 con 0x5613a37a3790 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.909+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000f80 tx=0x7f6c48004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f1c8a3b38ffed8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.909+0000 7f6c44ff9700 1 -- 172.21.15.3:0/2315925857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480135d0 con 0x5613a37a3790 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.909+0000 7f6c44ff9700 1 -- 172.21.15.3:0/2315925857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4801c400 con 0x5613a37a3790 2023-08-11T14:11:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.910+0000 7f6c44ff9700 1 -- 172.21.15.3:0/2315925857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48013bf0 con 0x5613a37a3790 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2315925857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 msgr2=0x5613a37aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000f80 tx=0x7f6c48004b60 comp rx=0 tx=0).stop 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2315925857 shutdown_connections 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37aa9f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37a1f40 0x5613a37a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2315925857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a28a0 0x5613a37b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.911+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2315925857 >> 172.21.15.3:0/2315925857 conn(0x5613a36cf8f0 msgr2=0x5613a37a9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:11:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.932+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2315925857 shutdown_connections 2023-08-11T14:11:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.937+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2315925857 wait complete. 2023-08-11T14:11:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.938+0000 7f6c6b915b80 1 -- start start 2023-08-11T14:11:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.938+0000 7f6c6b915b80 1 Processor -- start 2023-08-11T14:11:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a37b3130 con 0x5613a37a3790 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a37b32a0 con 0x5613a37a1f40 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a37b3460 con 0x5613a37b2770 2023-08-11T14:11:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.939+0000 7f6c6b915b80 10 client.0 _ll_register_callbacks cb 0x7f6c4000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:11:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35014/0 (socket says 172.21.15.3:35014) 2023-08-11T14:11:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 -- 172.21.15.3:0/2247631350 learned_addr learned my addr 172.21.15.3:0/2247631350 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:11:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 -- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 msgr2=0x5613a37b1220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:11:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37b1220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 -- 172.21.15.3:0/2247631350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 msgr2=0x5613a37b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:11:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.940+0000 7f6c5659c700 1 -- 172.21.15.3:0/2247631350 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613a35745d0 con 0x5613a37b2770 2023-08-11T14:11:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.941+0000 7f6c5659c700 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b7e0 tx=0x7f6c4801c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab832f8cb543613b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:11:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.941+0000 7f6c397fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:11:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.945+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480137e0 con 0x5613a37b2770 2023-08-11T14:11:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.945+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c48013940 con 0x5613a37b2770 2023-08-11T14:11:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.945+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613a37957e0 con 0x5613a37b2770 2023-08-11T14:11:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.946+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c48013b00 con 0x5613a37b2770 2023-08-11T14:11:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.950+0000 7f6c6b915b80 10 client.26359 fetch_fsmap learned FSMap version 36 2023-08-11T14:11:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.950+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5613a35471f0 con 0x5613a37b2770 2023-08-11T14:11:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613a37957e0 con 0x5613a37b2770 2023-08-11T14:11:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6c48008030 con 0x5613a37b2770 2023-08-11T14:11:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c6b915b80 10 client.26359 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:11:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c6b915b80 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:11:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c6b915b80 20 client.26359 populate_metadata read hostname 'smithi003' 2023-08-11T14:11:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.963+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a380c890 0x5613a380ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:11:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.964+0000 7f6c6b915b80 4 client.26359 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:11:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.964+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613a36adff0 con 0x5613a380c890 2023-08-11T14:11:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.966+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a380c890 0x5613a380ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:11:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.968+0000 7f6c55d9b700 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a380c890 0x5613a380ec80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a544bd19f2a5ae3b server_cookie=b9ee556b349d7f7c in_seq=0 out_seq=0 2023-08-11T14:11:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.969+0000 7f6c397fa700 10 client.26359 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:11:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c397fa700 1 -- 172.21.15.3:0/2247631350 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613a36adff0 con 0x5613a380c890 2023-08-11T14:11:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c397fa700 10 client.26359 handle_command_reply: tid=0 2023-08-11T14:11:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 client.26359 shutdown 2023-08-11T14:11:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 msgr2=0x5613a37b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:11:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b7e0 tx=0x7f6c4801c960 comp rx=0 tx=0).stop 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 shutdown_connections 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613a37b2770 0x5613a37b2bf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5613a380c890 0x5613a380ec80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5613a37a1f40 0x5613a37b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 --2- 172.21.15.3:0/2247631350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5613a37a3790 0x5613a37b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:11:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c397fa700 0 client.26359 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.996+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 >> 172.21.15.3:0/2247631350 conn(0x5613a36cf8f0 msgr2=0x5613a379bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.997+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 shutdown_connections 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.997+0000 7f6c6b915b80 1 -- 172.21.15.3:0/2247631350 wait complete. 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.997+0000 7f6c6b915b80 20 client.26359 trim_cache size 0 max 16384 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:12:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:12:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:12:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:12:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.998+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 msgr2=0x7f6c3c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.998+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 0x7f6c3c0a2610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5613a37b3910 tx=0x7f6c4002c000 comp rx=0 tx=0).stop 2023-08-11T14:12:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.998+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 msgr2=0x7f6c50114490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:11:59.998+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00e9c0 tx=0x7f6c4c00ed80 comp rx=0 tx=0).stop 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 shutdown_connections 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c5010cc60 0x7f6c50115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c3c0a0150 0x7f6c3c0a2610 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c50107920 0x7f6c50113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 --2- 172.21.15.3:0/313791791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c5010a290 0x7f6c50114490 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.001+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 >> 172.21.15.3:0/313791791 conn(0x7f6c5006d750 msgr2=0x7f6c5010bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.003+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 shutdown_connections 2023-08-11T14:12:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:00.003+0000 7f6c57fff700 1 -- 172.21.15.3:0/313791791 wait complete. 2023-08-11T14:12:00.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:00 smithi003 ceph-mon[97195]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:12:00.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:00 smithi187 ceph-mon[130973]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:12:00.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:00 smithi103 ceph-mon[130434]: pgmap v589: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:12:02.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:02 smithi003 ceph-mon[97195]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:12:02.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:02 smithi187 ceph-mon[130973]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:12:02.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:02 smithi103 ceph-mon[130434]: pgmap v590: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:12:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 305.0 s, 725.3 tps, lat 137.393 ms stddev 139.067 2023-08-11T14:12:04.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:04 smithi103 ceph-mon[130434]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:12:04.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:04 smithi003 ceph-mon[97195]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:12:04.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:04 smithi187 ceph-mon[130973]: pgmap v591: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:12:06.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:06 smithi103 ceph-mon[130434]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 936 op/s 2023-08-11T14:12:06.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:06 smithi003 ceph-mon[97195]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 936 op/s 2023-08-11T14:12:06.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:06 smithi187 ceph-mon[130973]: pgmap v592: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 936 op/s 2023-08-11T14:12:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 310.0 s, 804.1 tps, lat 123.114 ms stddev 195.883 2023-08-11T14:12:08.435 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:08 smithi103 ceph-mon[130434]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:12:08.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:08 smithi003 ceph-mon[97195]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:12:08.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:08 smithi187 ceph-mon[130973]: pgmap v593: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:12:09.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:09 smithi003 ceph-mon[97195]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-08-11T14:12:09.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:09 smithi187 ceph-mon[130973]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-08-11T14:12:09.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:09 smithi103 ceph-mon[130434]: pgmap v594: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 1.05k op/s 2023-08-11T14:12:12.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:11 smithi003 ceph-mon[97195]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2023-08-11T14:12:12.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:11 smithi187 ceph-mon[130973]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2023-08-11T14:12:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:11 smithi103 ceph-mon[130434]: pgmap v595: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2023-08-11T14:12:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 315.0 s, 615.7 tps, lat 165.813 ms stddev 463.322 2023-08-11T14:12:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.277+0000 7f097b7ff700 1 -- 172.21.15.3:0/368542835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0974107920 msgr2=0x7f0974109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.277+0000 7f097b7ff700 1 --2- 172.21.15.3:0/368542835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0974107920 0x7f0974109d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f097001a9a0 tx=0x7f097001acb0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.307+0000 7f097b7ff700 1 -- 172.21.15.3:0/368542835 shutdown_connections 2023-08-11T14:12:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.307+0000 7f097b7ff700 1 --2- 172.21.15.3:0/368542835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410cc60 0x7f097410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.307+0000 7f097b7ff700 1 --2- 172.21.15.3:0/368542835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0974107920 0x7f0974109d50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.307+0000 7f097b7ff700 1 --2- 172.21.15.3:0/368542835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097410a290 0x7f097410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.307+0000 7f097b7ff700 1 -- 172.21.15.3:0/368542835 >> 172.21.15.3:0/368542835 conn(0x7f097406d750 msgr2=0x7f097406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.317+0000 7f097b7ff700 1 -- 172.21.15.3:0/368542835 shutdown_connections 2023-08-11T14:12:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.317+0000 7f097b7ff700 1 -- 172.21.15.3:0/368542835 wait complete. 2023-08-11T14:12:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.318+0000 7f097b7ff700 1 Processor -- start 2023-08-11T14:12:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.319+0000 7f097b7ff700 1 -- start start 2023-08-11T14:12:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 0x7f0974118000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410a290 0x7f0974118540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 0x7f09741c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741c6b00 con 0x7f097410cc60 2023-08-11T14:12:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741c6c40 con 0x7f097410a290 2023-08-11T14:12:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.320+0000 7f097b7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741c6db0 con 0x7f0974107920 2023-08-11T14:12:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.322+0000 7f097959b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 0x7f0974118000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.322+0000 7f0979d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 0x7f09741c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.323+0000 7f097959b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 0x7f0974118000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51378/0 (socket says 172.21.15.3:51378) 2023-08-11T14:12:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.323+0000 7f097959b700 1 -- 172.21.15.3:0/9387636 learned_addr learned my addr 172.21.15.3:0/9387636 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.323+0000 7f0978d9a700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410a290 0x7f0974118540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.325+0000 7f0979d9c700 1 -- 172.21.15.3:0/9387636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 msgr2=0x7f0974118000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.325+0000 7f0979d9c700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 0x7f0974118000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.325+0000 7f0979d9c700 1 -- 172.21.15.3:0/9387636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410a290 msgr2=0x7f0974118540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.325+0000 7f0979d9c700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410a290 0x7f0974118540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.325+0000 7f0979d9c700 1 -- 172.21.15.3:0/9387636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097001a600 con 0x7f097410cc60 2023-08-11T14:12:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.326+0000 7f0979d9c700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 0x7f09741c65c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be20 tx=0x7f096c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.357+0000 7f096a7fc700 1 -- 172.21.15.3:0/9387636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00cd10 con 0x7f097410cc60 2023-08-11T14:12:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.357+0000 7f096a7fc700 1 -- 172.21.15.3:0/9387636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c00ce70 con 0x7f097410cc60 2023-08-11T14:12:13.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.357+0000 7f096a7fc700 1 -- 172.21.15.3:0/9387636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0127e0 con 0x7f097410cc60 2023-08-11T14:12:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.357+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09741c7090 con 0x7f097410cc60 2023-08-11T14:12:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.357+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09741c75d0 con 0x7f097410cc60 2023-08-11T14:12:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.382+0000 7f098f14eb80 1 Processor -- start 2023-08-11T14:12:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.382+0000 7f098f14eb80 1 -- start start 2023-08-11T14:12:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.382+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.382+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f40f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f40fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f098f14eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f3675d0 con 0x55de0f40f790 2023-08-11T14:12:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f098f14eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f364ef0 con 0x55de0f40df40 2023-08-11T14:12:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f098f14eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f3607e0 con 0x55de0f40e8a0 2023-08-11T14:12:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f0978d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f0978d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:54434/0 (socket says 172.21.15.3:54434) 2023-08-11T14:12:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.383+0000 7f0978d9a700 1 -- 172.21.15.3:0/4093931579 learned_addr learned my addr 172.21.15.3:0/4093931579 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f40fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 -- 172.21.15.3:0/4093931579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 msgr2=0x55de0f40f1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f40f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 -- 172.21.15.3:0/4093931579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 msgr2=0x55de0f40fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f40fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.387+0000 7f0978d9a700 1 -- 172.21.15.3:0/4093931579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de0f1e05d0 con 0x55de0f40df40 2023-08-11T14:12:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.388+0000 7f0978d9a700 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f096400cf10 tx=0x7f096400c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e383d1de3882ffae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.391+0000 7f096a7fc700 1 -- 172.21.15.3:0/9387636 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f096c012940 con 0x7f097410cc60 2023-08-11T14:12:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.393+0000 7f096a7fc700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 0x7f09600a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.394+0000 7f096a7fc700 1 -- 172.21.15.3:0/9387636 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f096c006e40 con 0x7f097410cc60 2023-08-11T14:12:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.394+0000 7f095f7fe700 1 -- 172.21.15.3:0/4093931579 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096400eb10 con 0x55de0f40df40 2023-08-11T14:12:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.394+0000 7f095f7fe700 1 -- 172.21.15.3:0/4093931579 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09640044e0 con 0x55de0f40df40 2023-08-11T14:12:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.394+0000 7f095f7fe700 1 -- 172.21.15.3:0/4093931579 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09640163f0 con 0x55de0f40df40 2023-08-11T14:12:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.396+0000 7f098f14eb80 1 -- 172.21.15.3:0/4093931579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 msgr2=0x55de0f40e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.396+0000 7f098f14eb80 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f096400cf10 tx=0x7f096400c390 comp rx=0 tx=0).stop 2023-08-11T14:12:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.397+0000 7f097959b700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 0x7f09600a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.398+0000 7f097959b700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 0x7f09600a2610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0970005850 tx=0x7f097001aef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.428+0000 7f098f14eb80 1 -- 172.21.15.3:0/4093931579 shutdown_connections 2023-08-11T14:12:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.428+0000 7f098f14eb80 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f40fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.428+0000 7f098f14eb80 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f40e360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.428+0000 7f098f14eb80 1 --2- 172.21.15.3:0/4093931579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f40f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.428+0000 7f098f14eb80 1 -- 172.21.15.3:0/4093931579 >> 172.21.15.3:0/4093931579 conn(0x55de0f33b8f0 msgr2=0x55de0f40a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.440+0000 7f098f14eb80 1 -- 172.21.15.3:0/4093931579 shutdown_connections 2023-08-11T14:12:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.440+0000 7f098f14eb80 1 -- 172.21.15.3:0/4093931579 wait complete. 2023-08-11T14:12:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.441+0000 7f098f14eb80 1 -- start start 2023-08-11T14:12:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.441+0000 7f098f14eb80 1 Processor -- start 2023-08-11T14:12:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f417c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f418190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f2fea60 con 0x55de0f40f790 2023-08-11T14:12:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f2fe760 con 0x55de0f40df40 2023-08-11T14:12:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f1e28c0 con 0x55de0f40e8a0 2023-08-11T14:12:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.442+0000 7f098f14eb80 10 client.0 _ll_register_callbacks cb 0x55de0f301a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.443+0000 7f0978d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f418190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.443+0000 7f0978d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f418190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-08-11T14:12:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.443+0000 7f0978d9a700 1 -- 172.21.15.3:0/2708157695 learned_addr learned my addr 172.21.15.3:0/2708157695 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.448+0000 7f0979d9c700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.451+0000 7f0979d9c700 1 -- 172.21.15.3:0/2708157695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 msgr2=0x55de0f418190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.451+0000 7f0979d9c700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f418190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.451+0000 7f0979d9c700 1 -- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 msgr2=0x55de0f417c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.451+0000 7f0979d9c700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f417c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.451+0000 7f0979d9c700 1 -- 172.21.15.3:0/2708157695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096400caa0 con 0x55de0f40f790 2023-08-11T14:12:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.452+0000 7f0979d9c700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41f1d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f096c0c81e0 tx=0x7f096c0981e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa905e5bce082dbf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.452+0000 7f095cff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:12:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.452+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c097820 con 0x55de0f40f790 2023-08-11T14:12:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.452+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f096c09fab0 con 0x55de0f40f790 2023-08-11T14:12:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.453+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de0f4017e0 con 0x55de0f40f790 2023-08-11T14:12:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.453+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096c09fc70 con 0x55de0f40f790 2023-08-11T14:12:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.453+0000 7f098f14eb80 10 client.16224 fetch_fsmap learned FSMap version 36 2023-08-11T14:12:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.453+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de0f18cd80 con 0x55de0f40f790 2023-08-11T14:12:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c0b4070 con 0x55de0f40f790 2023-08-11T14:12:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f096c097090 con 0x55de0f40f790 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 10 client.16224 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 20 client.16224 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f423ac0 0x55de0f425f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 4 client.16224 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:12:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.454+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de0f3dd480 con 0x55de0f423ac0 2023-08-11T14:12:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.459+0000 7f097959b700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f423ac0 0x55de0f425f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.498+0000 7f097959b700 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f423ac0 0x55de0f425f80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3be3aa82d2ceb2 server_cookie=2b9f6368abf1e5c0 in_seq=0 out_seq=0 2023-08-11T14:12:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.498+0000 7f095cff9700 10 client.16224 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.506+0000 7f095cff9700 1 -- 172.21.15.3:0/2708157695 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55de0f3dd480 con 0x55de0f423ac0 2023-08-11T14:12:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.506+0000 7f095cff9700 10 client.16224 handle_command_reply: tid=0 2023-08-11T14:12:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.506+0000 7f098f14eb80 1 client.16224 shutdown 2023-08-11T14:12:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.512+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 msgr2=0x55de0f41f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.512+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41f1d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f096c0c81e0 tx=0x7f096c0981e0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 shutdown_connections 2023-08-11T14:12:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41f1d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f423ac0 0x55de0f425f80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f417c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 --2- 172.21.15.3:0/2708157695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f418190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.530+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 >> 172.21.15.3:0/2708157695 conn(0x55de0f33b8f0 msgr2=0x55de0f409710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.532+0000 7f095cff9700 0 client.16224 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.533+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 shutdown_connections 2023-08-11T14:12:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.544+0000 7f098f14eb80 1 -- 172.21.15.3:0/2708157695 wait complete. 2023-08-11T14:12:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.550+0000 7f098f14eb80 20 client.16224 trim_cache size 0 max 16384 2023-08-11T14:12:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.579+0000 7f098f14eb80 1 Processor -- start 2023-08-11T14:12:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.579+0000 7f098f14eb80 1 -- start start 2023-08-11T14:12:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.579+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 0x55de0f40fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.579+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.580+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40f790 0x55de0f409f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.580+0000 7f098f14eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f42adc0 con 0x55de0f40df40 2023-08-11T14:12:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.580+0000 7f098f14eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f403ea0 con 0x55de0f40f790 2023-08-11T14:12:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.580+0000 7f098f14eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f40db40 con 0x55de0f40e8a0 2023-08-11T14:12:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.586+0000 7f097959b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 0x55de0f40fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.586+0000 7f0978d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.586+0000 7f097959b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 0x55de0f40fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46508/0 (socket says 172.21.15.3:46508) 2023-08-11T14:12:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.586+0000 7f097959b700 1 -- 172.21.15.3:0/229692736 learned_addr learned my addr 172.21.15.3:0/229692736 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.587+0000 7f0979d9c700 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40f790 0x55de0f409f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.596+0000 7f0978d9a700 1 -- 172.21.15.3:0/229692736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40f790 msgr2=0x55de0f409f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.596+0000 7f0978d9a700 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40f790 0x55de0f409f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.596+0000 7f0978d9a700 1 -- 172.21.15.3:0/229692736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 msgr2=0x55de0f40fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.596+0000 7f0978d9a700 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 0x55de0f40fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.597+0000 7f0978d9a700 1 -- 172.21.15.3:0/229692736 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de0f175c00 con 0x55de0f40e8a0 2023-08-11T14:12:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.612+0000 7f0978d9a700 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41f190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f096400c7f0 tx=0x7f09640046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eddb9b0221ec8e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.618+0000 7f095ffff700 1 -- 172.21.15.3:0/229692736 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096400e910 con 0x55de0f40e8a0 2023-08-11T14:12:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.618+0000 7f095ffff700 1 -- 172.21.15.3:0/229692736 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0964020460 con 0x55de0f40e8a0 2023-08-11T14:12:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.618+0000 7f095ffff700 1 -- 172.21.15.3:0/229692736 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964016650 con 0x55de0f40e8a0 2023-08-11T14:12:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 -- 172.21.15.3:0/229692736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 msgr2=0x55de0f41f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41f190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f096400c7f0 tx=0x7f09640046d0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 -- 172.21.15.3:0/229692736 shutdown_connections 2023-08-11T14:12:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40f790 0x55de0f409f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40df40 0x55de0f40fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 --2- 172.21.15.3:0/229692736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41f190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.619+0000 7f098f14eb80 1 -- 172.21.15.3:0/229692736 >> 172.21.15.3:0/229692736 conn(0x55de0f33b8f0 msgr2=0x55de0f4126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.625+0000 7f098f14eb80 1 -- 172.21.15.3:0/229692736 shutdown_connections 2023-08-11T14:12:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.626+0000 7f098f14eb80 1 -- 172.21.15.3:0/229692736 wait complete. 2023-08-11T14:12:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.626+0000 7f098f14eb80 1 -- start start 2023-08-11T14:12:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 Processor -- start 2023-08-11T14:12:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f41ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f41e4a0 con 0x55de0f40f790 2023-08-11T14:12:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f41e610 con 0x55de0f40df40 2023-08-11T14:12:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.627+0000 7f098f14eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0f41e7d0 con 0x55de0f40e8a0 2023-08-11T14:12:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.628+0000 7f098f14eb80 10 client.0 _ll_register_callbacks cb 0x7f096c097a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.628+0000 7f097959b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f41ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.629+0000 7f097959b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f41ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:54458/0 (socket says 172.21.15.3:54458) 2023-08-11T14:12:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.629+0000 7f097959b700 1 -- 172.21.15.3:0/601140940 learned_addr learned my addr 172.21.15.3:0/601140940 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.629+0000 7f0978d9a700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.630+0000 7f0979d9c700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.642+0000 7f0979d9c700 1 -- 172.21.15.3:0/601140940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 msgr2=0x55de0f41cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.642+0000 7f0979d9c700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.642+0000 7f0979d9c700 1 -- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 msgr2=0x55de0f41ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.642+0000 7f0979d9c700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f41ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.642+0000 7f0979d9c700 1 -- 172.21.15.3:0/601140940 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de0f1e05d0 con 0x55de0f40f790 2023-08-11T14:12:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.660+0000 7f0979d9c700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41df60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f096c094b50 tx=0x7f096c0988e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ba6d8839f1efda server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.663+0000 7f095cff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:12:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.687+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0b71a0 con 0x55de0f40f790 2023-08-11T14:12:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.687+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f096c097090 con 0x55de0f40f790 2023-08-11T14:12:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.688+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09640044e0 con 0x55de0f40f790 2023-08-11T14:12:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.697+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096c0c8070 con 0x55de0f40f790 2023-08-11T14:12:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.701+0000 7f098f14eb80 10 client.16230 fetch_fsmap learned FSMap version 36 2023-08-11T14:12:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.701+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55de0f1b31f0 con 0x55de0f40f790 2023-08-11T14:12:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09640044e0 con 0x55de0f40f790 2023-08-11T14:12:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 36) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f096c0be050 con 0x55de0f40f790 2023-08-11T14:12:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 10 client.16230 fetch_fsmap finished waiting for FSMap version 36 2023-08-11T14:12:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 20 client.16230 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f477eb0 0x55de0f47a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 4 client.16230 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:12:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.721+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de0f3597c0 con 0x55de0f477eb0 2023-08-11T14:12:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.732+0000 7f097959b700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f477eb0 0x55de0f47a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.748+0000 7f097959b700 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f477eb0 0x55de0f47a2d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e9e7bbd533ec69 server_cookie=9246c9c081b634bb in_seq=0 out_seq=0 2023-08-11T14:12:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:13.748+0000 7f095cff9700 10 client.16230 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:14.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:13 smithi003 ceph-mon[97195]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 19 MiB/s wr, 1.28k op/s 2023-08-11T14:12:14.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:13 smithi187 ceph-mon[130973]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 19 MiB/s wr, 1.28k op/s 2023-08-11T14:12:14.434 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:13 smithi103 ceph-mon[130434]: pgmap v596: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 19 MiB/s wr, 1.28k op/s 2023-08-11T14:12:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:15 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:12:15.434 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:15 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:12:15.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:15 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:12:16.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:16 smithi103 ceph-mon[130434]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:12:16.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:16 smithi187 ceph-mon[130973]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:12:16.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:16 smithi003 ceph-mon[97195]: pgmap v597: 129 pgs: 129 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:12:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.253+0000 7f095cff9700 1 -- 172.21.15.3:0/601140940 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de0f3597c0 con 0x55de0f477eb0 2023-08-11T14:12:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.253+0000 7f095cff9700 10 client.16230 handle_command_reply: tid=0 2023-08-11T14:12:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.253+0000 7f098f14eb80 1 client.16230 shutdown 2023-08-11T14:12:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 msgr2=0x55de0f41df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41df60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f096c094b50 tx=0x7f096c0988e0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 shutdown_connections 2023-08-11T14:12:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55de0f40f790 0x55de0f41df60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55de0f477eb0 0x55de0f47a2d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de0f40df40 0x55de0f41ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 --2- 172.21.15.3:0/601140940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de0f40e8a0 0x55de0f41cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 >> 172.21.15.3:0/601140940 conn(0x55de0f33b8f0 msgr2=0x55de0f409f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.257+0000 7f095cff9700 0 client.16230 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.267+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 shutdown_connections 2023-08-11T14:12:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.269+0000 7f098f14eb80 1 -- 172.21.15.3:0/601140940 wait complete. 2023-08-11T14:12:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.272+0000 7f098f14eb80 20 client.16230 trim_cache size 0 max 16384 2023-08-11T14:12:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:12:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:12:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:12:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:12:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.281+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 msgr2=0x7f09600a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.281+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 0x7f09600a2610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0970005850 tx=0x7f097001aef0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.281+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 msgr2=0x7f09741c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.281+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 0x7f09741c65c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f096c00be20 tx=0x7f096c00bf00 comp rx=0 tx=0).stop 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 shutdown_connections 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f097410cc60 0x7f09741c65c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0974107920 0x7f0974118000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f09600a0150 0x7f09600a2610 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 --2- 172.21.15.3:0/9387636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097410a290 0x7f0974118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.282+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 >> 172.21.15.3:0/9387636 conn(0x7f097406d750 msgr2=0x7f09741082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.284+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 shutdown_connections 2023-08-11T14:12:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:17.286+0000 7f097b7ff700 1 -- 172.21.15.3:0/9387636 wait complete. 2023-08-11T14:12:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 320.0 s, 373.4 tps, lat 263.354 ms stddev 320.042 2023-08-11T14:12:18.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:18 smithi103 ceph-mon[130434]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:12:18.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:12:18.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:12:18.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:18 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:12:18.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:18 smithi003 ceph-mon[97195]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:12:18.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:12:18.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:12:18.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:18 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:12:18.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:18 smithi187 ceph-mon[130973]: pgmap v598: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-08-11T14:12:18.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:12:18.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:12:18.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:18 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:12:20.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:20 smithi003 ceph-mon[97195]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:12:20.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:20 smithi187 ceph-mon[130973]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:12:20.780 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:20 smithi103 ceph-mon[130434]: pgmap v599: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:12:22.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:22 smithi187 ceph-mon[130973]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-08-11T14:12:22.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:22 smithi003 ceph-mon[97195]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-08-11T14:12:22.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:22 smithi103 ceph-mon[130434]: pgmap v600: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 16 MiB/s wr, 1.14k op/s 2023-08-11T14:12:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 325.0 s, 1061.2 tps, lat 75.497 ms stddev 56.352 2023-08-11T14:12:24.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:24 smithi003 ceph-mon[97195]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-08-11T14:12:24.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:24 smithi187 ceph-mon[130973]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-08-11T14:12:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:24 smithi103 ceph-mon[130434]: pgmap v601: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 1.14k op/s 2023-08-11T14:12:26.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:26 smithi103 ceph-mon[130434]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 882 op/s 2023-08-11T14:12:26.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:26 smithi187 ceph-mon[130973]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 882 op/s 2023-08-11T14:12:26.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:26 smithi003 ceph-mon[97195]: pgmap v602: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 882 op/s 2023-08-11T14:12:28.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 330.0 s, 530.1 tps, lat 227.909 ms stddev 373.001 2023-08-11T14:12:28.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:28 smithi103 ceph-mon[130434]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-08-11T14:12:28.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:28 smithi003 ceph-mon[97195]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-08-11T14:12:28.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:28 smithi187 ceph-mon[130973]: pgmap v603: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 1.26k op/s 2023-08-11T14:12:30.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:30 smithi103 ceph-mon[130434]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-08-11T14:12:30.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:30 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:30.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:30 smithi187 ceph-mon[130973]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-08-11T14:12:30.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:30 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:30.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:30 smithi003 ceph-mon[97195]: pgmap v604: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-08-11T14:12:30.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:30 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.947+0000 7f0b229ff700 1 -- 172.21.15.3:0/1116207103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c107920 msgr2=0x7f0b1c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.947+0000 7f0b229ff700 1 --2- 172.21.15.3:0/1116207103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c107920 0x7f0b1c109d50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009990 tx=0x7f0b10009ca0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.948+0000 7f0b229ff700 1 -- 172.21.15.3:0/1116207103 shutdown_connections 2023-08-11T14:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.948+0000 7f0b229ff700 1 --2- 172.21.15.3:0/1116207103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10cc60 0x7f0b1c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.948+0000 7f0b229ff700 1 --2- 172.21.15.3:0/1116207103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c107920 0x7f0b1c109d50 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009990 tx=0x7f0b10009ca0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.948+0000 7f0b229ff700 1 --2- 172.21.15.3:0/1116207103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b1c10a290 0x7f0b1c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.948+0000 7f0b229ff700 1 -- 172.21.15.3:0/1116207103 >> 172.21.15.3:0/1116207103 conn(0x7f0b1c06d750 msgr2=0x7f0b1c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.965+0000 7f0b229ff700 1 -- 172.21.15.3:0/1116207103 shutdown_connections 2023-08-11T14:12:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.966+0000 7f0b229ff700 1 -- 172.21.15.3:0/1116207103 wait complete. 2023-08-11T14:12:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.968+0000 7f0b229ff700 1 Processor -- start 2023-08-11T14:12:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.968+0000 7f0b229ff700 1 -- start start 2023-08-11T14:12:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.968+0000 7f0b229ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10a290 0x7f0b1c1141f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.968+0000 7f0b229ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b1c10cc60 0x7f0b1c114730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.969+0000 7f0b229ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.969+0000 7f0b229ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1160b0 con 0x7f0b1c115600 2023-08-11T14:12:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.969+0000 7f0b229ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c116220 con 0x7f0b1c10a290 2023-08-11T14:12:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.969+0000 7f0b229ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1163e0 con 0x7f0b1c10cc60 2023-08-11T14:12:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.974+0000 7f0b20f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44134/0 (socket says 172.21.15.3:44134) 2023-08-11T14:12:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 -- 172.21.15.3:0/2453814506 learned_addr learned my addr 172.21.15.3:0/2453814506 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10a290 0x7f0b1c1141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 -- 172.21.15.3:0/2453814506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b1c10cc60 msgr2=0x7f0b1c114730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:12:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b1c10cc60 0x7f0b1c114730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 -- 172.21.15.3:0/2453814506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10a290 msgr2=0x7f0b1c1141f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10a290 0x7f0b1c1141f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.975+0000 7f0b20f9c700 1 -- 172.21.15.3:0/2453814506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100095f0 con 0x7f0b1c115600 2023-08-11T14:12:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.977+0000 7f0b20f9c700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b780 tx=0x7f0b1400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.978+0000 7f0b197fa700 1 -- 172.21.15.3:0/2453814506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400f840 con 0x7f0b1c115600 2023-08-11T14:12:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.978+0000 7f0b197fa700 1 -- 172.21.15.3:0/2453814506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b1400fe60 con 0x7f0b1c115600 2023-08-11T14:12:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.978+0000 7f0b197fa700 1 -- 172.21.15.3:0/2453814506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400d3d0 con 0x7f0b1c115600 2023-08-11T14:12:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.978+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c1168a0 con 0x7f0b1c115600 2023-08-11T14:12:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.978+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c116db0 con 0x7f0b1c115600 2023-08-11T14:12:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:30.993+0000 7f0b36357b80 1 Processor -- start 2023-08-11T14:12:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.003+0000 7f0b36357b80 1 -- start start 2023-08-11T14:12:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.003+0000 7f0b197fa700 1 -- 172.21.15.3:0/2453814506 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0b1400d570 con 0x7f0b1c115600 2023-08-11T14:12:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.003+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e225086360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.003+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.004+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225087c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.004+0000 7f0b36357b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224fdf5d0 con 0x55e225087790 2023-08-11T14:12:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.004+0000 7f0b36357b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224fdcef0 con 0x55e2250868a0 2023-08-11T14:12:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.004+0000 7f0b36357b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224fd87e0 con 0x55e225085f40 2023-08-11T14:12:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.006+0000 7f0b197fa700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 0x7f0b040a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.006+0000 7f0b197fa700 1 -- 172.21.15.3:0/2453814506 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b140cb7e0 con 0x7f0b1c115600 2023-08-11T14:12:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.010+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225087c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.010+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225087c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44150/0 (socket says 172.21.15.3:44150) 2023-08-11T14:12:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.010+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/2559298741 learned_addr learned my addr 172.21.15.3:0/2559298741 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.010+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e225086360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.014+0000 7f0b20f9c700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 0x7f0b040a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.014+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.019+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/2559298741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 msgr2=0x55e2250871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.019+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250871b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.019+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/2559298741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 msgr2=0x55e225087c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.019+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225087c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.019+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/2559298741 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e224e585d0 con 0x55e225085f40 2023-08-11T14:12:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.029+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e225086360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0121b0 tx=0x7f0b0c006960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b14b02ce2220fc98 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.043+0000 7f0b20f9c700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 0x7f0b040a2610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009fd0 tx=0x7f0b140098d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.045+0000 7f0b027fc700 1 -- 172.21.15.3:0/2559298741 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c012ba0 con 0x55e225085f40 2023-08-11T14:12:31.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.045+0000 7f0b027fc700 1 -- 172.21.15.3:0/2559298741 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b0c004d90 con 0x55e225085f40 2023-08-11T14:12:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.045+0000 7f0b027fc700 1 -- 172.21.15.3:0/2559298741 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0173c0 con 0x55e225085f40 2023-08-11T14:12:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.046+0000 7f0b36357b80 1 -- 172.21.15.3:0/2559298741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 msgr2=0x55e225086360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.046+0000 7f0b36357b80 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e225086360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0121b0 tx=0x7f0b0c006960 comp rx=0 tx=0).stop 2023-08-11T14:12:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.050+0000 7f0b36357b80 1 -- 172.21.15.3:0/2559298741 shutdown_connections 2023-08-11T14:12:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.050+0000 7f0b36357b80 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225087c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.050+0000 7f0b36357b80 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e225086360 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.050+0000 7f0b36357b80 1 --2- 172.21.15.3:0/2559298741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.050+0000 7f0b36357b80 1 -- 172.21.15.3:0/2559298741 >> 172.21.15.3:0/2559298741 conn(0x55e224fb38f0 msgr2=0x55e225082920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:31.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.051+0000 7f0b36357b80 1 -- 172.21.15.3:0/2559298741 shutdown_connections 2023-08-11T14:12:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.057+0000 7f0b36357b80 1 -- 172.21.15.3:0/2559298741 wait complete. 2023-08-11T14:12:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.057+0000 7f0b36357b80 1 -- start start 2023-08-11T14:12:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.058+0000 7f0b36357b80 1 Processor -- start 2023-08-11T14:12:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e22508fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250901d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224f76a60 con 0x55e225087790 2023-08-11T14:12:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224f76760 con 0x55e2250868a0 2023-08-11T14:12:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e224e5a8c0 con 0x55e225085f40 2023-08-11T14:12:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.059+0000 7f0b36357b80 10 client.0 _ll_register_callbacks cb 0x55e224f79a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.060+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.060+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44160/0 (socket says 172.21.15.3:44160) 2023-08-11T14:12:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.060+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/447048949 learned_addr learned my addr 172.21.15.3:0/447048949 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.061+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/447048949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 msgr2=0x55e22508fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.061+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e22508fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.061+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 msgr2=0x55e2250901d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.061+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250901d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.061+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/447048949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c013040 con 0x55e225087790 2023-08-11T14:12:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.062+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f10 tx=0x7f0b0c0173c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91da0bd63d1890a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.062+0000 7f0afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:12:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.063+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c024070 con 0x55e225087790 2023-08-11T14:12:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.063+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b0c004a70 con 0x55e225087790 2023-08-11T14:12:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.063+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2250797e0 con 0x55e225087790 2023-08-11T14:12:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.064+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b0c004c30 con 0x55e225087790 2023-08-11T14:12:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.069+0000 7f0b36357b80 10 client.16242 fetch_fsmap learned FSMap version 37 2023-08-11T14:12:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.069+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e224e04d80 con 0x55e225087790 2023-08-11T14:12:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2250797e0 con 0x55e225087790 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0b0c027020 con 0x55e225087790 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 37 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 20 client.16242 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e22509b9e0 0x55e22509de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 4 client.16242 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:12:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.081+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e225055480 con 0x55e22509b9e0 2023-08-11T14:12:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.093+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e22509b9e0 0x55e22509de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.130+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e22509b9e0 0x55e22509de00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af2266dfba98dbf0 server_cookie=9e9ffd82b8fbbf1f in_seq=0 out_seq=0 2023-08-11T14:12:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.137+0000 7f0afb7fe700 10 client.16242 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.148+0000 7f0afb7fe700 1 -- 172.21.15.3:0/447048949 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e225055480 con 0x55e22509b9e0 2023-08-11T14:12:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.148+0000 7f0afb7fe700 10 client.16242 handle_command_reply: tid=0 2023-08-11T14:12:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.148+0000 7f0b36357b80 1 client.16242 shutdown 2023-08-11T14:12:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.164+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 msgr2=0x55e225097210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.164+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f10 tx=0x7f0b0c0173c0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 shutdown_connections 2023-08-11T14:12:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225087790 0x55e225097210 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225085f40 0x55e22508fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e22509b9e0 0x55e22509de00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 --2- 172.21.15.3:0/447048949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250901d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.175+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 >> 172.21.15.3:0/447048949 conn(0x55e224fb38f0 msgr2=0x55e225081710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.176+0000 7f0afb7fe700 0 client.16242 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.181+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 shutdown_connections 2023-08-11T14:12:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.183+0000 7f0b36357b80 1 -- 172.21.15.3:0/447048949 wait complete. 2023-08-11T14:12:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.189+0000 7f0b36357b80 20 client.16242 trim_cache size 0 max 16384 2023-08-11T14:12:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 Processor -- start 2023-08-11T14:12:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 -- start start 2023-08-11T14:12:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e22508a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 0x55e22508ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e225085b70 con 0x55e225085f40 2023-08-11T14:12:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e225090910 con 0x55e2250868a0 2023-08-11T14:12:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.219+0000 7f0b36357b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22508d980 con 0x55e225087790 2023-08-11T14:12:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.226+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 0x55e22508ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.226+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 0x55e22508ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52266/0 (socket says 172.21.15.3:52266) 2023-08-11T14:12:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.226+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/1473826876 learned_addr learned my addr 172.21.15.3:0/1473826876 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.226+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e22508a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.231+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.240+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/1473826876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 msgr2=0x55e22508ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.240+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 0x55e22508ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.240+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/1473826876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 msgr2=0x55e2250971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.240+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.240+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/1473826876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e224dedc00 con 0x55e225085f40 2023-08-11T14:12:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.268+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e22508a720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00b9f0 tx=0x7f0b0c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7c6864c36551ed server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.280+0000 7f0b02ffd700 1 -- 172.21.15.3:0/1473826876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c024040 con 0x55e225085f40 2023-08-11T14:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.281+0000 7f0b02ffd700 1 -- 172.21.15.3:0/1473826876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b0c006230 con 0x55e225085f40 2023-08-11T14:12:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.281+0000 7f0b02ffd700 1 -- 172.21.15.3:0/1473826876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c004c30 con 0x55e225085f40 2023-08-11T14:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.282+0000 7f0b36357b80 1 -- 172.21.15.3:0/1473826876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 msgr2=0x55e22508a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.282+0000 7f0b36357b80 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e22508a720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00b9f0 tx=0x7f0b0c000bc0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.283+0000 7f0b36357b80 1 -- 172.21.15.3:0/1473826876 shutdown_connections 2023-08-11T14:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.283+0000 7f0b36357b80 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e225087790 0x55e22508ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.283+0000 7f0b36357b80 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e22508a720 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.283+0000 7f0b36357b80 1 --2- 172.21.15.3:0/1473826876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2250868a0 0x55e2250971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.283+0000 7f0b36357b80 1 -- 172.21.15.3:0/1473826876 >> 172.21.15.3:0/1473826876 conn(0x55e224fb38f0 msgr2=0x55e22508e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.297+0000 7f0b36357b80 1 -- 172.21.15.3:0/1473826876 shutdown_connections 2023-08-11T14:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.305+0000 7f0b36357b80 1 -- 172.21.15.3:0/1473826876 wait complete. 2023-08-11T14:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.306+0000 7f0b36357b80 1 -- start start 2023-08-11T14:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.306+0000 7f0b36357b80 1 Processor -- start 2023-08-11T14:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e225094d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2250868a0 0x55e2250952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 0x55e2250962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2250967f0 con 0x55e225085f40 2023-08-11T14:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e225096960 con 0x55e225087790 2023-08-11T14:12:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e225096b20 con 0x55e2250868a0 2023-08-11T14:12:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.307+0000 7f0b36357b80 10 client.0 _ll_register_callbacks cb 0x7f0b10009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.312+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 0x55e2250962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.320+0000 7f0b1b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 0x55e2250962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49336/0 (socket says 172.21.15.3:49336) 2023-08-11T14:12:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.320+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/3697220973 learned_addr learned my addr 172.21.15.3:0/3697220973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.325+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2250868a0 0x55e2250952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.325+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e225094d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.326+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/3697220973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2250868a0 msgr2=0x55e2250952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.326+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2250868a0 0x55e2250952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.326+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 msgr2=0x55e2250962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.326+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 0x55e2250962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.326+0000 7f0b1b7fe700 1 -- 172.21.15.3:0/3697220973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e224e04d80 con 0x55e225085f40 2023-08-11T14:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.327+0000 7f0b1b7fe700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e225094d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00a010 tx=0x7f0b0c006110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ae02b34db11af4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.329+0000 7f0afb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.329+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c024070 con 0x55e225085f40 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.329+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f0b0c003810 con 0x55e225085f40 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.330+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b0c004d90 con 0x55e225085f40 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.337+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b0c020ea0 con 0x55e225085f40 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.338+0000 7f0b36357b80 10 client.16248 fetch_fsmap learned FSMap version 37 2023-08-11T14:12:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.338+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e224e2b1f0 con 0x55e225085f40 2023-08-11T14:12:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.360+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b0c004d90 con 0x55e225085f40 2023-08-11T14:12:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.360+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f0b0c027070 con 0x55e225085f40 2023-08-11T14:12:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 37 2023-08-11T14:12:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 20 client.16248 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e2250eff00 0x55e2250f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 4 client.16248 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:12:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.361+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e224fd17c0 con 0x55e2250eff00 2023-08-11T14:12:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.370+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e2250eff00 0x55e2250f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.406+0000 7f0b1bfff700 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e2250eff00 0x55e2250f2320 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3deacea90dd54b0 server_cookie=baff25e69e33bd in_seq=0 out_seq=0 2023-08-11T14:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:31.406+0000 7f0afb7fe700 10 client.16248 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:32.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:32 smithi187 ceph-mon[130973]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:12:32.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:32 smithi003 ceph-mon[97195]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:12:32.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:32 smithi103 ceph-mon[130434]: pgmap v605: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:12:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.895+0000 7f0afb7fe700 1 -- 172.21.15.3:0/3697220973 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e224fd17c0 con 0x55e2250eff00 2023-08-11T14:12:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.895+0000 7f0afb7fe700 10 client.16248 handle_command_reply: tid=0 2023-08-11T14:12:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.895+0000 7f0b36357b80 1 client.16248 shutdown 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 msgr2=0x55e225094d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e225094d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00a010 tx=0x7f0b0c006110 comp rx=0 tx=0).stop 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 shutdown_connections 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e2250eff00 0x55e2250f2320 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e225087790 0x55e2250962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e225085f40 0x55e225094d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 --2- 172.21.15.3:0/3697220973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2250868a0 0x55e2250952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.896+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 >> 172.21.15.3:0/3697220973 conn(0x55e224fb38f0 msgr2=0x55e22508a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.898+0000 7f0afb7fe700 0 client.16248 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.899+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 shutdown_connections 2023-08-11T14:12:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.899+0000 7f0b36357b80 1 -- 172.21.15.3:0/3697220973 wait complete. 2023-08-11T14:12:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.902+0000 7f0b36357b80 20 client.16248 trim_cache size 0 max 16384 2023-08-11T14:12:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:12:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:12:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:12:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:12:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.903+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 msgr2=0x7f0b040a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.903+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 0x7f0b040a2610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009fd0 tx=0x7f0b140098d0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.903+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 msgr2=0x7f0b1c115ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.903+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b780 tx=0x7f0b1400bb40 comp rx=0 tx=0).stop 2023-08-11T14:12:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 shutdown_connections 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1c115600 0x7f0b1c115ab0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f0b040a0150 0x7f0b040a2610 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b1c10a290 0x7f0b1c1141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 --2- 172.21.15.3:0/2453814506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b1c10cc60 0x7f0b1c114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.904+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 >> 172.21.15.3:0/2453814506 conn(0x7f0b1c06d750 msgr2=0x7f0b1c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.905+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 shutdown_connections 2023-08-11T14:12:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:32.905+0000 7f0b229ff700 1 -- 172.21.15.3:0/2453814506 wait complete. 2023-08-11T14:12:33.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 335.0 s, 762.9 tps, lat 131.089 ms stddev 345.644 2023-08-11T14:12:34.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:34 smithi003 ceph-mon[97195]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.37k op/s 2023-08-11T14:12:34.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:34 smithi187 ceph-mon[130973]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.37k op/s 2023-08-11T14:12:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:34 smithi103 ceph-mon[130434]: pgmap v606: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 21 MiB/s wr, 1.37k op/s 2023-08-11T14:12:35.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:35 smithi103 ceph-mon[130434]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-08-11T14:12:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:35 smithi003 ceph-mon[97195]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-08-11T14:12:35.801 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:35 smithi187 ceph-mon[130973]: pgmap v607: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 1.41k op/s 2023-08-11T14:12:38.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:37 smithi187 ceph-mon[130973]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-08-11T14:12:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:37 smithi003 ceph-mon[97195]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-08-11T14:12:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 340.0 s, 804.7 tps, lat 124.329 ms stddev 297.185 2023-08-11T14:12:38.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:37 smithi103 ceph-mon[130434]: pgmap v608: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 22 MiB/s wr, 1.67k op/s 2023-08-11T14:12:39.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:38 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:39.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:38 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:39.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:38 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:40.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:40 smithi003 ceph-mon[97195]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-08-11T14:12:40.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:40 smithi187 ceph-mon[130973]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-08-11T14:12:40.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:40 smithi103 ceph-mon[130434]: pgmap v609: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-08-11T14:12:42.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:42 smithi103 ceph-mon[130434]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 19 MiB/s wr, 1.34k op/s 2023-08-11T14:12:42.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:42 smithi003 ceph-mon[97195]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 19 MiB/s wr, 1.34k op/s 2023-08-11T14:12:42.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:42 smithi187 ceph-mon[130973]: pgmap v610: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 19 MiB/s wr, 1.34k op/s 2023-08-11T14:12:43.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 345.0 s, 348.4 tps, lat 275.328 ms stddev 531.714 2023-08-11T14:12:44.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:44 smithi187 ceph-mon[130973]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.58k op/s 2023-08-11T14:12:44.561 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:44 smithi003 ceph-mon[97195]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.58k op/s 2023-08-11T14:12:44.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:44 smithi103 ceph-mon[130434]: pgmap v611: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.58k op/s 2023-08-11T14:12:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.518+0000 7fac569ff700 1 -- 172.21.15.3:0/241859928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac5010cc60 msgr2=0x7fac5010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.518+0000 7fac569ff700 1 --2- 172.21.15.3:0/241859928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac5010cc60 0x7fac5010d0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac4801c0d0 tx=0x7fac4801c3e0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.528+0000 7fac569ff700 1 -- 172.21.15.3:0/241859928 shutdown_connections 2023-08-11T14:12:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.528+0000 7fac569ff700 1 --2- 172.21.15.3:0/241859928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac5010cc60 0x7fac5010d0e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.528+0000 7fac569ff700 1 --2- 172.21.15.3:0/241859928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac50107920 0x7fac50109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.528+0000 7fac569ff700 1 --2- 172.21.15.3:0/241859928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 0x7fac5010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.528+0000 7fac569ff700 1 -- 172.21.15.3:0/241859928 >> 172.21.15.3:0/241859928 conn(0x7fac5006d750 msgr2=0x7fac5006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:46.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:46 smithi187 ceph-mon[130973]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:12:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.540+0000 7fac569ff700 1 -- 172.21.15.3:0/241859928 shutdown_connections 2023-08-11T14:12:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.542+0000 7fac569ff700 1 -- 172.21.15.3:0/241859928 wait complete. 2023-08-11T14:12:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.543+0000 7fac569ff700 1 Processor -- start 2023-08-11T14:12:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 -- start start 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac50107920 0x7fac5019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 0x7fac5019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5019e850 con 0x7fac5010a290 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5019e9c0 con 0x7fac50107920 2023-08-11T14:12:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.544+0000 7fac569ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5019eb80 con 0x7fac5010cc60 2023-08-11T14:12:46.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:46 smithi003 ceph-mon[97195]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:12:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.548+0000 7fac54f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.548+0000 7fac54f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44274/0 (socket says 172.21.15.3:44274) 2023-08-11T14:12:46.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.548+0000 7fac54f9c700 1 -- 172.21.15.3:0/524056938 learned_addr learned my addr 172.21.15.3:0/524056938 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.553+0000 7fac4ffff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac50107920 0x7fac5019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.557+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 0x7fac5019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.563+0000 7fac54f9c700 1 -- 172.21.15.3:0/524056938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac50107920 msgr2=0x7fac5019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.563+0000 7fac54f9c700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac50107920 0x7fac5019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.563+0000 7fac54f9c700 1 -- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 msgr2=0x7fac5019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.563+0000 7fac54f9c700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 0x7fac5019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.563+0000 7fac54f9c700 1 -- 172.21.15.3:0/524056938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4801ad10 con 0x7fac5010cc60 2023-08-11T14:12:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.579+0000 7fac54f9c700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac48018f20 tx=0x7fac48003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.581+0000 7fac4d7fa700 1 -- 172.21.15.3:0/524056938 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4801f030 con 0x7fac5010cc60 2023-08-11T14:12:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.581+0000 7fac4d7fa700 1 -- 172.21.15.3:0/524056938 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac48005980 con 0x7fac5010cc60 2023-08-11T14:12:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.581+0000 7fac4d7fa700 1 -- 172.21.15.3:0/524056938 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4802be30 con 0x7fac5010cc60 2023-08-11T14:12:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.582+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5019ee00 con 0x7fac5010cc60 2023-08-11T14:12:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.582+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac50078d60 con 0x7fac5010cc60 2023-08-11T14:12:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.589+0000 7fac6a30bb80 1 Processor -- start 2023-08-11T14:12:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.601+0000 7fac6a30bb80 1 -- start start 2023-08-11T14:12:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d3e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864c965d0 con 0x55a864d3d8a0 2023-08-11T14:12:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864c93ef0 con 0x55a864d3cf40 2023-08-11T14:12:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864c8f7e0 con 0x55a864d3e790 2023-08-11T14:12:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.602+0000 7fac4d7fa700 1 -- 172.21.15.3:0/524056938 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fac480054c0 con 0x7fac5010cc60 2023-08-11T14:12:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.603+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.603+0000 7fac4f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40544/0 (socket says 172.21.15.3:40544) 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.603+0000 7fac4f7fe700 1 -- 172.21.15.3:0/1206414585 learned_addr learned my addr 172.21.15.3:0/1206414585 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.603+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 -- 172.21.15.3:0/1206414585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 msgr2=0x55a864d3ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d3ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 -- 172.21.15.3:0/1206414585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 msgr2=0x55a864d3e1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:12:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d3e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 -- 172.21.15.3:0/1206414585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a864b0f5d0 con 0x55a864d3cf40 2023-08-11T14:12:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac44006b30 tx=0x7fac44007220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b2ffa1888fdee5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.604+0000 7fac4d7fa700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 0x7fac380a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.605+0000 7fac4d7fa700 1 -- 172.21.15.3:0/524056938 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fac48020030 con 0x7fac5010cc60 2023-08-11T14:12:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.614+0000 7fac367fc700 1 -- 172.21.15.3:0/1206414585 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44018420 con 0x55a864d3cf40 2023-08-11T14:12:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.614+0000 7fac367fc700 1 -- 172.21.15.3:0/1206414585 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac44018a40 con 0x55a864d3cf40 2023-08-11T14:12:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.614+0000 7fac367fc700 1 -- 172.21.15.3:0/1206414585 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44017820 con 0x55a864d3cf40 2023-08-11T14:12:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.615+0000 7fac6a30bb80 1 -- 172.21.15.3:0/1206414585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 msgr2=0x55a864d3d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.615+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac44006b30 tx=0x7fac44007220 comp rx=0 tx=0).stop 2023-08-11T14:12:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.619+0000 7fac6a30bb80 1 -- 172.21.15.3:0/1206414585 shutdown_connections 2023-08-11T14:12:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.619+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d3ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.619+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d3d360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.619+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/1206414585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d3e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.619+0000 7fac6a30bb80 1 -- 172.21.15.3:0/1206414585 >> 172.21.15.3:0/1206414585 conn(0x55a864c6a8f0 msgr2=0x55a864d39920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.624+0000 7fac6a30bb80 1 -- 172.21.15.3:0/1206414585 shutdown_connections 2023-08-11T14:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.633+0000 7fac4ffff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 0x7fac380a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.634+0000 7fac4ffff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 0x7fac380a2610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a864d392f0 tx=0x7fac40008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.634+0000 7fac6a30bb80 1 -- 172.21.15.3:0/1206414585 wait complete. 2023-08-11T14:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.635+0000 7fac6a30bb80 1 -- start start 2023-08-11T14:12:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.635+0000 7fac6a30bb80 1 Processor -- start 2023-08-11T14:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d4e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864c2da60 con 0x55a864d3d8a0 2023-08-11T14:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864c2d760 con 0x55a864d3cf40 2023-08-11T14:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864b118c0 con 0x55a864d3e790 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.639+0000 7fac6a30bb80 10 client.0 _ll_register_callbacks cb 0x55a864c30a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.640+0000 7fac4ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.640+0000 7fac4ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40552/0 (socket says 172.21.15.3:40552) 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.640+0000 7fac4ffff700 1 -- 172.21.15.3:0/3065737315 learned_addr learned my addr 172.21.15.3:0/3065737315 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.640+0000 7fac54f9c700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d4e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 -- 172.21.15.3:0/3065737315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 msgr2=0x55a864d4e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d4e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 -- 172.21.15.3:0/3065737315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 msgr2=0x55a864d471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 -- 172.21.15.3:0/3065737315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac44016310 con 0x55a864d3cf40 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.641+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac4000b470 tx=0x7fac400047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6501a7308b09d55f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.642+0000 7fac2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.642+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40014500 con 0x55a864d3cf40 2023-08-11T14:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.642+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fac40012e10 con 0x55a864d3cf40 2023-08-11T14:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.642+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a864d307e0 con 0x55a864d3cf40 2023-08-11T14:12:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.650+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac40014e40 con 0x55a864d3cf40 2023-08-11T14:12:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.654+0000 7fac6a30bb80 10 client.26375 fetch_fsmap learned FSMap version 38 2023-08-11T14:12:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.654+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a864abbd80 con 0x55a864d3cf40 2023-08-11T14:12:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a864d307e0 con 0x55a864d3cf40 2023-08-11T14:12:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fac40027070 con 0x55a864d3cf40 2023-08-11T14:12:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 38 2023-08-11T14:12:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 20 client.26375 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864d52b80 0x55a864d54fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 4 client.26375 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.659+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a864d0c480 con 0x55a864d52b80 2023-08-11T14:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.660+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864d52b80 0x55a864d54fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.662+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864d52b80 0x55a864d54fa0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f638186060b43c17 server_cookie=ba035b747cc7d92b in_seq=0 out_seq=0 2023-08-11T14:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.662+0000 7fac2f7fe700 10 client.26375 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.668+0000 7fac2f7fe700 1 -- 172.21.15.3:0/3065737315 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a864d0c480 con 0x55a864d52b80 2023-08-11T14:12:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.668+0000 7fac2f7fe700 10 client.26375 handle_command_reply: tid=0 2023-08-11T14:12:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.669+0000 7fac6a30bb80 1 client.26375 shutdown 2023-08-11T14:12:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.669+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 msgr2=0x55a864d46ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.669+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac4000b470 tx=0x7fac400047e0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 shutdown_connections 2023-08-11T14:12:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864d4e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864d52b80 0x55a864d54fa0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3cf40 0x55a864d46ca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3065737315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 >> 172.21.15.3:0/3065737315 conn(0x55a864c6a8f0 msgr2=0x55a864d3aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac2f7fe700 0 client.26375 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.670+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 shutdown_connections 2023-08-11T14:12:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.671+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3065737315 wait complete. 2023-08-11T14:12:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.671+0000 7fac6a30bb80 20 client.26375 trim_cache size 0 max 16384 2023-08-11T14:12:46.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:46 smithi103 ceph-mon[130434]: pgmap v612: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:12:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.706+0000 7fac6a30bb80 1 Processor -- start 2023-08-11T14:12:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 -- start start 2023-08-11T14:12:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d4e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3e790 0x55a864d39860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864d38d50 con 0x55a864d3d8a0 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864d4fee0 con 0x55a864d3e790 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.707+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864d38f50 con 0x55a864d3cf40 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.708+0000 7fac4ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.708+0000 7fac4ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44296/0 (socket says 172.21.15.3:44296) 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.708+0000 7fac4ffff700 1 -- 172.21.15.3:0/3433068746 learned_addr learned my addr 172.21.15.3:0/3433068746 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.708+0000 7fac54f9c700 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3e790 0x55a864d39860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 -- 172.21.15.3:0/3433068746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3e790 msgr2=0x55a864d39860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3e790 0x55a864d39860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 -- 172.21.15.3:0/3433068746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 msgr2=0x55a864d4e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d4e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 -- 172.21.15.3:0/3433068746 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a864aa4c00 con 0x55a864d3cf40 2023-08-11T14:12:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.709+0000 7fac4ffff700 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac40014040 tx=0x7fac40028a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb588c2bb782127 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.710+0000 7fac36ffd700 1 -- 172.21.15.3:0/3433068746 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40012e10 con 0x55a864d3cf40 2023-08-11T14:12:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.710+0000 7fac36ffd700 1 -- 172.21.15.3:0/3433068746 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac40028da0 con 0x55a864d3cf40 2023-08-11T14:12:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.710+0000 7fac36ffd700 1 -- 172.21.15.3:0/3433068746 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40006040 con 0x55a864d3cf40 2023-08-11T14:12:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.711+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3433068746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 msgr2=0x55a864d39320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.711+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac40014040 tx=0x7fac40028a00 comp rx=0 tx=0).stop 2023-08-11T14:12:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.714+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3433068746 shutdown_connections 2023-08-11T14:12:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.715+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3e790 0x55a864d39860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.715+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3cf40 0x55a864d39320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.715+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/3433068746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3d8a0 0x55a864d4e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.715+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3433068746 >> 172.21.15.3:0/3433068746 conn(0x55a864c6a8f0 msgr2=0x55a864d416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.717+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3433068746 shutdown_connections 2023-08-11T14:12:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.717+0000 7fac6a30bb80 1 -- 172.21.15.3:0/3433068746 wait complete. 2023-08-11T14:12:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.718+0000 7fac6a30bb80 1 -- start start 2023-08-11T14:12:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.718+0000 7fac6a30bb80 1 Processor -- start 2023-08-11T14:12:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3cf40 0x55a864d9ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3d8a0 0x55a864d9efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864da5ff0 con 0x55a864d3cf40 2023-08-11T14:12:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864da6130 con 0x55a864d3d8a0 2023-08-11T14:12:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a864da62f0 con 0x55a864d3e790 2023-08-11T14:12:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.720+0000 7fac6a30bb80 10 client.0 _ll_register_callbacks cb 0x7fac40014660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:12:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.721+0000 7fac54f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.721+0000 7fac54f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:44304/0 (socket says 172.21.15.3:44304) 2023-08-11T14:12:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.721+0000 7fac54f9c700 1 -- 172.21.15.3:0/2921767851 learned_addr learned my addr 172.21.15.3:0/2921767851 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:12:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac4f7fe700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3d8a0 0x55a864d9efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac54f9c700 1 -- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3d8a0 msgr2=0x55a864d9efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac54f9c700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3d8a0 0x55a864d9efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac54f9c700 1 -- 172.21.15.3:0/2921767851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3cf40 msgr2=0x55a864d9ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:12:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac54f9c700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3cf40 0x55a864d9ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.727+0000 7fac54f9c700 1 -- 172.21.15.3:0/2921767851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a864abbd80 con 0x55a864d3e790 2023-08-11T14:12:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.728+0000 7fac54f9c700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fac480aad50 tx=0x7fac4801a9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49bf665042daea5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:12:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.731+0000 7fac2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:12:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.737+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480b2b40 con 0x55a864d3e790 2023-08-11T14:12:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.737+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fac480ab020 con 0x55a864d3e790 2023-08-11T14:12:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.739+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac44018a40 con 0x55a864d3e790 2023-08-11T14:12:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.744+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac480d4ea0 con 0x55a864d3e790 2023-08-11T14:12:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.746+0000 7fac6a30bb80 10 client.26392 fetch_fsmap learned FSMap version 38 2023-08-11T14:12:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.746+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a864ae21f0 con 0x55a864d3e790 2023-08-11T14:12:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac480d8020 con 0x55a864d3e790 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 38) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fac480b26b0 con 0x55a864d3e790 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 10 client.26392 fetch_fsmap finished waiting for FSMap version 38 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 20 client.26392 populate_metadata read hostname 'smithi003' 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864da7b50 0x55a864da9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:12:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 4 client.26392 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:12:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.756+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a864c887c0 con 0x55a864da7b50 2023-08-11T14:12:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.764+0000 7fac4ffff700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864da7b50 0x55a864da9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:12:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.789+0000 7fac4ffff700 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864da7b50 0x55a864da9f70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6004f8964ea2cd server_cookie=f4acc0a5a0882d73 in_seq=0 out_seq=0 2023-08-11T14:12:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.789+0000 7fac2f7fe700 10 client.26392 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.975+0000 7fac2f7fe700 1 -- 172.21.15.3:0/2921767851 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a864c887c0 con 0x55a864da7b50 2023-08-11T14:12:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.975+0000 7fac2f7fe700 10 client.26392 handle_command_reply: tid=0 2023-08-11T14:12:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.976+0000 7fac6a30bb80 1 client.26392 shutdown 2023-08-11T14:12:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.976+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 msgr2=0x55a864da5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.976+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fac480aad50 tx=0x7fac4801a9f0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 shutdown_connections 2023-08-11T14:12:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a864d3e790 0x55a864da5ab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a864d3cf40 0x55a864d9ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a864da7b50 0x55a864da9f70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 --2- 172.21.15.3:0/2921767851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a864d3d8a0 0x55a864d9efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.977+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 >> 172.21.15.3:0/2921767851 conn(0x55a864c6a8f0 msgr2=0x55a864d39320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.978+0000 7fac2f7fe700 0 client.26392 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:12:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.978+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 shutdown_connections 2023-08-11T14:12:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.978+0000 7fac6a30bb80 1 -- 172.21.15.3:0/2921767851 wait complete. 2023-08-11T14:12:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.984+0000 7fac6a30bb80 20 client.26392 trim_cache size 0 max 16384 2023-08-11T14:12:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:12:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:12:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:12:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:12:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.996+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 msgr2=0x7fac380a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.996+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 0x7fac380a2610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a864d392f0 tx=0x7fac40008040 comp rx=0 tx=0).stop 2023-08-11T14:12:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.996+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 msgr2=0x7fac5019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:12:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:46.996+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac48018f20 tx=0x7fac48003ab0 comp rx=0 tx=0).stop 2023-08-11T14:12:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 shutdown_connections 2023-08-11T14:12:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac5010cc60 0x7fac5019e150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fac380a0150 0x7fac380a2610 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac50107920 0x7fac5019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 --2- 172.21.15.3:0/524056938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac5010a290 0x7fac5019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:12:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.006+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 >> 172.21.15.3:0/524056938 conn(0x7fac5006d750 msgr2=0x7fac5010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:12:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.014+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 shutdown_connections 2023-08-11T14:12:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:12:47.017+0000 7fac569ff700 1 -- 172.21.15.3:0/524056938 wait complete. 2023-08-11T14:12:48.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 350.0 s, 625.6 tps, lat 164.162 ms stddev 179.155 2023-08-11T14:12:48.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:48 smithi003 ceph-mon[97195]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.35k op/s 2023-08-11T14:12:48.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:48 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:48.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:48 smithi187 ceph-mon[130973]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.35k op/s 2023-08-11T14:12:48.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:48 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:48.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:48 smithi103 ceph-mon[130434]: pgmap v613: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 16 MiB/s wr, 1.35k op/s 2023-08-11T14:12:48.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:48 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:12:49.683 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:49 smithi103 ceph-mon[130434]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:12:49.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:49 smithi003 ceph-mon[97195]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:12:49.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:49 smithi187 ceph-mon[130973]: pgmap v614: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:12:52.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:51 smithi103 ceph-mon[130434]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:12:52.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:51 smithi003 ceph-mon[97195]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:12:52.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:51 smithi187 ceph-mon[130973]: pgmap v615: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:12:53.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 355.0 s, 802.6 tps, lat 127.274 ms stddev 171.646 2023-08-11T14:12:54.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:54 smithi187 ceph-mon[130973]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:12:54.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:54 smithi003 ceph-mon[97195]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:12:54.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:54 smithi103 ceph-mon[130434]: pgmap v616: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:12:56.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:56 smithi103 ceph-mon[130434]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 912 op/s 2023-08-11T14:12:56.551 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:56 smithi187 ceph-mon[130973]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 912 op/s 2023-08-11T14:12:56.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:56 smithi003 ceph-mon[97195]: pgmap v617: 129 pgs: 129 active+clean; 8.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 912 op/s 2023-08-11T14:12:58.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 360.0 s, 639.2 tps, lat 135.335 ms stddev 377.623 2023-08-11T14:12:58.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:12:58 smithi103 ceph-mon[130434]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:12:58.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:12:58 smithi003 ceph-mon[97195]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:12:58.547 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:12:58 smithi187 ceph-mon[130973]: pgmap v618: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:13:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.344+0000 7fc88b7ff700 1 -- 172.21.15.3:0/101297542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc884107920 msgr2=0x7fc884109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.344+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/101297542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc884107920 0x7fc884109d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009f90 tx=0x7fc87c00b330 comp rx=0 tx=0).stop 2023-08-11T14:13:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.358+0000 7fc88b7ff700 1 -- 172.21.15.3:0/101297542 shutdown_connections 2023-08-11T14:13:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.358+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/101297542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc88410cc60 0x7fc88410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.358+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/101297542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc884107920 0x7fc884109d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.358+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/101297542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410a290 0x7fc88410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.358+0000 7fc88b7ff700 1 -- 172.21.15.3:0/101297542 >> 172.21.15.3:0/101297542 conn(0x7fc88406d750 msgr2=0x7fc88406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.369+0000 7fc88b7ff700 1 -- 172.21.15.3:0/101297542 shutdown_connections 2023-08-11T14:13:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.383+0000 7fc88b7ff700 1 -- 172.21.15.3:0/101297542 wait complete. 2023-08-11T14:13:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.385+0000 7fc88b7ff700 1 Processor -- start 2023-08-11T14:13:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.386+0000 7fc88b7ff700 1 -- start start 2023-08-11T14:13:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc884107920 0x7fc884113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc88410a290 0x7fc884114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884115950 con 0x7fc88410cc60 2023-08-11T14:13:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884115ac0 con 0x7fc88410a290 2023-08-11T14:13:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.387+0000 7fc88b7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884115c80 con 0x7fc884107920 2023-08-11T14:13:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.388+0000 7fc88959b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc884107920 0x7fc884113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.389+0000 7fc889d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.389+0000 7fc889d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34216/0 (socket says 172.21.15.3:34216) 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.389+0000 7fc889d9c700 1 -- 172.21.15.3:0/2276390686 learned_addr learned my addr 172.21.15.3:0/2276390686 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 -- 172.21.15.3:0/2276390686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc884107920 msgr2=0x7fc884113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc884107920 0x7fc884113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 -- 172.21.15.3:0/2276390686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc88410a290 msgr2=0x7fc884114420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:13:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc88410a290 0x7fc884114420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 -- 172.21.15.3:0/2276390686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c009c40 con 0x7fc88410cc60 2023-08-11T14:13:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.390+0000 7fc889d9c700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc88000d980 tx=0x7fc88000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.395+0000 7fc87a7fc700 1 -- 172.21.15.3:0/2276390686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880009b40 con 0x7fc88410cc60 2023-08-11T14:13:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.395+0000 7fc87a7fc700 1 -- 172.21.15.3:0/2276390686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc88000be40 con 0x7fc88410cc60 2023-08-11T14:13:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.395+0000 7fc87a7fc700 1 -- 172.21.15.3:0/2276390686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8800054e0 con 0x7fc88410cc60 2023-08-11T14:13:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.398+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc884115f60 con 0x7fc88410cc60 2023-08-11T14:13:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.398+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc884116710 con 0x7fc88410cc60 2023-08-11T14:13:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.404+0000 7fc87a7fc700 1 -- 172.21.15.3:0/2276390686 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fc880004750 con 0x7fc88410cc60 2023-08-11T14:13:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.406+0000 7fc87a7fc700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 0x7fc8700a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.407+0000 7fc87a7fc700 1 -- 172.21.15.3:0/2276390686 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc8800cb420 con 0x7fc88410cc60 2023-08-11T14:13:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.411+0000 7fc88959b700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 0x7fc8700a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.412+0000 7fc88959b700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 0x7fc8700a2610 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007ab0 tx=0x7fc87c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.417+0000 7fc89f13eb80 1 Processor -- start 2023-08-11T14:13:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.417+0000 7fc89f13eb80 1 -- start start 2023-08-11T14:13:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed1065d0 con 0x55abed1ad8a0 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed103ef0 con 0x55abed1ae790 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.418+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed0ff7e0 con 0x55abed1acf40 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.419+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.419+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34224/0 (socket says 172.21.15.3:34224) 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.419+0000 7fc888d9a700 1 -- 172.21.15.3:0/1300713807 learned_addr learned my addr 172.21.15.3:0/1300713807 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 -- 172.21.15.3:0/1300713807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 msgr2=0x55abed1ad360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:13:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1ad360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 -- 172.21.15.3:0/1300713807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 msgr2=0x55abed1aec10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:13:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1aec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 -- 172.21.15.3:0/1300713807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abecf7f5d0 con 0x55abed1ad8a0 2023-08-11T14:13:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.420+0000 7fc888d9a700 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc87400bae0 tx=0x7fc87400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ba2b3af6093b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.424+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1300713807 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400da90 con 0x55abed1ad8a0 2023-08-11T14:13:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.424+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1300713807 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc874004cf0 con 0x55abed1ad8a0 2023-08-11T14:13:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.424+0000 7fc86f7fe700 1 -- 172.21.15.3:0/1300713807 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874005610 con 0x55abed1ad8a0 2023-08-11T14:13:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.425+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1300713807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 msgr2=0x55abed1ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.425+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc87400bae0 tx=0x7fc87400bdf0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.429+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1300713807 shutdown_connections 2023-08-11T14:13:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.429+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.429+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.429+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1300713807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1ad8a0 0x55abed1ae1b0 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc87400bae0 tx=0x7fc87400bdf0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.429+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1300713807 >> 172.21.15.3:0/1300713807 conn(0x55abed0da8f0 msgr2=0x55abed1a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.450+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1300713807 shutdown_connections 2023-08-11T14:13:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.450+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1300713807 wait complete. 2023-08-11T14:13:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.451+0000 7fc89f13eb80 1 -- start start 2023-08-11T14:13:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.451+0000 7fc89f13eb80 1 Processor -- start 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed1be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed09da60 con 0x55abed1bc4a0 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed09d760 con 0x55abed1ae790 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abecf818c0 con 0x55abed1acf40 2023-08-11T14:13:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.452+0000 7fc89f13eb80 10 client.0 _ll_register_callbacks cb 0x55abed0a0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.459+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.459+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:56798/0 (socket says 172.21.15.3:56798) 2023-08-11T14:13:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.459+0000 7fc888d9a700 1 -- 172.21.15.3:0/1488810416 learned_addr learned my addr 172.21.15.3:0/1488810416 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.459+0000 7fc88959b700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc889d9c700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed1be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc88959b700 1 -- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 msgr2=0x55abed1b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc88959b700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc88959b700 1 -- 172.21.15.3:0/1488810416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 msgr2=0x55abed1be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc88959b700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed1be890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.463+0000 7fc88959b700 1 -- 172.21.15.3:0/1488810416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87400b740 con 0x55abed1acf40 2023-08-11T14:13:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.475+0000 7fc88959b700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1b6f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009550 tx=0x7fc87c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa3fffcb169fc6f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.476+0000 7fc86cff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:13:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.484+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00e040 con 0x55abed1acf40 2023-08-11T14:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.484+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc87c022050 con 0x55abed1acf40 2023-08-11T14:13:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.484+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abed1a07e0 con 0x55abed1acf40 2023-08-11T14:13:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.494+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc87c01da30 con 0x55abed1acf40 2023-08-11T14:13:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.495+0000 7fc89f13eb80 10 client.26398 fetch_fsmap learned FSMap version 39 2023-08-11T14:13:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.495+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55abecf2bd80 con 0x55abed1acf40 2023-08-11T14:13:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.496+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abed1a07e0 con 0x55abed1acf40 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.496+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc87c027420 con 0x55abed1acf40 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 10 client.26398 fetch_fsmap finished waiting for FSMap version 39 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 20 client.26398 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed1c30f0 0x55abed1c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 4 client.26398 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.503+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abed17c480 con 0x55abed1c30f0 2023-08-11T14:13:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.511+0000 7fc888d9a700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed1c30f0 0x55abed1c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:00 smithi187 ceph-mon[130973]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-08-11T14:13:00.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:00 smithi003 ceph-mon[97195]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-08-11T14:13:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.542+0000 7fc888d9a700 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed1c30f0 0x55abed1c5510 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675fb0e4592f802f server_cookie=4affded49cfe7a74 in_seq=0 out_seq=0 2023-08-11T14:13:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.544+0000 7fc86cff9700 10 client.26398 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.560+0000 7fc86cff9700 1 -- 172.21.15.3:0/1488810416 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55abed17c480 con 0x55abed1c30f0 2023-08-11T14:13:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.560+0000 7fc86cff9700 10 client.26398 handle_command_reply: tid=0 2023-08-11T14:13:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.562+0000 7fc89f13eb80 1 client.26398 shutdown 2023-08-11T14:13:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 msgr2=0x55abed1b6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1b6f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009550 tx=0x7fc87c009920 comp rx=0 tx=0).stop 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 shutdown_connections 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed1be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1acf40 0x55abed1b6f30 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009550 tx=0x7fc87c009920 comp rx=0 tx=0).stop 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed1c30f0 0x55abed1c5510 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1488810416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 >> 172.21.15.3:0/1488810416 conn(0x55abed0da8f0 msgr2=0x55abed1a8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc86cff9700 0 client.26398 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.571+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 shutdown_connections 2023-08-11T14:13:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.572+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1488810416 wait complete. 2023-08-11T14:13:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.572+0000 7fc89f13eb80 20 client.26398 trim_cache size 0 max 16384 2023-08-11T14:13:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 Processor -- start 2023-08-11T14:13:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 -- start start 2023-08-11T14:13:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 0x55abed1b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1b5ed0 0x55abed1b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed1b6b00 con 0x55abed1b5ed0 2023-08-11T14:13:00.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed1a07e0 con 0x55abed1ae790 2023-08-11T14:13:00.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.607+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed1ca310 con 0x55abed1bc4a0 2023-08-11T14:13:00.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.612+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 0x55abed1b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.612+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 0x55abed1b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:37760/0 (socket says 172.21.15.3:37760) 2023-08-11T14:13:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.612+0000 7fc888d9a700 1 -- 172.21.15.3:0/1053398599 learned_addr learned my addr 172.21.15.3:0/1053398599 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.624+0000 7fc88959b700 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.628+0000 7fc88959b700 1 -- 172.21.15.3:0/1053398599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 msgr2=0x55abed1b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.628+0000 7fc88959b700 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 0x55abed1b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.628+0000 7fc88959b700 1 -- 172.21.15.3:0/1053398599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1b5ed0 msgr2=0x55abed1b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.628+0000 7fc88959b700 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1b5ed0 0x55abed1b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.628+0000 7fc88959b700 1 -- 172.21.15.3:0/1053398599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abed1a4a10 con 0x55abed1ae790 2023-08-11T14:13:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.650+0000 7fc88959b700 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1d2b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc87c026000 tx=0x7fc87c00bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1931121ae473684c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.670+0000 7fc86ffff700 1 -- 172.21.15.3:0/1053398599 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00e070 con 0x55abed1ae790 2023-08-11T14:13:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.670+0000 7fc86ffff700 1 -- 172.21.15.3:0/1053398599 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc87c027420 con 0x55abed1ae790 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.670+0000 7fc86ffff700 1 -- 172.21.15.3:0/1053398599 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c01b680 con 0x55abed1ae790 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.671+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1053398599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 msgr2=0x55abed1d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.671+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1d2b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc87c026000 tx=0x7fc87c00bc00 comp rx=0 tx=0).stop 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.672+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1053398599 shutdown_connections 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.672+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1b5ed0 0x55abed1b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.672+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1ae790 0x55abed1d2b70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.672+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/1053398599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1bc4a0 0x55abed1b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.672+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1053398599 >> 172.21.15.3:0/1053398599 conn(0x55abed0da8f0 msgr2=0x55abed1b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:00.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:00 smithi103 ceph-mon[130434]: pgmap v619: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.23k op/s 2023-08-11T14:13:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.689+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1053398599 shutdown_connections 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.689+0000 7fc89f13eb80 1 -- 172.21.15.3:0/1053398599 wait complete. 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.690+0000 7fc89f13eb80 1 -- start start 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.690+0000 7fc89f13eb80 1 Processor -- start 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 0x55abed1cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 0x55abed1d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed215400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed215940 con 0x55abed1bc4a0 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed215a80 con 0x55abed1b5ed0 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abed215bc0 con 0x55abed1ae790 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.691+0000 7fc89f13eb80 10 client.0 _ll_register_callbacks cb 0x7fc87c004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.692+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 0x55abed1d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.692+0000 7fc888d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 0x55abed1d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:56818/0 (socket says 172.21.15.3:56818) 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.692+0000 7fc888d9a700 1 -- 172.21.15.3:0/2617175043 learned_addr learned my addr 172.21.15.3:0/2617175043 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.692+0000 7fc88959b700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 0x55abed1cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc889d9c700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed215400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc88959b700 1 -- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 msgr2=0x55abed1d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc88959b700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 0x55abed1d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc88959b700 1 -- 172.21.15.3:0/2617175043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 msgr2=0x55abed215400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc88959b700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed215400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.699+0000 7fc88959b700 1 -- 172.21.15.3:0/2617175043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abecf2bd80 con 0x55abed1ae790 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.700+0000 7fc88959b700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 0x55abed1cfdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55abed1c59b0 tx=0x7fc87c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51b4c6d09fff842d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.700+0000 7fc86cff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.701+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00e040 con 0x55abed1ae790 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.701+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fc87c022040 con 0x55abed1ae790 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.701+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc874004cf0 con 0x55abed1ae790 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.705+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc87c027590 con 0x55abed1ae790 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.705+0000 7fc89f13eb80 10 client.26404 fetch_fsmap learned FSMap version 39 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.705+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55abecf14c00 con 0x55abed1ae790 2023-08-11T14:13:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc874004cf0 con 0x55abed1ae790 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc87c043970 con 0x55abed1ae790 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc89f13eb80 10 client.26404 fetch_fsmap finished waiting for FSMap version 39 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc89f13eb80 10 client.26404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc89f13eb80 20 client.26404 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.713+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed2170f0 0x55abed2194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.714+0000 7fc89f13eb80 4 client.26404 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:13:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.714+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55abed0f87c0 con 0x55abed2170f0 2023-08-11T14:13:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.716+0000 7fc888d9a700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed2170f0 0x55abed2194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.737+0000 7fc888d9a700 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed2170f0 0x55abed2194e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebcdc63bf5efc55b server_cookie=309d0809e882260e in_seq=0 out_seq=0 2023-08-11T14:13:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:00.742+0000 7fc86cff9700 10 client.26404 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.073+0000 7fc86cff9700 1 -- 172.21.15.3:0/2617175043 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55abed0f87c0 con 0x55abed2170f0 2023-08-11T14:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.073+0000 7fc86cff9700 10 client.26404 handle_command_reply: tid=0 2023-08-11T14:13:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.073+0000 7fc89f13eb80 1 client.26404 shutdown 2023-08-11T14:13:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.086+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 msgr2=0x55abed1cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.086+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 0x55abed1cfdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55abed1c59b0 tx=0x7fc87c00b330 comp rx=0 tx=0).stop 2023-08-11T14:13:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 shutdown_connections 2023-08-11T14:13:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55abed1bc4a0 0x55abed215400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55abed1ae790 0x55abed1cfdf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55abed2170f0 0x55abed2194e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 --2- 172.21.15.3:0/2617175043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abed1b5ed0 0x55abed1d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 >> 172.21.15.3:0/2617175043 conn(0x55abed0da8f0 msgr2=0x55abed1b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.087+0000 7fc86cff9700 0 client.26404 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.095+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 shutdown_connections 2023-08-11T14:13:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.095+0000 7fc89f13eb80 1 -- 172.21.15.3:0/2617175043 wait complete. 2023-08-11T14:13:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.096+0000 7fc89f13eb80 20 client.26404 trim_cache size 0 max 16384 2023-08-11T14:13:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:13:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:13:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:13:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:13:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.099+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 msgr2=0x7fc8700a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.099+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 0x7fc8700a2610 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007ab0 tx=0x7fc87c000f40 comp rx=0 tx=0).stop 2023-08-11T14:13:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.099+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 msgr2=0x7fc8841152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.099+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc88000d980 tx=0x7fc88000dd40 comp rx=0 tx=0).stop 2023-08-11T14:13:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 shutdown_connections 2023-08-11T14:13:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc88410cc60 0x7fc8841152f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc884107920 0x7fc884113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fc8700a0150 0x7fc8700a2610 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 --2- 172.21.15.3:0/2276390686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc88410a290 0x7fc884114420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.111+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 >> 172.21.15.3:0/2276390686 conn(0x7fc88406d750 msgr2=0x7fc88410bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.117+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 shutdown_connections 2023-08-11T14:13:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:02.118+0000 7fc88b7ff700 1 -- 172.21.15.3:0/2276390686 wait complete. 2023-08-11T14:13:02.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:02 smithi003 ceph-mon[97195]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-08-11T14:13:02.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:02 smithi187 ceph-mon[130973]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-08-11T14:13:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:02 smithi103 ceph-mon[130434]: pgmap v620: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2023-08-11T14:13:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 365.0 s, 653.1 tps, lat 170.945 ms stddev 421.384 2023-08-11T14:13:04.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:04 smithi103 ceph-mon[130434]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-08-11T14:13:04.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:04 smithi003 ceph-mon[97195]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-08-11T14:13:04.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:04 smithi187 ceph-mon[130973]: pgmap v621: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 23 MiB/s wr, 1.56k op/s 2023-08-11T14:13:05.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:05 smithi187 ceph-mon[130973]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:13:05.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:05 smithi003 ceph-mon[97195]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:13:05.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:05 smithi103 ceph-mon[130434]: pgmap v622: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:13:08.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 370.0 s, 734.5 tps, lat 137.508 ms stddev 252.710 2023-08-11T14:13:08.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:07 smithi103 ceph-mon[130434]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-08-11T14:13:08.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:07 smithi003 ceph-mon[97195]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-08-11T14:13:08.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:07 smithi187 ceph-mon[130973]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 1.35k op/s 2023-08-11T14:13:10.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:10 smithi103 ceph-mon[130434]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.42k op/s 2023-08-11T14:13:10.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:10 smithi187 ceph-mon[130973]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.42k op/s 2023-08-11T14:13:10.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:10 smithi003 ceph-mon[97195]: pgmap v624: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 1.42k op/s 2023-08-11T14:13:12.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:12 smithi103 ceph-mon[130434]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:13:12.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:12 smithi003 ceph-mon[97195]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:13:12.550 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:12 smithi187 ceph-mon[130973]: pgmap v625: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:13:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 375.0 s, 761.9 tps, lat 131.544 ms stddev 293.192 2023-08-11T14:13:14.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:14 smithi103 ceph-mon[130434]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:13:14.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:14 smithi003 ceph-mon[97195]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:13:14.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:14 smithi187 ceph-mon[130973]: pgmap v626: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.51k op/s 2023-08-11T14:13:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 -- 172.21.15.3:0/774271780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10cc60 msgr2=0x7f36dc10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 --2- 172.21.15.3:0/774271780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10cc60 0x7f36dc10d0e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f36d401c0d0 tx=0x7f36d401c3e0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 -- 172.21.15.3:0/774271780 shutdown_connections 2023-08-11T14:13:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 --2- 172.21.15.3:0/774271780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10cc60 0x7f36dc10d0e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 --2- 172.21.15.3:0/774271780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 0x7f36dc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 --2- 172.21.15.3:0/774271780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10a290 0x7f36dc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.463+0000 7f36e33ff700 1 -- 172.21.15.3:0/774271780 >> 172.21.15.3:0/774271780 conn(0x7f36dc06d750 msgr2=0x7f36dc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.472+0000 7f36e33ff700 1 -- 172.21.15.3:0/774271780 shutdown_connections 2023-08-11T14:13:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.480+0000 7f36e33ff700 1 -- 172.21.15.3:0/774271780 wait complete. 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.481+0000 7f36e33ff700 1 Processor -- start 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 -- start start 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 0x7f36dc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10a290 0x7f36dc19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19e750 con 0x7f36dc107920 2023-08-11T14:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19e8c0 con 0x7f36dc10cc60 2023-08-11T14:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.483+0000 7f36e33ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc19ea80 con 0x7f36dc10a290 2023-08-11T14:13:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.489+0000 7f36e199c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.489+0000 7f36e099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10a290 0x7f36dc19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.489+0000 7f36e199c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39434/0 (socket says 172.21.15.3:39434) 2023-08-11T14:13:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.489+0000 7f36e199c700 1 -- 172.21.15.3:0/2228694110 learned_addr learned my addr 172.21.15.3:0/2228694110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.489+0000 7f36e119b700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 0x7f36dc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.496+0000 7f36e199c700 1 -- 172.21.15.3:0/2228694110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10a290 msgr2=0x7f36dc19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.496+0000 7f36e199c700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10a290 0x7f36dc19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.496+0000 7f36e199c700 1 -- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 msgr2=0x7f36dc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.496+0000 7f36e199c700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 0x7f36dc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.496+0000 7f36e199c700 1 -- 172.21.15.3:0/2228694110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d401ad10 con 0x7f36dc10cc60 2023-08-11T14:13:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.497+0000 7f36e199c700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36d4018bb0 tx=0x7f36d4018960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.511+0000 7f36d27fc700 1 -- 172.21.15.3:0/2228694110 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401f040 con 0x7f36dc10cc60 2023-08-11T14:13:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.511+0000 7f36d27fc700 1 -- 172.21.15.3:0/2228694110 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d4007960 con 0x7f36dc10cc60 2023-08-11T14:13:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.511+0000 7f36d27fc700 1 -- 172.21.15.3:0/2228694110 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d40055a0 con 0x7f36dc10cc60 2023-08-11T14:13:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.511+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc19ed00 con 0x7f36dc10cc60 2023-08-11T14:13:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.511+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc1c74b0 con 0x7f36dc10cc60 2023-08-11T14:13:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.518+0000 7f36f6dcab80 1 Processor -- start 2023-08-11T14:13:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.525+0000 7f36f6dcab80 1 -- start start 2023-08-11T14:13:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e27238a0 0x55a5e27241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e2724790 0x55a5e2724c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e267c5d0 con 0x55a5e2724790 2023-08-11T14:13:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2679ef0 con 0x55a5e2722f40 2023-08-11T14:13:15.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.526+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e26757e0 con 0x55a5e27238a0 2023-08-11T14:13:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.532+0000 7f36e099a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.534+0000 7f36e099a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:39436/0 (socket says 172.21.15.3:39436) 2023-08-11T14:13:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.534+0000 7f36e099a700 1 -- 172.21.15.3:0/3348580867 learned_addr learned my addr 172.21.15.3:0/3348580867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.538+0000 7f36e119b700 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e27238a0 0x55a5e27241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.545+0000 7f36d27fc700 1 -- 172.21.15.3:0/2228694110 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f36d40182a0 con 0x7f36dc10cc60 2023-08-11T14:13:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.547+0000 7f36d27fc700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 0x7f36c80a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.548+0000 7f36d27fc700 1 -- 172.21.15.3:0/2228694110 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f36d400d030 con 0x7f36dc10cc60 2023-08-11T14:13:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.549+0000 7f36e099a700 1 -- 172.21.15.3:0/3348580867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e27238a0 msgr2=0x55a5e27241b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.549+0000 7f36e099a700 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e27238a0 0x55a5e27241b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.549+0000 7f36e099a700 1 -- 172.21.15.3:0/3348580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e2724790 msgr2=0x55a5e2724c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.549+0000 7f36e099a700 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e2724790 0x55a5e2724c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.549+0000 7f36e099a700 1 -- 172.21.15.3:0/3348580867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e24f55d0 con 0x55a5e2722f40 2023-08-11T14:13:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.556+0000 7f36e199c700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 0x7f36c80a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.559+0000 7f36e099a700 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00a970 tx=0x7f36cc00ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0c67716887588f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.566+0000 7f36e199c700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 0x7f36c80a2610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36d4099860 tx=0x7f36d401c3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.570+0000 7f36c77fe700 1 -- 172.21.15.3:0/3348580867 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00f760 con 0x55a5e2722f40 2023-08-11T14:13:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.570+0000 7f36c77fe700 1 -- 172.21.15.3:0/3348580867 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc00c030 con 0x55a5e2722f40 2023-08-11T14:13:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.570+0000 7f36c77fe700 1 -- 172.21.15.3:0/3348580867 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc013d30 con 0x55a5e2722f40 2023-08-11T14:13:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.571+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3348580867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 msgr2=0x55a5e2723360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.571+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00a970 tx=0x7f36cc00ac80 comp rx=0 tx=0).stop 2023-08-11T14:13:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.572+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3348580867 shutdown_connections 2023-08-11T14:13:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.572+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e2724790 0x55a5e2724c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.572+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2723360 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.572+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3348580867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e27238a0 0x55a5e27241b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.572+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3348580867 >> 172.21.15.3:0/3348580867 conn(0x55a5e26508f0 msgr2=0x55a5e271f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.578+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3348580867 shutdown_connections 2023-08-11T14:13:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.579+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3348580867 wait complete. 2023-08-11T14:13:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.579+0000 7f36f6dcab80 1 -- start start 2023-08-11T14:13:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.580+0000 7f36f6dcab80 1 Processor -- start 2023-08-11T14:13:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e272cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2613a60 con 0x55a5e27238a0 2023-08-11T14:13:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2613760 con 0x55a5e2722f40 2023-08-11T14:13:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e24f78c0 con 0x55a5e2724790 2023-08-11T14:13:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36f6dcab80 10 client.0 _ll_register_callbacks cb 0x55a5e2616a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.581+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.582+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49604/0 (socket says 172.21.15.3:49604) 2023-08-11T14:13:15.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.582+0000 7f36e119b700 1 -- 172.21.15.3:0/3947922865 learned_addr learned my addr 172.21.15.3:0/3947922865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.585+0000 7f36e099a700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e272cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 -- 172.21.15.3:0/3947922865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 msgr2=0x55a5e27341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 -- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 msgr2=0x55a5e272cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e272cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 -- 172.21.15.3:0/3947922865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc00e040 con 0x55a5e27238a0 2023-08-11T14:13:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.586+0000 7f36e119b700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f36d800bfd0 tx=0x7f36d80127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f8361f7b748e87 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.587+0000 7f36c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.594+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004440 con 0x55a5e27238a0 2023-08-11T14:13:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.594+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f36d8003680 con 0x55a5e27238a0 2023-08-11T14:13:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.597+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5e27167e0 con 0x55a5e27238a0 2023-08-11T14:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.605+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d8003840 con 0x55a5e27238a0 2023-08-11T14:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.605+0000 7f36f6dcab80 10 client.16275 fetch_fsmap learned FSMap version 39 2023-08-11T14:13:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.605+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a5e24a1d80 con 0x55a5e27238a0 2023-08-11T14:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5e27167e0 con 0x55a5e27238a0 2023-08-11T14:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f36d801bb50 con 0x55a5e27238a0 2023-08-11T14:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 10 client.16275 fetch_fsmap finished waiting for FSMap version 39 2023-08-11T14:13:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 20 client.16275 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e2738b60 0x55a5e273af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 4 client.16275 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:13:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.608+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5e26f2480 con 0x55a5e2738b60 2023-08-11T14:13:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.614+0000 7f36e099a700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e2738b60 0x55a5e273af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.655+0000 7f36e099a700 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e2738b60 0x55a5e273af80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdfe714b1f8a38ff server_cookie=3b1bbd80c7fe8a38 in_seq=0 out_seq=0 2023-08-11T14:13:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.656+0000 7f36c4ff9700 10 client.16275 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.670+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3947922865 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a5e26f2480 con 0x55a5e2738b60 2023-08-11T14:13:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.670+0000 7f36c4ff9700 10 client.16275 handle_command_reply: tid=0 2023-08-11T14:13:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.670+0000 7f36f6dcab80 1 client.16275 shutdown 2023-08-11T14:13:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.670+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 msgr2=0x55a5e272d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.670+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f36d800bfd0 tx=0x7f36d80127e0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 shutdown_connections 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e2738b60 0x55a5e273af80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e272cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3947922865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e272d190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 >> 172.21.15.3:0/3947922865 conn(0x55a5e26508f0 msgr2=0x55a5e271e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36c4ff9700 0 client.16275 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 shutdown_connections 2023-08-11T14:13:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.671+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3947922865 wait complete. 2023-08-11T14:13:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.672+0000 7f36f6dcab80 20 client.16275 trim_cache size 0 max 16384 2023-08-11T14:13:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.703+0000 7f36f6dcab80 1 Processor -- start 2023-08-11T14:13:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.703+0000 7f36f6dcab80 1 -- start start 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e27276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e272aa00 con 0x55a5e27238a0 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2728250 con 0x55a5e2722f40 2023-08-11T14:13:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.704+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e272d8b0 con 0x55a5e2724790 2023-08-11T14:13:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.724+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.724+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49610/0 (socket says 172.21.15.3:49610) 2023-08-11T14:13:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.724+0000 7f36e119b700 1 -- 172.21.15.3:0/2930501477 learned_addr learned my addr 172.21.15.3:0/2930501477 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.726+0000 7f36e099a700 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.726+0000 7f36e099a700 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e27276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.746+0000 7f36e119b700 1 -- 172.21.15.3:0/2930501477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 msgr2=0x55a5e27340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.746+0000 7f36e119b700 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.746+0000 7f36e119b700 1 -- 172.21.15.3:0/2930501477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 msgr2=0x55a5e27276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.746+0000 7f36e119b700 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e27276e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.746+0000 7f36e119b700 1 -- 172.21.15.3:0/2930501477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e248ac00 con 0x55a5e27238a0 2023-08-11T14:13:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.761+0000 7f36e119b700 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f36d801bf10 tx=0x7f36d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f59ba5d1a4e29c0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.771+0000 7f36c7fff700 1 -- 172.21.15.3:0/2930501477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8025640 con 0x55a5e27238a0 2023-08-11T14:13:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.771+0000 7f36c7fff700 1 -- 172.21.15.3:0/2930501477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d8025d90 con 0x55a5e27238a0 2023-08-11T14:13:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.771+0000 7f36c7fff700 1 -- 172.21.15.3:0/2930501477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8005290 con 0x55a5e27238a0 2023-08-11T14:13:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.776+0000 7f36f6dcab80 1 -- 172.21.15.3:0/2930501477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 msgr2=0x55a5e2727c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.776+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f36d801bf10 tx=0x7f36d8005880 comp rx=0 tx=0).stop 2023-08-11T14:13:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.787+0000 7f36f6dcab80 1 -- 172.21.15.3:0/2930501477 shutdown_connections 2023-08-11T14:13:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.787+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e27340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.787+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e27276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.787+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/2930501477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2727c20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.787+0000 7f36f6dcab80 1 -- 172.21.15.3:0/2930501477 >> 172.21.15.3:0/2930501477 conn(0x55a5e26508f0 msgr2=0x55a5e272abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.798+0000 7f36f6dcab80 1 -- 172.21.15.3:0/2930501477 shutdown_connections 2023-08-11T14:13:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.800+0000 7f36f6dcab80 1 -- 172.21.15.3:0/2930501477 wait complete. 2023-08-11T14:13:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.800+0000 7f36f6dcab80 1 -- start start 2023-08-11T14:13:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.801+0000 7f36f6dcab80 1 Processor -- start 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.801+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2731ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.801+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.801+0000 7f36f6dcab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e2733420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.801+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2733960 con 0x55a5e27238a0 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.802+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2733ad0 con 0x55a5e2722f40 2023-08-11T14:13:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.802+0000 7f36f6dcab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2733c90 con 0x55a5e2724790 2023-08-11T14:13:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.802+0000 7f36f6dcab80 10 client.0 _ll_register_callbacks cb 0x7f36d800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.809+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.809+0000 7f36e119b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49626/0 (socket says 172.21.15.3:49626) 2023-08-11T14:13:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 -- 172.21.15.3:0/3124137291 learned_addr learned my addr 172.21.15.3:0/3124137291 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 -- 172.21.15.3:0/3124137291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 msgr2=0x55a5e2733420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e2733420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 -- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 msgr2=0x55a5e2731ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2731ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.810+0000 7f36e119b700 1 -- 172.21.15.3:0/3124137291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e24a1d80 con 0x55a5e27238a0 2023-08-11T14:13:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.811+0000 7f36e119b700 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36d80045b0 tx=0x7f36d80057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75fcf5148cc0f78 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.811+0000 7f36c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:13:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.817+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8025b60 con 0x55a5e27238a0 2023-08-11T14:13:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.817+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f36d8003680 con 0x55a5e27238a0 2023-08-11T14:13:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.817+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36cc00c030 con 0x55a5e27238a0 2023-08-11T14:13:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.818+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36d8003840 con 0x55a5e27238a0 2023-08-11T14:13:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.823+0000 7f36f6dcab80 10 client.16281 fetch_fsmap learned FSMap version 39 2023-08-11T14:13:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.823+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a5e24c81f0 con 0x55a5e27238a0 2023-08-11T14:13:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36cc00c030 con 0x55a5e27238a0 2023-08-11T14:13:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 39) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f36d802f460 con 0x55a5e27238a0 2023-08-11T14:13:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 10 client.16281 fetch_fsmap finished waiting for FSMap version 39 2023-08-11T14:13:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 20 client.16281 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e278d4c0 0x55a5e278f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 4 client.16281 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:13:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.840+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5e266e7c0 con 0x55a5e278d4c0 2023-08-11T14:13:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.866+0000 7f36e099a700 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e278d4c0 0x55a5e278f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.913+0000 7f36e099a700 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e278d4c0 0x55a5e278f8e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96cac41cd8f9ed9a server_cookie=226e9acd0d304ce0 in_seq=0 out_seq=0 2023-08-11T14:13:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:15.914+0000 7f36c4ff9700 10 client.16281 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:16.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:16 smithi003 ceph-mon[97195]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:13:16.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:16 smithi187 ceph-mon[130973]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:13:16.683 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:16 smithi103 ceph-mon[130434]: pgmap v627: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:13:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.057+0000 7f36c4ff9700 1 -- 172.21.15.3:0/3124137291 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5e266e7c0 con 0x55a5e278d4c0 2023-08-11T14:13:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.057+0000 7f36c4ff9700 10 client.16281 handle_command_reply: tid=0 2023-08-11T14:13:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.057+0000 7f36f6dcab80 1 client.16281 shutdown 2023-08-11T14:13:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.062+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 msgr2=0x55a5e2732410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.062+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f36d80045b0 tx=0x7f36d80057d0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 shutdown_connections 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a5e2724790 0x55a5e2733420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a5e278d4c0 0x55a5e278f8e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5e2722f40 0x55a5e2731ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 --2- 172.21.15.3:0/3124137291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5e27238a0 0x55a5e2732410 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 >> 172.21.15.3:0/3124137291 conn(0x55a5e26508f0 msgr2=0x55a5e27276e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36c4ff9700 0 client.16281 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.063+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 shutdown_connections 2023-08-11T14:13:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.064+0000 7f36f6dcab80 1 -- 172.21.15.3:0/3124137291 wait complete. 2023-08-11T14:13:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.064+0000 7f36f6dcab80 20 client.16281 trim_cache size 0 max 16384 2023-08-11T14:13:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:13:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:13:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:13:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:13:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.070+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 msgr2=0x7f36c80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.070+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 0x7f36c80a2610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36d4099860 tx=0x7f36d401c3e0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.070+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 msgr2=0x7f36dc19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:18.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.070+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36d4018bb0 tx=0x7f36d4018960 comp rx=0 tx=0).stop 2023-08-11T14:13:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 shutdown_connections 2023-08-11T14:13:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f36c80a0150 0x7f36c80a2610 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc10cc60 0x7f36dc19e070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36dc107920 0x7f36dc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 --2- 172.21.15.3:0/2228694110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36dc10a290 0x7f36dc19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.071+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 >> 172.21.15.3:0/2228694110 conn(0x7f36dc06d750 msgr2=0x7f36dc1082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.072+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 shutdown_connections 2023-08-11T14:13:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:18.072+0000 7f36e33ff700 1 -- 172.21.15.3:0/2228694110 wait complete. 2023-08-11T14:13:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 380.0 s, 690.8 tps, lat 143.434 ms stddev 394.220 2023-08-11T14:13:18.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:18 smithi003 ceph-mon[97195]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.24k op/s 2023-08-11T14:13:18.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:18 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:18.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:18 smithi187 ceph-mon[130973]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.24k op/s 2023-08-11T14:13:18.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:18 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:18.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:18 smithi103 ceph-mon[130434]: pgmap v628: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.24k op/s 2023-08-11T14:13:18.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:18 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:19.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:19 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:13:19.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:19 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:13:19.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:19 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:13:20.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:20 smithi003 ceph-mon[97195]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2023-08-11T14:13:20.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:20 smithi187 ceph-mon[130973]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2023-08-11T14:13:20.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:20 smithi103 ceph-mon[130434]: pgmap v629: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.48k op/s 2023-08-11T14:13:21.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:21 smithi187 ceph-mon[130973]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:13:21.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:21 smithi003 ceph-mon[97195]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:13:21.935 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:21 smithi103 ceph-mon[130434]: pgmap v630: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:13:22.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:22 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:22.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:22 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:22.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:22 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 385.0 s, 984.5 tps, lat 77.956 ms stddev 74.495 2023-08-11T14:13:24.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:24 smithi103 ceph-mon[130434]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 20 MiB/s wr, 1.34k op/s 2023-08-11T14:13:24.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:24 smithi187 ceph-mon[130973]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 20 MiB/s wr, 1.34k op/s 2023-08-11T14:13:24.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:24 smithi003 ceph-mon[97195]: pgmap v631: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 20 MiB/s wr, 1.34k op/s 2023-08-11T14:13:25.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:13:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:13:25.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:25 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:13:25.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:13:25.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:13:25.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:25 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:13:25.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:13:25.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:13:25.549 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:25 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:13:26.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:26 smithi103 ceph-mon[130434]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-08-11T14:13:26.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:26 smithi003 ceph-mon[97195]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-08-11T14:13:26.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:26 smithi187 ceph-mon[130973]: pgmap v632: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 19 MiB/s wr, 1.23k op/s 2023-08-11T14:13:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 390.0 s, 354.1 tps, lat 346.220 ms stddev 511.682 2023-08-11T14:13:28.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:28 smithi003 ceph-mon[97195]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:13:28.553 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:28 smithi187 ceph-mon[130973]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:13:28.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:28 smithi103 ceph-mon[130434]: pgmap v633: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:13:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:30 smithi103 ceph-mon[130434]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:13:30.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:30 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:30 smithi187 ceph-mon[130973]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:13:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:30 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:30.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:30 smithi003 ceph-mon[97195]: pgmap v634: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 20 MiB/s wr, 1.52k op/s 2023-08-11T14:13:30.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:30 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.393+0000 7f62633ff700 1 -- 172.21.15.3:0/995182006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c10a550 msgr2=0x7f625c10c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.393+0000 7f62633ff700 1 --2- 172.21.15.3:0/995182006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c10a550 0x7f625c10c980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6254009f90 tx=0x7f625400b330 comp rx=0 tx=0).stop 2023-08-11T14:13:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.394+0000 7f62633ff700 1 -- 172.21.15.3:0/995182006 shutdown_connections 2023-08-11T14:13:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.394+0000 7f62633ff700 1 --2- 172.21.15.3:0/995182006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c10d010 0x7f625c10f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.394+0000 7f62633ff700 1 --2- 172.21.15.3:0/995182006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c107b90 0x7f625c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.394+0000 7f62633ff700 1 --2- 172.21.15.3:0/995182006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c10a550 0x7f625c10c980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.394+0000 7f62633ff700 1 -- 172.21.15.3:0/995182006 >> 172.21.15.3:0/995182006 conn(0x7f625c06daf0 msgr2=0x7f625c06ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.402+0000 7f62633ff700 1 -- 172.21.15.3:0/995182006 shutdown_connections 2023-08-11T14:13:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.404+0000 7f62633ff700 1 -- 172.21.15.3:0/995182006 wait complete. 2023-08-11T14:13:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.406+0000 7f62633ff700 1 Processor -- start 2023-08-11T14:13:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.406+0000 7f62633ff700 1 -- start start 2023-08-11T14:13:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.406+0000 7f62633ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c107b90 0x7f625c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.406+0000 7f62633ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f62633ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c10d010 0x7f625c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f62633ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c19e7e0 con 0x7f625c10d010 2023-08-11T14:13:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f62633ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c19e950 con 0x7f625c107b90 2023-08-11T14:13:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f62633ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c19eb10 con 0x7f625c10a550 2023-08-11T14:13:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60448/0 (socket says 172.21.15.3:60448) 2023-08-11T14:13:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.407+0000 7f626099a700 1 -- 172.21.15.3:0/41904978 learned_addr learned my addr 172.21.15.3:0/41904978 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 -- 172.21.15.3:0/41904978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c107b90 msgr2=0x7f625c19cdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:13:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c107b90 0x7f625c19cdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 -- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c10d010 msgr2=0x7f625c19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c10d010 0x7f625c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 -- 172.21.15.3:0/41904978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009c40 con 0x7f625c10a550 2023-08-11T14:13:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.408+0000 7f626099a700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f625400b770 tx=0x7f6254007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.409+0000 7f62527fc700 1 -- 172.21.15.3:0/41904978 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400e040 con 0x7f625c10a550 2023-08-11T14:13:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.409+0000 7f62527fc700 1 -- 172.21.15.3:0/41904978 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6254004e40 con 0x7f625c10a550 2023-08-11T14:13:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.409+0000 7f62527fc700 1 -- 172.21.15.3:0/41904978 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625401b880 con 0x7f625c10a550 2023-08-11T14:13:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.409+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c19ed90 con 0x7f625c10a550 2023-08-11T14:13:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.409+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c078de0 con 0x7f625c10a550 2023-08-11T14:13:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.421+0000 7f6276c63b80 1 Processor -- start 2023-08-11T14:13:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.424+0000 7f6276c63b80 1 -- start start 2023-08-11T14:13:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fce10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fc3a5d0 con 0x56199fce0c80 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fc37ef0 con 0x56199fce15e0 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f6276c63b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fc337e0 con 0x56199fcef790 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.425+0000 7f62527fc700 1 -- 172.21.15.3:0/41904978 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f625401dac0 con 0x7f625c10a550 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.427+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.427+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60454/0 (socket says 172.21.15.3:60454) 2023-08-11T14:13:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.427+0000 7f626099a700 1 -- 172.21.15.3:0/1205676310 learned_addr learned my addr 172.21.15.3:0/1205676310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.429+0000 7f626099a700 1 -- 172.21.15.3:0/1205676310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 msgr2=0x56199fcef250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:13:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.429+0000 7f626099a700 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcef250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.429+0000 7f626099a700 1 -- 172.21.15.3:0/1205676310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 msgr2=0x56199fce10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.429+0000 7f626099a700 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fce10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.429+0000 7f626099a700 1 -- 172.21.15.3:0/1205676310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56199fab35d0 con 0x56199fcef790 2023-08-11T14:13:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.440+0000 7f626099a700 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62540d2a90 tx=0x7f62540d2ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2fbdabb1dfaf9d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.441+0000 7f62527fc700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 0x7f62480a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.442+0000 7f62527fc700 1 -- 172.21.15.3:0/41904978 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f62540cc960 con 0x7f625c10a550 2023-08-11T14:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.443+0000 7f62477fe700 1 -- 172.21.15.3:0/1205676310 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540d48d0 con 0x56199fcef790 2023-08-11T14:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.443+0000 7f62477fe700 1 -- 172.21.15.3:0/1205676310 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6254096400 con 0x56199fcef790 2023-08-11T14:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.443+0000 7f62477fe700 1 -- 172.21.15.3:0/1205676310 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254099a20 con 0x56199fcef790 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.444+0000 7f6276c63b80 1 -- 172.21.15.3:0/1205676310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 msgr2=0x56199fcf1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.444+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62540d2a90 tx=0x7f62540d2ac0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.445+0000 7f6276c63b80 1 -- 172.21.15.3:0/1205676310 shutdown_connections 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.445+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf1b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.445+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fce10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.445+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1205676310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.446+0000 7f6276c63b80 1 -- 172.21.15.3:0/1205676310 >> 172.21.15.3:0/1205676310 conn(0x56199fc0e8f0 msgr2=0x56199fcdd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.449+0000 7f626199c700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 0x7f62480a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.450+0000 7f6276c63b80 1 -- 172.21.15.3:0/1205676310 shutdown_connections 2023-08-11T14:13:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.459+0000 7f6276c63b80 1 -- 172.21.15.3:0/1205676310 wait complete. 2023-08-11T14:13:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.460+0000 7f6276c63b80 1 -- start start 2023-08-11T14:13:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.461+0000 7f6276c63b80 1 Processor -- start 2023-08-11T14:13:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.461+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce0c80 0x56199fcf3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.461+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.461+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fcef790 0x56199fcfa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.462+0000 7f6276c63b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fbd1a60 con 0x56199fcef790 2023-08-11T14:13:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.462+0000 7f6276c63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fbd1760 con 0x56199fce0c80 2023-08-11T14:13:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.462+0000 7f6276c63b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fab58c0 con 0x56199fce15e0 2023-08-11T14:13:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.462+0000 7f6276c63b80 10 client.0 _ll_register_callbacks cb 0x56199fbd4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.463+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.463+0000 7f626099a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:60460/0 (socket says 172.21.15.3:60460) 2023-08-11T14:13:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.463+0000 7f626099a700 1 -- 172.21.15.3:0/792013913 learned_addr learned my addr 172.21.15.3:0/792013913 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.466+0000 7f626199c700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fcef790 0x56199fcfa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.478+0000 7f626099a700 1 -- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce0c80 msgr2=0x56199fcf3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.478+0000 7f626099a700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce0c80 0x56199fcf3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.478+0000 7f626099a700 1 -- 172.21.15.3:0/792013913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fcef790 msgr2=0x56199fcfa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.478+0000 7f626099a700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fcef790 0x56199fcfa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.478+0000 7f626099a700 1 -- 172.21.15.3:0/792013913 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c078de0 con 0x56199fce15e0 2023-08-11T14:13:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.483+0000 7f626099a700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6254099d70 tx=0x7f625401de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70e3476aef7def4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.488+0000 7f626199c700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 0x7f62480a2610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56199fce2410 tx=0x7f624c009ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.488+0000 7f6244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.488+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625409a0c0 con 0x56199fce15e0 2023-08-11T14:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.488+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f625409def0 con 0x56199fce15e0 2023-08-11T14:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.489+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56199fcd47e0 con 0x56199fce15e0 2023-08-11T14:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62540a1e40 con 0x56199fce15e0 2023-08-11T14:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 10 client.26428 fetch_fsmap learned FSMap version 42 2023-08-11T14:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56199fa5fd80 con 0x56199fce15e0 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62540b2020 con 0x56199fce15e0 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 42) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f62540995b0 con 0x56199fce15e0 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 10 client.26428 fetch_fsmap finished waiting for FSMap version 42 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 10 client.26428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 20 client.26428 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fce6a00 0x56199fce8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 4 client.26428 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:13:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.490+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56199fcb0480 con 0x56199fce6a00 2023-08-11T14:13:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.491+0000 7f626119b700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fce6a00 0x56199fce8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.493+0000 7f626119b700 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fce6a00 0x56199fce8e20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87604b6d6870a95 server_cookie=309fd587d87c9da in_seq=0 out_seq=0 2023-08-11T14:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.493+0000 7f6244ff9700 10 client.26428 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.494+0000 7f6244ff9700 1 -- 172.21.15.3:0/792013913 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56199fcb0480 con 0x56199fce6a00 2023-08-11T14:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.494+0000 7f6244ff9700 10 client.26428 handle_command_reply: tid=0 2023-08-11T14:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.494+0000 7f6276c63b80 1 client.26428 shutdown 2023-08-11T14:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.495+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 msgr2=0x56199fcf3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.495+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6254099d70 tx=0x7f625401de60 comp rx=0 tx=0).stop 2023-08-11T14:13:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 shutdown_connections 2023-08-11T14:13:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fcef790 0x56199fcfa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fce6a00 0x56199fce8e20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce0c80 0x56199fcf3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 --2- 172.21.15.3:0/792013913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fce15e0 0x56199fcf3770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.508+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 >> 172.21.15.3:0/792013913 conn(0x56199fc0e8f0 msgr2=0x56199fcdb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.511+0000 7f6244ff9700 0 client.26428 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.513+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 shutdown_connections 2023-08-11T14:13:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.514+0000 7f6276c63b80 1 -- 172.21.15.3:0/792013913 wait complete. 2023-08-11T14:13:31.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.524+0000 7f6276c63b80 20 client.26428 trim_cache size 0 max 16384 2023-08-11T14:13:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.559+0000 7f6276c63b80 1 Processor -- start 2023-08-11T14:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.560+0000 7f6276c63b80 1 -- start start 2023-08-11T14:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fcdcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcfa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcdd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fcd6bf0 con 0x56199fce0c80 2023-08-11T14:13:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fced990 con 0x56199fce15e0 2023-08-11T14:13:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.561+0000 7f6276c63b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fcb04b0 con 0x56199fcef790 2023-08-11T14:13:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.563+0000 7f626199c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcdd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.563+0000 7f626199c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcdd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33944/0 (socket says 172.21.15.3:33944) 2023-08-11T14:13:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.563+0000 7f626199c700 1 -- 172.21.15.3:0/1029781061 learned_addr learned my addr 172.21.15.3:0/1029781061 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.565+0000 7f626099a700 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcfa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fcdcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 -- 172.21.15.3:0/1029781061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 msgr2=0x56199fcdd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcdd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 -- 172.21.15.3:0/1029781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 msgr2=0x56199fcfa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcfa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.568+0000 7f626119b700 1 -- 172.21.15.3:0/1029781061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56199fa48c00 con 0x56199fce0c80 2023-08-11T14:13:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.574+0000 7f626119b700 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fcdcea0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56199fce96c0 tx=0x7f62580058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5bc9f6fee59d71 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.580+0000 7f6247fff700 1 -- 172.21.15.3:0/1029781061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625800f040 con 0x56199fce0c80 2023-08-11T14:13:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.581+0000 7f6247fff700 1 -- 172.21.15.3:0/1029781061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62580178c0 con 0x56199fce0c80 2023-08-11T14:13:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.581+0000 7f6247fff700 1 -- 172.21.15.3:0/1029781061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258003970 con 0x56199fce0c80 2023-08-11T14:13:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 -- 172.21.15.3:0/1029781061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 msgr2=0x56199fcdcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fcdcea0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56199fce96c0 tx=0x7f62580058e0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 -- 172.21.15.3:0/1029781061 shutdown_connections 2023-08-11T14:13:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcdd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce0c80 0x56199fcdcea0 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x56199fce96c0 tx=0x7f62580058e0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 --2- 172.21.15.3:0/1029781061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fce15e0 0x56199fcfa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.582+0000 7f6276c63b80 1 -- 172.21.15.3:0/1029781061 >> 172.21.15.3:0/1029781061 conn(0x56199fc0e8f0 msgr2=0x56199fcf3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.589+0000 7f6276c63b80 1 -- 172.21.15.3:0/1029781061 shutdown_connections 2023-08-11T14:13:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.590+0000 7f6276c63b80 1 -- 172.21.15.3:0/1029781061 wait complete. 2023-08-11T14:13:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.590+0000 7f6276c63b80 1 -- start start 2023-08-11T14:13:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 Processor -- start 2023-08-11T14:13:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce15e0 0x56199fcfa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fd5ac30 con 0x56199fce15e0 2023-08-11T14:13:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fd5ad70 con 0x56199fcf90b0 2023-08-11T14:13:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.591+0000 7f6276c63b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56199fd5aeb0 con 0x56199fcef790 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.592+0000 7f6276c63b80 10 client.0 _ll_register_callbacks cb 0x7f62540d28e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.592+0000 7f626199c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.592+0000 7f626199c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:34588/0 (socket says 172.21.15.3:34588) 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.592+0000 7f626199c700 1 -- 172.21.15.3:0/210476833 learned_addr learned my addr 172.21.15.3:0/210476833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 -- 172.21.15.3:0/210476833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 msgr2=0x56199fcf8b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:13:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 -- 172.21.15.3:0/210476833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce15e0 msgr2=0x56199fcfa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce15e0 0x56199fcfa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 -- 172.21.15.3:0/210476833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625800a9c0 con 0x56199fcf90b0 2023-08-11T14:13:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.593+0000 7f626199c700 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f624c014620 tx=0x7f624c004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302149bef25cc2e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.594+0000 7f6244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:13:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.594+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c016e80 con 0x56199fcf90b0 2023-08-11T14:13:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.594+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f624c021680 con 0x56199fcf90b0 2023-08-11T14:13:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.594+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6254096400 con 0x56199fcf90b0 2023-08-11T14:13:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.595+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624c021840 con 0x56199fcf90b0 2023-08-11T14:13:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.595+0000 7f6276c63b80 10 client.26414 fetch_fsmap learned FSMap version 42 2023-08-11T14:13:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.595+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56199fa861f0 con 0x56199fcf90b0 2023-08-11T14:13:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.596+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6254096400 con 0x56199fcf90b0 2023-08-11T14:13:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.596+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 42) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f624c02ab60 con 0x56199fcf90b0 2023-08-11T14:13:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 42 2023-08-11T14:13:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 20 client.26414 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fd5c9d0 0x56199fd5edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 4 client.26414 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:13:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56199fc2c7c0 con 0x56199fd5c9d0 2023-08-11T14:13:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.597+0000 7f626119b700 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fd5c9d0 0x56199fd5edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.598+0000 7f626119b700 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fd5c9d0 0x56199fd5edf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b35333b9698e0e8 server_cookie=12d4c2f6d145155e in_seq=0 out_seq=0 2023-08-11T14:13:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.599+0000 7f6244ff9700 10 client.26414 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.751+0000 7f6244ff9700 1 -- 172.21.15.3:0/210476833 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56199fc2c7c0 con 0x56199fd5c9d0 2023-08-11T14:13:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.751+0000 7f6244ff9700 10 client.26414 handle_command_reply: tid=0 2023-08-11T14:13:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.751+0000 7f6276c63b80 1 client.26414 shutdown 2023-08-11T14:13:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.755+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 msgr2=0x56199fcf9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.755+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f624c014620 tx=0x7f624c004860 comp rx=0 tx=0).stop 2023-08-11T14:13:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 shutdown_connections 2023-08-11T14:13:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56199fd5c9d0 0x56199fd5edf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56199fcf90b0 0x56199fcf9530 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56199fce15e0 0x56199fcfa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 --2- 172.21.15.3:0/210476833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56199fcef790 0x56199fcf8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 >> 172.21.15.3:0/210476833 conn(0x56199fc0e8f0 msgr2=0x56199fcdcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6244ff9700 0 client.26414 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 shutdown_connections 2023-08-11T14:13:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.757+0000 7f6276c63b80 1 -- 172.21.15.3:0/210476833 wait complete. 2023-08-11T14:13:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.769+0000 7f6276c63b80 20 client.26414 trim_cache size 0 max 16384 2023-08-11T14:13:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:13:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:13:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:13:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:13:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.794+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 msgr2=0x7f62480a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.794+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 0x7f62480a2610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56199fce2410 tx=0x7f624c009ad0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.794+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 msgr2=0x7f625c19d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.794+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f625400b770 tx=0x7f6254007730 comp rx=0 tx=0).stop 2023-08-11T14:13:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 shutdown_connections 2023-08-11T14:13:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f625c10d010 0x7f625c19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f62480a0150 0x7f62480a2610 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f625c107b90 0x7f625c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 --2- 172.21.15.3:0/41904978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f625c10a550 0x7f625c19d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.797+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 >> 172.21.15.3:0/41904978 conn(0x7f625c06daf0 msgr2=0x7f625c108800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.798+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 shutdown_connections 2023-08-11T14:13:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:31.798+0000 7f62633ff700 1 -- 172.21.15.3:0/41904978 wait complete. 2023-08-11T14:13:32.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:32 smithi103 ceph-mon[130434]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-08-11T14:13:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:32 smithi003 ceph-mon[97195]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-08-11T14:13:32.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:32 smithi187 ceph-mon[130973]: pgmap v635: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 1.16k op/s 2023-08-11T14:13:33.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 395.0 s, 365.3 tps, lat 273.054 ms stddev 258.035 2023-08-11T14:13:34.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:34 smithi103 ceph-mon[130434]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:13:34.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:34 smithi003 ceph-mon[97195]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:13:34.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:34 smithi187 ceph-mon[130973]: pgmap v636: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:13:35.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:35 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:35.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:35 smithi103 ceph-mon[130434]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:13:36.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:35 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:36.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:35 smithi187 ceph-mon[130973]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:13:36.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:35 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:36.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:35 smithi003 ceph-mon[97195]: pgmap v637: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:13:37.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:37 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:37.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:37 smithi103 ceph-mon[130434]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:13:38.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:37 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:38.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:37 smithi003 ceph-mon[97195]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:13:38.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:37 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:38.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:37 smithi187 ceph-mon[130973]: pgmap v638: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:13:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 400.0 s, 524.9 tps, lat 193.801 ms stddev 156.101 2023-08-11T14:13:40.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:40 smithi103 ceph-mon[130434]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 14 MiB/s wr, 966 op/s 2023-08-11T14:13:40.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:40 smithi187 ceph-mon[130973]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 14 MiB/s wr, 966 op/s 2023-08-11T14:13:40.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:40 smithi003 ceph-mon[97195]: pgmap v639: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 14 MiB/s wr, 966 op/s 2023-08-11T14:13:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:42 smithi003 ceph-mon[97195]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 702 op/s 2023-08-11T14:13:42.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:42 smithi187 ceph-mon[130973]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 702 op/s 2023-08-11T14:13:42.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:42 smithi103 ceph-mon[130434]: pgmap v640: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 11 MiB/s wr, 702 op/s 2023-08-11T14:13:43.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 405.0 s, 837.7 tps, lat 81.808 ms stddev 77.429 2023-08-11T14:13:44.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:44 smithi003 ceph-mon[97195]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 821 op/s 2023-08-11T14:13:44.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:44 smithi187 ceph-mon[130973]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 821 op/s 2023-08-11T14:13:44.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:44 smithi103 ceph-mon[130434]: pgmap v641: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 821 op/s 2023-08-11T14:13:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.071+0000 7f8057fff700 1 -- 172.21.15.3:0/1163667561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80501013b0 msgr2=0x7f8050107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.071+0000 7f8057fff700 1 --2- 172.21.15.3:0/1163667561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80501013b0 0x7f8050107c60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f804800b0d0 tx=0x7f804800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.072+0000 7f8057fff700 1 -- 172.21.15.3:0/1163667561 shutdown_connections 2023-08-11T14:13:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.072+0000 7f8057fff700 1 --2- 172.21.15.3:0/1163667561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80501013b0 0x7f8050107c60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.072+0000 7f8057fff700 1 --2- 172.21.15.3:0/1163667561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80500ff820 0x7f80500ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.072+0000 7f8057fff700 1 --2- 172.21.15.3:0/1163667561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8050100a10 0x7f8050100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.072+0000 7f8057fff700 1 -- 172.21.15.3:0/1163667561 >> 172.21.15.3:0/1163667561 conn(0x7f80500fad70 msgr2=0x7f80500fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.073+0000 7f8057fff700 1 -- 172.21.15.3:0/1163667561 shutdown_connections 2023-08-11T14:13:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.073+0000 7f8057fff700 1 -- 172.21.15.3:0/1163667561 wait complete. 2023-08-11T14:13:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.075+0000 7f8057fff700 1 Processor -- start 2023-08-11T14:13:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.075+0000 7f8057fff700 1 -- start start 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80500ff820 0x7f8050067f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80501013b0 0x7f80501ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8050068b50 con 0x7f8050100a10 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80501a4780 con 0x7f80501013b0 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.076+0000 7f8057fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80501a4940 con 0x7f80500ff820 2023-08-11T14:13:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.077+0000 7f805559a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.077+0000 7f805559a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59814/0 (socket says 172.21.15.3:59814) 2023-08-11T14:13:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.077+0000 7f805559a700 1 -- 172.21.15.3:0/3755129309 learned_addr learned my addr 172.21.15.3:0/3755129309 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 -- 172.21.15.3:0/3755129309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80500ff820 msgr2=0x7f8050067f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80500ff820 0x7f8050067f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 -- 172.21.15.3:0/3755129309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80501013b0 msgr2=0x7f80501ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80501013b0 0x7f80501ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 -- 172.21.15.3:0/3755129309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009d00 con 0x7f8050100a10 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.078+0000 7f805559a700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b550 tx=0x7f804c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.079+0000 7f8042ffd700 1 -- 172.21.15.3:0/3755129309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c011830 con 0x7f8050100a10 2023-08-11T14:13:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.079+0000 7f8042ffd700 1 -- 172.21.15.3:0/3755129309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f804c011e50 con 0x7f8050100a10 2023-08-11T14:13:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.079+0000 7f8042ffd700 1 -- 172.21.15.3:0/3755129309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c010e70 con 0x7f8050100a10 2023-08-11T14:13:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.079+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80501a4bc0 con 0x7f8050100a10 2023-08-11T14:13:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.079+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501a5100 con 0x7f8050100a10 2023-08-11T14:13:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.086+0000 7f806b896b80 1 Processor -- start 2023-08-11T14:13:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.087+0000 7f8042ffd700 1 -- 172.21.15.3:0/3755129309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f804c010510 con 0x7f8050100a10 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 -- start start 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89a8a0 0x564a1d89b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d7f35d0 con 0x564a1d899f40 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d7f0ef0 con 0x564a1d89a8a0 2023-08-11T14:13:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.088+0000 7f806b896b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d7ec7e0 con 0x564a1d89b790 2023-08-11T14:13:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.089+0000 7f805559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.089+0000 7f805559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:56674/0 (socket says 172.21.15.3:56674) 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 -- 172.21.15.3:0/87371012 learned_addr learned my addr 172.21.15.3:0/87371012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805659c700 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 -- 172.21.15.3:0/87371012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89a8a0 msgr2=0x564a1d89b1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89a8a0 0x564a1d89b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 -- 172.21.15.3:0/87371012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 msgr2=0x564a1d89a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805559a700 1 -- 172.21.15.3:0/87371012 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1d66c5d0 con 0x564a1d89b790 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.090+0000 7f805659c700 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.091+0000 7f805559a700 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f804c0cef00 tx=0x7f804c0d33c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f65367ebe26dab2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.091+0000 7f8042ffd700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 0x7f803c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.091+0000 7f8042ffd700 1 -- 172.21.15.3:0/3755129309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f804c0cafd0 con 0x7f8050100a10 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.092+0000 7f803bfff700 1 -- 172.21.15.3:0/87371012 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c010380 con 0x564a1d89b790 2023-08-11T14:13:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.092+0000 7f803bfff700 1 -- 172.21.15.3:0/87371012 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f804c0949e0 con 0x564a1d89b790 2023-08-11T14:13:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.092+0000 7f803bfff700 1 -- 172.21.15.3:0/87371012 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0981e0 con 0x564a1d89b790 2023-08-11T14:13:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.093+0000 7f806b896b80 1 -- 172.21.15.3:0/87371012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 msgr2=0x564a1d89bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.093+0000 7f806b896b80 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f804c0cef00 tx=0x7f804c0d33c0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f805659c700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 0x7f803c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f806b896b80 1 -- 172.21.15.3:0/87371012 shutdown_connections 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f806b896b80 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89b790 0x564a1d89bc10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f806b896b80 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f806b896b80 1 --2- 172.21.15.3:0/87371012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89a8a0 0x564a1d89b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.095+0000 7f806b896b80 1 -- 172.21.15.3:0/87371012 >> 172.21.15.3:0/87371012 conn(0x564a1d7c78f0 msgr2=0x564a1d896920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.096+0000 7f805659c700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 0x7f803c0a2610 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8048009fd0 tx=0x7f804800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.096+0000 7f806b896b80 1 -- 172.21.15.3:0/87371012 shutdown_connections 2023-08-11T14:13:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.097+0000 7f806b896b80 1 -- 172.21.15.3:0/87371012 wait complete. 2023-08-11T14:13:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.097+0000 7f806b896b80 1 -- start start 2023-08-11T14:13:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 Processor -- start 2023-08-11T14:13:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d8ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d78aa60 con 0x564a1d899f40 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d78a760 con 0x564a1d89b790 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d66e8c0 con 0x564a1d89a8a0 2023-08-11T14:13:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.098+0000 7f806b896b80 10 client.0 _ll_register_callbacks cb 0x564a1d78da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59834/0 (socket says 172.21.15.3:59834) 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 -- 172.21.15.3:0/250639037 learned_addr learned my addr 172.21.15.3:0/250639037 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f805559a700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f805659c700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d8ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 -- 172.21.15.3:0/250639037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 msgr2=0x564a1d8a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 -- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 msgr2=0x564a1d8ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d8ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f8055d9b700 1 -- 172.21.15.3:0/250639037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501a5100 con 0x564a1d899f40 2023-08-11T14:13:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.099+0000 7f805659c700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d8ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:13:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.100+0000 7f8055d9b700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8044009e00 tx=0x7f804400b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245ab3f76c5012fc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.100+0000 7f80397fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:13:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.100+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400be60 con 0x564a1d899f40 2023-08-11T14:13:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.100+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8044011420 con 0x564a1d899f40 2023-08-11T14:13:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.100+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a1d88d7e0 con 0x564a1d899f40 2023-08-11T14:13:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.101+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80440115e0 con 0x564a1d899f40 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.101+0000 7f806b896b80 10 client.16296 fetch_fsmap learned FSMap version 44 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.101+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x564a1d618d80 con 0x564a1d899f40 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.102+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a1d88d7e0 con 0x564a1d899f40 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.102+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 44) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8044019870 con 0x564a1d899f40 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.102+0000 7f806b896b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 44 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.102+0000 7f806b896b80 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.103+0000 7f806b896b80 20 client.16296 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.103+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d8afb20 0x564a1d8b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.103+0000 7f806b896b80 4 client.16296 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:13:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.103+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a1d869480 con 0x564a1d8afb20 2023-08-11T14:13:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.105+0000 7f805559a700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d8afb20 0x564a1d8b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.106+0000 7f805559a700 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d8afb20 0x564a1d8b1f40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ef1b4cb219b731 server_cookie=a67ea59c6f5dacb6 in_seq=0 out_seq=0 2023-08-11T14:13:45.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.106+0000 7f80397fa700 10 client.16296 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.107+0000 7f80397fa700 1 -- 172.21.15.3:0/250639037 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564a1d869480 con 0x564a1d8afb20 2023-08-11T14:13:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.107+0000 7f80397fa700 10 client.16296 handle_command_reply: tid=0 2023-08-11T14:13:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.107+0000 7f806b896b80 1 client.16296 shutdown 2023-08-11T14:13:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.111+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 msgr2=0x564a1d8a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.111+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8044009e00 tx=0x7f804400b3c0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 shutdown_connections 2023-08-11T14:13:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d8afb20 0x564a1d8b1f40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d8ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d8a3c50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 --2- 172.21.15.3:0/250639037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 >> 172.21.15.3:0/250639037 conn(0x564a1d7c78f0 msgr2=0x564a1d895710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.112+0000 7f80397fa700 0 client.16296 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.113+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 shutdown_connections 2023-08-11T14:13:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.113+0000 7f806b896b80 1 -- 172.21.15.3:0/250639037 wait complete. 2023-08-11T14:13:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.114+0000 7f806b896b80 20 client.16296 trim_cache size 0 max 16384 2023-08-11T14:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 Processor -- start 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 -- start start 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d89ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8a1960 con 0x564a1d899f40 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d89f220 con 0x564a1d89b790 2023-08-11T14:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.146+0000 7f806b896b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8af8a0 con 0x564a1d89a8a0 2023-08-11T14:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.148+0000 7f805559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.148+0000 7f805559a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:56690/0 (socket says 172.21.15.3:56690) 2023-08-11T14:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.148+0000 7f805559a700 1 -- 172.21.15.3:0/1087893653 learned_addr learned my addr 172.21.15.3:0/1087893653 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.148+0000 7f805659c700 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d89ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.148+0000 7f8055d9b700 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.149+0000 7f805659c700 1 -- 172.21.15.3:0/1087893653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 msgr2=0x564a1d8ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.149+0000 7f805659c700 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.149+0000 7f805659c700 1 -- 172.21.15.3:0/1087893653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 msgr2=0x564a1d89e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.149+0000 7f805659c700 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.149+0000 7f805659c700 1 -- 172.21.15.3:0/1087893653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1d601c00 con 0x564a1d89b790 2023-08-11T14:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.150+0000 7f805659c700 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d89ebf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f804800b0d0 tx=0x7f804801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa61f6e881ae2bac server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.155+0000 7f8040ff9700 1 -- 172.21.15.3:0/1087893653 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048007920 con 0x564a1d89b790 2023-08-11T14:13:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.155+0000 7f8040ff9700 1 -- 172.21.15.3:0/1087893653 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f804801b8f0 con 0x564a1d89b790 2023-08-11T14:13:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.155+0000 7f8040ff9700 1 -- 172.21.15.3:0/1087893653 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480297e0 con 0x564a1d89b790 2023-08-11T14:13:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.156+0000 7f806b896b80 1 -- 172.21.15.3:0/1087893653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 msgr2=0x564a1d89ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:45.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.156+0000 7f806b896b80 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d89ebf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f804800b0d0 tx=0x7f804801b000 comp rx=0 tx=0).stop 2023-08-11T14:13:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.157+0000 7f806b896b80 1 -- 172.21.15.3:0/1087893653 shutdown_connections 2023-08-11T14:13:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.157+0000 7f806b896b80 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d89b790 0x564a1d89ebf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.157+0000 7f806b896b80 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d899f40 0x564a1d89e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.157+0000 7f806b896b80 1 --2- 172.21.15.3:0/1087893653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.157+0000 7f806b896b80 1 -- 172.21.15.3:0/1087893653 >> 172.21.15.3:0/1087893653 conn(0x564a1d7c78f0 msgr2=0x564a1d8a29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.168+0000 7f806b896b80 1 -- 172.21.15.3:0/1087893653 shutdown_connections 2023-08-11T14:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.168+0000 7f806b896b80 1 -- 172.21.15.3:0/1087893653 wait complete. 2023-08-11T14:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.168+0000 7f806b896b80 1 -- start start 2023-08-11T14:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 Processor -- start 2023-08-11T14:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d899f40 0x564a1d8a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8aa780 con 0x564a1d89b790 2023-08-11T14:13:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8aa8f0 con 0x564a1d899f40 2023-08-11T14:13:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.169+0000 7f806b896b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8aaab0 con 0x564a1d89a8a0 2023-08-11T14:13:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.170+0000 7f806b896b80 10 client.0 _ll_register_callbacks cb 0x7f8044009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:13:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.173+0000 7f805659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.173+0000 7f805659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59852/0 (socket says 172.21.15.3:59852) 2023-08-11T14:13:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.173+0000 7f805659c700 1 -- 172.21.15.3:0/1359235722 learned_addr learned my addr 172.21.15.3:0/1359235722 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:13:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.173+0000 7f805559a700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.177+0000 7f8055d9b700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d899f40 0x564a1d8a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.183+0000 7f805659c700 1 -- 172.21.15.3:0/1359235722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 msgr2=0x564a1d8a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.183+0000 7f805659c700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.183+0000 7f805659c700 1 -- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d899f40 msgr2=0x564a1d8a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:13:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.183+0000 7f805659c700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d899f40 0x564a1d8a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.183+0000 7f805659c700 1 -- 172.21.15.3:0/1359235722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80501085b0 con 0x564a1d89b790 2023-08-11T14:13:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.216+0000 7f805659c700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8048009610 tx=0x7f804801bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1366c7b5341f54a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:13:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.217+0000 7f80397fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:13:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.240+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480296e0 con 0x564a1d89b790 2023-08-11T14:13:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.240+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8048023070 con 0x564a1d89b790 2023-08-11T14:13:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.240+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f804c0949e0 con 0x564a1d89b790 2023-08-11T14:13:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.255+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8048029840 con 0x564a1d89b790 2023-08-11T14:13:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.255+0000 7f806b896b80 10 client.16302 fetch_fsmap learned FSMap version 44 2023-08-11T14:13:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.255+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x564a1d63f1f0 con 0x564a1d89b790 2023-08-11T14:13:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.261+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f804c0949e0 con 0x564a1d89b790 2023-08-11T14:13:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.261+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 44) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8048032ab0 con 0x564a1d89b790 2023-08-11T14:13:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.262+0000 7f806b896b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 44 2023-08-11T14:13:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.262+0000 7f806b896b80 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:13:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.262+0000 7f806b896b80 20 client.16302 populate_metadata read hostname 'smithi003' 2023-08-11T14:13:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.262+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d9041b0 0x564a1d9065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:13:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.263+0000 7f806b896b80 4 client.16302 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:13:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.263+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a1d7e57c0 con 0x564a1d9041b0 2023-08-11T14:13:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.269+0000 7f8055d9b700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d9041b0 0x564a1d9065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:13:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.284+0000 7f8055d9b700 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d9041b0 0x564a1d9065d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318f67d3b595f3c5 server_cookie=c2ba12a5815a3ec in_seq=0 out_seq=0 2023-08-11T14:13:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:45.284+0000 7f80397fa700 10 client.16302 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:46.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:46 smithi103 ceph-mon[130434]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-08-11T14:13:46.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:46 smithi187 ceph-mon[130973]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-08-11T14:13:46.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:46 smithi003 ceph-mon[97195]: pgmap v642: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-08-11T14:13:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.917+0000 7f80397fa700 1 -- 172.21.15.3:0/1359235722 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a1d7e57c0 con 0x564a1d9041b0 2023-08-11T14:13:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.917+0000 7f80397fa700 10 client.16302 handle_command_reply: tid=0 2023-08-11T14:13:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.917+0000 7f806b896b80 1 client.16302 shutdown 2023-08-11T14:13:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 msgr2=0x564a1d8aa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8048009610 tx=0x7f804801bdd0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 shutdown_connections 2023-08-11T14:13:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564a1d89b790 0x564a1d8aa240 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564a1d9041b0 0x564a1d9065d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a1d899f40 0x564a1d8a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 --2- 172.21.15.3:0/1359235722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a1d89a8a0 0x564a1d8a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 >> 172.21.15.3:0/1359235722 conn(0x564a1d7c78f0 msgr2=0x564a1d8962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f80397fa700 0 client.16302 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:13:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.919+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 shutdown_connections 2023-08-11T14:13:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.920+0000 7f806b896b80 1 -- 172.21.15.3:0/1359235722 wait complete. 2023-08-11T14:13:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.921+0000 7f806b896b80 20 client.16302 trim_cache size 0 max 16384 2023-08-11T14:13:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:13:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:13:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:13:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:13:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.925+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 msgr2=0x7f803c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.925+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 0x7f803c0a2610 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8048009fd0 tx=0x7f804800b040 comp rx=0 tx=0).stop 2023-08-11T14:13:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.925+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 msgr2=0x7f8050068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:13:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.925+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b550 tx=0x7f804c00b910 comp rx=0 tx=0).stop 2023-08-11T14:13:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 shutdown_connections 2023-08-11T14:13:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f803c0a0150 0x7f803c0a2610 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80501013b0 0x7f80501ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80500ff820 0x7f8050067f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 --2- 172.21.15.3:0/3755129309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8050100a10 0x7f8050068470 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:13:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.926+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 >> 172.21.15.3:0/3755129309 conn(0x7f80500fad70 msgr2=0x7f8050103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:13:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.927+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 shutdown_connections 2023-08-11T14:13:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:13:46.928+0000 7f8057fff700 1 -- 172.21.15.3:0/3755129309 wait complete. 2023-08-11T14:13:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 410.0 s, 671.4 tps, lat 196.417 ms stddev 447.963 2023-08-11T14:13:48.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:48 smithi103 ceph-mon[130434]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 996 op/s 2023-08-11T14:13:48.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:48 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:48.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:48 smithi003 ceph-mon[97195]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 996 op/s 2023-08-11T14:13:48.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:48 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:48.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:48 smithi187 ceph-mon[130973]: pgmap v643: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 996 op/s 2023-08-11T14:13:48.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:48 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:50.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:50 smithi003 ceph-mon[97195]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:13:50.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:50 smithi187 ceph-mon[130973]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:13:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:50 smithi103 ceph-mon[130434]: pgmap v644: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:13:51.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:51 smithi003 ceph-mon[97195]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-08-11T14:13:51.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:51 smithi187 ceph-mon[130973]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-08-11T14:13:51.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:51 smithi103 ceph-mon[130434]: pgmap v645: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.03k op/s 2023-08-11T14:13:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 415.0 s, 524.7 tps, lat 188.180 ms stddev 372.302 2023-08-11T14:13:53.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:53 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:53.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:53 smithi003 ceph-mon[97195]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.19k op/s 2023-08-11T14:13:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:53 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:53.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:53 smithi187 ceph-mon[130973]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.19k op/s 2023-08-11T14:13:53.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:53 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:13:53.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:53 smithi103 ceph-mon[130434]: pgmap v646: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.19k op/s 2023-08-11T14:13:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:55 smithi187 ceph-mon[130973]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-08-11T14:13:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:55 smithi003 ceph-mon[97195]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-08-11T14:13:56.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:55 smithi103 ceph-mon[130434]: pgmap v647: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-08-11T14:13:58.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 420.0 s, 952.4 tps, lat 83.855 ms stddev 65.608 2023-08-11T14:13:58.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:13:58 smithi003 ceph-mon[97195]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:13:58.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:13:58 smithi187 ceph-mon[130973]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:13:58.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:13:58 smithi103 ceph-mon[130434]: pgmap v648: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:14:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.240+0000 7fd9a01ff700 1 -- 172.21.15.3:0/3703649185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998106f80 msgr2=0x7fd9981073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.240+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/3703649185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998106f80 0x7fd9981073a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd9940099c0 tx=0x7fd994009cd0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.241+0000 7fd9a01ff700 1 -- 172.21.15.3:0/3703649185 shutdown_connections 2023-08-11T14:14:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.241+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/3703649185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108b10 0x7fd99810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.241+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/3703649185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998106f80 0x7fd9981073a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.241+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/3703649185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd998108170 0x7fd9981085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.241+0000 7fd9a01ff700 1 -- 172.21.15.3:0/3703649185 >> 172.21.15.3:0/3703649185 conn(0x7fd998075f00 msgr2=0x7fd998078360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.244+0000 7fd9a01ff700 1 -- 172.21.15.3:0/3703649185 shutdown_connections 2023-08-11T14:14:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.244+0000 7fd9a01ff700 1 -- 172.21.15.3:0/3703649185 wait complete. 2023-08-11T14:14:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.246+0000 7fd9a01ff700 1 Processor -- start 2023-08-11T14:14:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.246+0000 7fd9a01ff700 1 -- start start 2023-08-11T14:14:00.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd998106f80 0x7fd99806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108170 0x7fd99806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998070c00 con 0x7fd998108170 2023-08-11T14:14:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998070d70 con 0x7fd998108b10 2023-08-11T14:14:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.247+0000 7fd9a01ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998070f30 con 0x7fd998106f80 2023-08-11T14:14:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:56758/0 (socket says 172.21.15.3:56758) 2023-08-11T14:14:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 -- 172.21.15.3:0/773707177 learned_addr learned my addr 172.21.15.3:0/773707177 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99d79a700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108170 0x7fd99806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 -- 172.21.15.3:0/773707177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd998106f80 msgr2=0x7fd99806f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd998106f80 0x7fd99806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 -- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108170 msgr2=0x7fd99806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108170 0x7fd99806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.248+0000 7fd99e79c700 1 -- 172.21.15.3:0/773707177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940095f0 con 0x7fd998108b10 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd99e79c700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd990007d70 tx=0x7fd99000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd98effd700 1 -- 172.21.15.3:0/773707177 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000ad40 con 0x7fd998108b10 2023-08-11T14:14:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd98effd700 1 -- 172.21.15.3:0/773707177 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9900128e0 con 0x7fd998108b10 2023-08-11T14:14:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd998071210 con 0x7fd998108b10 2023-08-11T14:14:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd98effd700 1 -- 172.21.15.3:0/773707177 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990011780 con 0x7fd998108b10 2023-08-11T14:14:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.249+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9980719c0 con 0x7fd998108b10 2023-08-11T14:14:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.255+0000 7fd98effd700 1 -- 172.21.15.3:0/773707177 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd990012420 con 0x7fd998108b10 2023-08-11T14:14:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.255+0000 7fd9b3a37b80 1 Processor -- start 2023-08-11T14:14:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 -- start start 2023-08-11T14:14:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5776360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db57768a0 0x561db57771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db56cf5d0 con 0x561db5775f40 2023-08-11T14:14:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db56ccef0 con 0x561db57768a0 2023-08-11T14:14:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.256+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db56c87e0 con 0x561db5777790 2023-08-11T14:14:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.257+0000 7fd99df9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db57768a0 0x561db57771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.257+0000 7fd99d79a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.257+0000 7fd99d79a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:55822/0 (socket says 172.21.15.3:55822) 2023-08-11T14:14:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.257+0000 7fd99d79a700 1 -- 172.21.15.3:0/3932543864 learned_addr learned my addr 172.21.15.3:0/3932543864 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 -- 172.21.15.3:0/3932543864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db57768a0 msgr2=0x561db57771b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db57768a0 0x561db57771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 -- 172.21.15.3:0/3932543864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 msgr2=0x561db5776360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5776360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 -- 172.21.15.3:0/3932543864 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db55485d0 con 0x561db5777790 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd99d79a700 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd988009d30 tx=0x7fd988012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30537c47b0bb3809 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd98effd700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 0x7fd9840a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.258+0000 7fd98effd700 1 -- 172.21.15.3:0/773707177 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd9900cb7f0 con 0x7fd998108b10 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.259+0000 7fd983fff700 1 -- 172.21.15.3:0/3932543864 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988009e80 con 0x561db5777790 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.259+0000 7fd983fff700 1 -- 172.21.15.3:0/3932543864 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98800f8f0 con 0x561db5777790 2023-08-11T14:14:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.259+0000 7fd983fff700 1 -- 172.21.15.3:0/3932543864 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98801a8e0 con 0x561db5777790 2023-08-11T14:14:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.260+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3932543864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 msgr2=0x561db5777c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.260+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd988009d30 tx=0x7fd988012730 comp rx=0 tx=0).stop 2023-08-11T14:14:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.260+0000 7fd99df9b700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 0x7fd9840a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.262+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3932543864 shutdown_connections 2023-08-11T14:14:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.262+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5777c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.262+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5776360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.262+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3932543864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db57768a0 0x561db57771b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.262+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3932543864 >> 172.21.15.3:0/3932543864 conn(0x561db56a38f0 msgr2=0x561db5772920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.263+0000 7fd99df9b700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 0x7fd9840a24e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x561db57722f0 tx=0x7fd994000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.266+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3932543864 shutdown_connections 2023-08-11T14:14:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.266+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3932543864 wait complete. 2023-08-11T14:14:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.267+0000 7fd9b3a37b80 1 -- start start 2023-08-11T14:14:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.267+0000 7fd9b3a37b80 1 Processor -- start 2023-08-11T14:14:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db57801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5666a60 con 0x561db57768a0 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5666760 con 0x561db5775f40 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db554a8c0 con 0x561db5777790 2023-08-11T14:14:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd9b3a37b80 10 client.0 _ll_register_callbacks cb 0x561db5669a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd99df9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd99df9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:56774/0 (socket says 172.21.15.3:56774) 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.268+0000 7fd99df9b700 1 -- 172.21.15.3:0/2345400868 learned_addr learned my addr 172.21.15.3:0/2345400868 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99d79a700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db57801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 -- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 msgr2=0x561db577fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 -- 172.21.15.3:0/2345400868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 msgr2=0x561db57801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db57801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.269+0000 7fd99e79c700 1 -- 172.21.15.3:0/2345400868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9880099e0 con 0x561db5777790 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.270+0000 7fd99e79c700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd9900c9020 tx=0x7fd990098fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30cbbe45dd438796 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.270+0000 7fd9817fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.270+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990098410 con 0x561db5777790 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.270+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd990098570 con 0x561db5777790 2023-08-11T14:14:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.270+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561db57697e0 con 0x561db5777790 2023-08-11T14:14:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.271+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9900a08a0 con 0x561db5777790 2023-08-11T14:14:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.271+0000 7fd9b3a37b80 10 client.26452 fetch_fsmap learned FSMap version 46 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.271+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561db54f4d80 con 0x561db5777790 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.271+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561db57697e0 con 0x561db5777790 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.271+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 46) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd990094d10 con 0x561db5777790 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 46 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 20 client.26452 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db579cb80 0x561db579efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 4 client.26452 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:14:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561db5745480 con 0x561db579cb80 2023-08-11T14:14:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.272+0000 7fd99df9b700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db579cb80 0x561db579efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.273+0000 7fd99df9b700 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db579cb80 0x561db579efa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d366c3a6b1d62af6 server_cookie=33b12282961d55 in_seq=0 out_seq=0 2023-08-11T14:14:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.273+0000 7fd9817fa700 10 client.26452 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.274+0000 7fd9817fa700 1 -- 172.21.15.3:0/2345400868 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561db5745480 con 0x561db579cb80 2023-08-11T14:14:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.274+0000 7fd9817fa700 10 client.26452 handle_command_reply: tid=0 2023-08-11T14:14:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.274+0000 7fd9b3a37b80 1 client.26452 shutdown 2023-08-11T14:14:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 msgr2=0x561db5787200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd9900c9020 tx=0x7fd990098fa0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 shutdown_connections 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db579cb80 0x561db579efa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/2345400868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db57801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 >> 172.21.15.3:0/2345400868 conn(0x561db56a38f0 msgr2=0x561db57716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9817fa700 0 client.26452 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.275+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 shutdown_connections 2023-08-11T14:14:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.276+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/2345400868 wait complete. 2023-08-11T14:14:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.276+0000 7fd9b3a37b80 20 client.26452 trim_cache size 0 max 16384 2023-08-11T14:14:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.315+0000 7fd9b3a37b80 1 Processor -- start 2023-08-11T14:14:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.315+0000 7fd9b3a37b80 1 -- start start 2023-08-11T14:14:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.315+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.315+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.316+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.316+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5799e80 con 0x561db57768a0 2023-08-11T14:14:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.316+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db57a3d80 con 0x561db5775f40 2023-08-11T14:14:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.316+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5776fa0 con 0x561db5777790 2023-08-11T14:14:00.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.317+0000 7fd99d79a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.317+0000 7fd99d79a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44910/0 (socket says 172.21.15.3:44910) 2023-08-11T14:14:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.317+0000 7fd99d79a700 1 -- 172.21.15.3:0/542053527 learned_addr learned my addr 172.21.15.3:0/542053527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.318+0000 7fd99d79a700 1 -- 172.21.15.3:0/542053527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 msgr2=0x561db5787120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.318+0000 7fd99d79a700 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.318+0000 7fd99d79a700 1 -- 172.21.15.3:0/542053527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 msgr2=0x561db577a670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.318+0000 7fd99d79a700 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577a670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.318+0000 7fd99d79a700 1 -- 172.21.15.3:0/542053527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db54ddc00 con 0x561db57768a0 2023-08-11T14:14:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.319+0000 7fd99d79a700 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd988009cd0 tx=0x7fd988005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb56af5202c8710e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.333+0000 7fd98cff9700 1 -- 172.21.15.3:0/542053527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988009e80 con 0x561db57768a0 2023-08-11T14:14:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.333+0000 7fd98cff9700 1 -- 172.21.15.3:0/542053527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98800fa60 con 0x561db57768a0 2023-08-11T14:14:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.333+0000 7fd98cff9700 1 -- 172.21.15.3:0/542053527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988003680 con 0x561db57768a0 2023-08-11T14:14:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.334+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/542053527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 msgr2=0x561db577abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.334+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd988009cd0 tx=0x7fd988005d40 comp rx=0 tx=0).stop 2023-08-11T14:14:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.356+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/542053527 shutdown_connections 2023-08-11T14:14:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.356+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db5777790 0x561db5787120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.356+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5775f40 0x561db577a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.356+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/542053527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db57768a0 0x561db577abb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.356+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/542053527 >> 172.21.15.3:0/542053527 conn(0x561db56a38f0 msgr2=0x561db577fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.366+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/542053527 shutdown_connections 2023-08-11T14:14:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.372+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/542053527 wait complete. 2023-08-11T14:14:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.373+0000 7fd9b3a37b80 1 -- start start 2023-08-11T14:14:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.373+0000 7fd9b3a37b80 1 Processor -- start 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5783250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db57768a0 0x561db5783790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5786cf0 con 0x561db5775f40 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5786e60 con 0x561db5777790 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5787020 con 0x561db57768a0 2023-08-11T14:14:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.383+0000 7fd9b3a37b80 10 client.0 _ll_register_callbacks cb 0x7fd990098880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.387+0000 7fd99e79c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.387+0000 7fd99e79c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:56790/0 (socket says 172.21.15.3:56790) 2023-08-11T14:14:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.387+0000 7fd99e79c700 1 -- 172.21.15.3:0/3104137138 learned_addr learned my addr 172.21.15.3:0/3104137138 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.388+0000 7fd99e79c700 1 -- 172.21.15.3:0/3104137138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db57768a0 msgr2=0x561db5783790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.388+0000 7fd99e79c700 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db57768a0 0x561db5783790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.388+0000 7fd99e79c700 1 -- 172.21.15.3:0/3104137138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 msgr2=0x561db5783250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.388+0000 7fd99e79c700 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5783250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.388+0000 7fd99e79c700 1 -- 172.21.15.3:0/3104137138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98801f040 con 0x561db5777790 2023-08-11T14:14:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.392+0000 7fd99e79c700 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd990098eb0 tx=0x7fd9900974d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638e4c08980c0252 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.392+0000 7fd9817fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:14:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.395+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990098610 con 0x561db5777790 2023-08-11T14:14:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.395+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd990004b50 con 0x561db5777790 2023-08-11T14:14:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.395+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd98800f8f0 con 0x561db5777790 2023-08-11T14:14:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.396+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd990004d30 con 0x561db5777790 2023-08-11T14:14:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.396+0000 7fd9b3a37b80 10 client.26435 fetch_fsmap learned FSMap version 46 2023-08-11T14:14:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.396+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561db551b1f0 con 0x561db5777790 2023-08-11T14:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98800f8f0 con 0x561db5777790 2023-08-11T14:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd990012a50 con 0x561db5777790 2023-08-11T14:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 10 client.26435 fetch_fsmap finished waiting for FSMap version 46 2023-08-11T14:14:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 20 client.26435 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db57f19a0 0x561db57f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 4 client.26435 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:14:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.397+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561db56c17c0 con 0x561db57f19a0 2023-08-11T14:14:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.404+0000 7fd99df9b700 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db57f19a0 0x561db57f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.412+0000 7fd99df9b700 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db57f19a0 0x561db57f3dc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce8037ef4de6262 server_cookie=e700b9ab253beb84 in_seq=0 out_seq=0 2023-08-11T14:14:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.412+0000 7fd9817fa700 10 client.26435 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.480+0000 7fd9817fa700 1 -- 172.21.15.3:0/3104137138 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561db56c17c0 con 0x561db57f19a0 2023-08-11T14:14:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.480+0000 7fd9817fa700 10 client.26435 handle_command_reply: tid=0 2023-08-11T14:14:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.481+0000 7fd9b3a37b80 1 client.26435 shutdown 2023-08-11T14:14:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 msgr2=0x561db57867b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd990098eb0 tx=0x7fd9900974d0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 shutdown_connections 2023-08-11T14:14:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561db57f19a0 0x561db57f3dc0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561db5777790 0x561db57867b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561db5775f40 0x561db5783250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 --2- 172.21.15.3:0/3104137138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db57768a0 0x561db5783790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 >> 172.21.15.3:0/3104137138 conn(0x561db56a38f0 msgr2=0x561db577fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9817fa700 0 client.26435 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.484+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 shutdown_connections 2023-08-11T14:14:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.485+0000 7fd9b3a37b80 1 -- 172.21.15.3:0/3104137138 wait complete. 2023-08-11T14:14:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.485+0000 7fd9b3a37b80 20 client.26435 trim_cache size 0 max 16384 2023-08-11T14:14:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:14:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:14:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:14:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:14:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 msgr2=0x7fd9840a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 0x7fd9840a24e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x561db57722f0 tx=0x7fd994000bc0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 msgr2=0x7fd9980705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd990007d70 tx=0x7fd99000a340 comp rx=0 tx=0).stop 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 shutdown_connections 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd9840a0020 0x7fd9840a24e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd998108b10 0x7fd9980705a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd998106f80 0x7fd99806f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 --2- 172.21.15.3:0/773707177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd998108170 0x7fd99806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.486+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 >> 172.21.15.3:0/773707177 conn(0x7fd998075f00 msgr2=0x7fd9980780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.487+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 shutdown_connections 2023-08-11T14:14:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:00.488+0000 7fd9a01ff700 1 -- 172.21.15.3:0/773707177 wait complete. 2023-08-11T14:14:00.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:00 smithi003 ceph-mon[97195]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:14:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:00 smithi187 ceph-mon[130973]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:14:00.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:00 smithi103 ceph-mon[130434]: pgmap v649: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 18 MiB/s wr, 1.16k op/s 2023-08-11T14:14:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:02 smithi103 ceph-mon[130434]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-08-11T14:14:02.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:02 smithi003 ceph-mon[97195]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-08-11T14:14:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:02 smithi187 ceph-mon[130973]: pgmap v650: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-08-11T14:14:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 425.0 s, 859.3 tps, lat 141.329 ms stddev 321.627 2023-08-11T14:14:04.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:04 smithi103 ceph-mon[130434]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:04.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:04 smithi003 ceph-mon[97195]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:04.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:04 smithi187 ceph-mon[130973]: pgmap v651: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:05.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:05 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:05.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:05 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:05.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:05 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:06.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:06 smithi003 ceph-mon[97195]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.18k op/s 2023-08-11T14:14:06.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:06 smithi187 ceph-mon[130973]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.18k op/s 2023-08-11T14:14:06.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:06 smithi103 ceph-mon[130434]: pgmap v652: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 18 MiB/s wr, 1.18k op/s 2023-08-11T14:14:07.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:07 smithi103 ceph-mon[130434]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-08-11T14:14:08.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:07 smithi003 ceph-mon[97195]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-08-11T14:14:08.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:07 smithi187 ceph-mon[130973]: pgmap v653: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.23k op/s 2023-08-11T14:14:08.094 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 430.0 s, 590.9 tps, lat 167.139 ms stddev 391.053 2023-08-11T14:14:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:10 smithi103 ceph-mon[130434]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:14:10.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:10 smithi003 ceph-mon[97195]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:14:10.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:10 smithi187 ceph-mon[130973]: pgmap v654: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:14:12.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:12 smithi103 ceph-mon[130434]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-08-11T14:14:12.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:12 smithi003 ceph-mon[97195]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-08-11T14:14:12.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:12 smithi187 ceph-mon[130973]: pgmap v655: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.34k op/s 2023-08-11T14:14:13.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 435.0 s, 777.6 tps, lat 128.140 ms stddev 294.146 2023-08-11T14:14:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.898+0000 7f456b5ff700 1 -- 172.21.15.3:0/1725968446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f456410a290 msgr2=0x7f456410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.898+0000 7f456b5ff700 1 --2- 172.21.15.3:0/1725968446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f456410a290 0x7f456410c720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f45540098f0 tx=0x7f4554009c00 comp rx=0 tx=0).stop 2023-08-11T14:14:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.906+0000 7f456b5ff700 1 -- 172.21.15.3:0/1725968446 shutdown_connections 2023-08-11T14:14:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.906+0000 7f456b5ff700 1 --2- 172.21.15.3:0/1725968446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 0x7f456410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.906+0000 7f456b5ff700 1 --2- 172.21.15.3:0/1725968446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4564107920 0x7f4564109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.906+0000 7f456b5ff700 1 --2- 172.21.15.3:0/1725968446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f456410a290 0x7f456410c720 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.906+0000 7f456b5ff700 1 -- 172.21.15.3:0/1725968446 >> 172.21.15.3:0/1725968446 conn(0x7f456406d750 msgr2=0x7f456406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.920+0000 7f456b5ff700 1 -- 172.21.15.3:0/1725968446 shutdown_connections 2023-08-11T14:14:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.923+0000 7f456b5ff700 1 -- 172.21.15.3:0/1725968446 wait complete. 2023-08-11T14:14:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.924+0000 7f456b5ff700 1 Processor -- start 2023-08-11T14:14:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.933+0000 7f456b5ff700 1 -- start start 2023-08-11T14:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456410a290 0x7f456419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 0x7f456419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456419e790 con 0x7f456410a290 2023-08-11T14:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456419e900 con 0x7f456410cc60 2023-08-11T14:14:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.934+0000 7f456b5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456419eac0 con 0x7f4564107920 2023-08-11T14:14:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.942+0000 7f456939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.942+0000 7f456939b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34254/0 (socket says 172.21.15.3:34254) 2023-08-11T14:14:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.942+0000 7f456939b700 1 -- 172.21.15.3:0/2635134591 learned_addr learned my addr 172.21.15.3:0/2635134591 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.942+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 0x7f456419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f4568b9a700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456410a290 0x7f456419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f456939b700 1 -- 172.21.15.3:0/2635134591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 msgr2=0x7f456419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f456939b700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 0x7f456419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f456939b700 1 -- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456410a290 msgr2=0x7f456419d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f456939b700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456410a290 0x7f456419d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.950+0000 7f456939b700 1 -- 172.21.15.3:0/2635134591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009520 con 0x7f4564107920 2023-08-11T14:14:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.964+0000 7f456939b700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f455c01d0c0 tx=0x7f455c01d480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.974+0000 7f455a7fc700 1 -- 172.21.15.3:0/2635134591 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c029070 con 0x7f4564107920 2023-08-11T14:14:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.983+0000 7f455a7fc700 1 -- 172.21.15.3:0/2635134591 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f455c020040 con 0x7f4564107920 2023-08-11T14:14:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.983+0000 7f455a7fc700 1 -- 172.21.15.3:0/2635134591 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c01f020 con 0x7f4564107920 2023-08-11T14:14:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.989+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456419ed40 con 0x7f4564107920 2023-08-11T14:14:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.989+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641a54b0 con 0x7f4564107920 2023-08-11T14:14:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:13.995+0000 7f457ef0ab80 1 Processor -- start 2023-08-11T14:14:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 -- start start 2023-08-11T14:14:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d183021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18302c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d1825a5d0 con 0x564d18300f40 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d18257ef0 con 0x564d18302790 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.002+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d182537e0 con 0x564d183018a0 2023-08-11T14:14:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.008+0000 7f4568b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18302c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.020+0000 7f4568b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18302c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53312/0 (socket says 172.21.15.3:53312) 2023-08-11T14:14:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.020+0000 7f4568b9a700 1 -- 172.21.15.3:0/2826553553 learned_addr learned my addr 172.21.15.3:0/2826553553 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.021+0000 7f456939b700 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d183021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.021+0000 7f455a7fc700 1 -- 172.21.15.3:0/2635134591 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f455c01a0d0 con 0x7f4564107920 2023-08-11T14:14:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.023+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.024+0000 7f455a7fc700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 0x7f45500a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.024+0000 7f455a7fc700 1 -- 172.21.15.3:0/2635134591 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f455c0ddf80 con 0x7f4564107920 2023-08-11T14:14:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.025+0000 7f4569b9c700 1 -- 172.21.15.3:0/2826553553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 msgr2=0x564d183021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.025+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d183021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.025+0000 7f4569b9c700 1 -- 172.21.15.3:0/2826553553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 msgr2=0x564d18302c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.025+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18302c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.025+0000 7f4569b9c700 1 -- 172.21.15.3:0/2826553553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d180d35d0 con 0x564d18300f40 2023-08-11T14:14:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.034+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 0x7f45500a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.035+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4560009c10 tx=0x7f4560009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8efb31ca037b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.038+0000 7f454f7fe700 1 -- 172.21.15.3:0/2826553553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560004040 con 0x564d18300f40 2023-08-11T14:14:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.038+0000 7f454f7fe700 1 -- 172.21.15.3:0/2826553553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4560003680 con 0x564d18300f40 2023-08-11T14:14:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.038+0000 7f454f7fe700 1 -- 172.21.15.3:0/2826553553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560014750 con 0x564d18300f40 2023-08-11T14:14:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.039+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2826553553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 msgr2=0x564d18301360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.039+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4560009c10 tx=0x7f4560009f20 comp rx=0 tx=0).stop 2023-08-11T14:14:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.045+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 0x7f45500a2610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4560020000 tx=0x7f45600148b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.069+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2826553553 shutdown_connections 2023-08-11T14:14:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.069+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18302c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.069+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301360 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.069+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2826553553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d183021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.069+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2826553553 >> 172.21.15.3:0/2826553553 conn(0x564d1822e8f0 msgr2=0x564d182fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.073+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2826553553 shutdown_connections 2023-08-11T14:14:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.081+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2826553553 wait complete. 2023-08-11T14:14:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.081+0000 7f457ef0ab80 1 -- start start 2023-08-11T14:14:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.082+0000 7f457ef0ab80 1 Processor -- start 2023-08-11T14:14:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.085+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18300f40 0x564d1830ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.085+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d1830b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.085+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 0x564d18312110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.086+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d181f1a60 con 0x564d18302790 2023-08-11T14:14:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.086+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d181f1760 con 0x564d18300f40 2023-08-11T14:14:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.086+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d180d58c0 con 0x564d183018a0 2023-08-11T14:14:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.086+0000 7f457ef0ab80 10 client.0 _ll_register_callbacks cb 0x564d181f4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.099+0000 7f4568b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d1830b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.099+0000 7f4568b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d1830b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34278/0 (socket says 172.21.15.3:34278) 2023-08-11T14:14:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.099+0000 7f4568b9a700 1 -- 172.21.15.3:0/2204168139 learned_addr learned my addr 172.21.15.3:0/2204168139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.105+0000 7f456939b700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18300f40 0x564d1830ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.108+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 0x564d18312110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 -- 172.21.15.3:0/2204168139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 msgr2=0x564d1830b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d1830b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 -- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18300f40 msgr2=0x564d1830ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18300f40 0x564d1830ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 -- 172.21.15.3:0/2204168139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45600097d0 con 0x564d18302790 2023-08-11T14:14:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.109+0000 7f4569b9c700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 0x564d18312110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4560009c10 tx=0x7f456000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983950a7d4d9b040 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.110+0000 7f454cff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:14:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.134+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456002d070 con 0x564d18302790 2023-08-11T14:14:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.134+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4560011cd0 con 0x564d18302790 2023-08-11T14:14:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.135+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d182f47e0 con 0x564d18302790 2023-08-11T14:14:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.135+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560014b40 con 0x564d18302790 2023-08-11T14:14:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.135+0000 7f457ef0ab80 10 client.16323 fetch_fsmap learned FSMap version 47 2023-08-11T14:14:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.135+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x564d1807fd80 con 0x564d18302790 2023-08-11T14:14:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d182f47e0 con 0x564d18302790 2023-08-11T14:14:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 47) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f456000fc00 con 0x564d18302790 2023-08-11T14:14:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 10 client.16323 fetch_fsmap finished waiting for FSMap version 47 2023-08-11T14:14:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 20 client.16323 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d18316aa0 0x564d18318ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 4 client.16323 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:14:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.136+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d182d0480 con 0x564d18316aa0 2023-08-11T14:14:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.153+0000 7f456939b700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d18316aa0 0x564d18318ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.182+0000 7f456939b700 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d18316aa0 0x564d18318ec0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8c965a2f943816 server_cookie=281782db1ec290ba in_seq=0 out_seq=0 2023-08-11T14:14:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.182+0000 7f454cff9700 10 client.16323 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.190+0000 7f454cff9700 1 -- 172.21.15.3:0/2204168139 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x564d182d0480 con 0x564d18316aa0 2023-08-11T14:14:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.190+0000 7f454cff9700 10 client.16323 handle_command_reply: tid=0 2023-08-11T14:14:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.196+0000 7f457ef0ab80 1 client.16323 shutdown 2023-08-11T14:14:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.196+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 msgr2=0x564d18312110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.196+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 0x564d18312110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4560009c10 tx=0x7f456000f000 comp rx=0 tx=0).stop 2023-08-11T14:14:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 shutdown_connections 2023-08-11T14:14:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18302790 0x564d18312110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d18316aa0 0x564d18318ec0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18300f40 0x564d1830ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/2204168139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d1830b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 >> 172.21.15.3:0/2204168139 conn(0x564d1822e8f0 msgr2=0x564d182fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.197+0000 7f454cff9700 0 client.16323 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.203+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 shutdown_connections 2023-08-11T14:14:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.210+0000 7f457ef0ab80 1 -- 172.21.15.3:0/2204168139 wait complete. 2023-08-11T14:14:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.217+0000 7f457ef0ab80 20 client.16323 trim_cache size 0 max 16384 2023-08-11T14:14:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.254+0000 7f457ef0ab80 1 Processor -- start 2023-08-11T14:14:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.254+0000 7f457ef0ab80 1 -- start start 2023-08-11T14:14:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.254+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.254+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d18302b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.255+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18312030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.255+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d182f47c0 con 0x564d18300f40 2023-08-11T14:14:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.255+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d182f6bd0 con 0x564d18302790 2023-08-11T14:14:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.255+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d18308880 con 0x564d183018a0 2023-08-11T14:14:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.266+0000 7f4569b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18312030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.267+0000 7f4568b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d18302b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.271+0000 7f456939b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.271+0000 7f456939b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51520/0 (socket says 172.21.15.3:51520) 2023-08-11T14:14:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.271+0000 7f456939b700 1 -- 172.21.15.3:0/3380796000 learned_addr learned my addr 172.21.15.3:0/3380796000 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.279+0000 7f4568b9a700 1 -- 172.21.15.3:0/3380796000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 msgr2=0x564d18312030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.279+0000 7f4568b9a700 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18312030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.279+0000 7f4568b9a700 1 -- 172.21.15.3:0/3380796000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 msgr2=0x564d18301320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.279+0000 7f4568b9a700 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.279+0000 7f4568b9a700 1 -- 172.21.15.3:0/3380796000 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d18068c00 con 0x564d183018a0 2023-08-11T14:14:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.300+0000 7f4568b9a700 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d18302b70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f4554005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7162d19457fdedd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.318+0000 7f454ffff700 1 -- 172.21.15.3:0/3380796000 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455401d040 con 0x564d183018a0 2023-08-11T14:14:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.318+0000 7f454ffff700 1 -- 172.21.15.3:0/3380796000 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45540058e0 con 0x564d183018a0 2023-08-11T14:14:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.318+0000 7f454ffff700 1 -- 172.21.15.3:0/3380796000 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400ba60 con 0x564d183018a0 2023-08-11T14:14:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.319+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3380796000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 msgr2=0x564d18302b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.319+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d18302b70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4554000c00 tx=0x7f4554005ce0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.331+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3380796000 shutdown_connections 2023-08-11T14:14:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.331+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d18312030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.331+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d18300f40 0x564d18301320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.331+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3380796000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d183018a0 0x564d18302b70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.331+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3380796000 >> 172.21.15.3:0/3380796000 conn(0x564d1822e8f0 msgr2=0x564d182f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.363+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3380796000 shutdown_connections 2023-08-11T14:14:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.370+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3380796000 wait complete. 2023-08-11T14:14:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.370+0000 7f457ef0ab80 1 -- start start 2023-08-11T14:14:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 Processor -- start 2023-08-11T14:14:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d18300f40 0x564d1830fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d183018a0 0x564d183100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d18311620 con 0x564d183018a0 2023-08-11T14:14:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d18311790 con 0x564d18302790 2023-08-11T14:14:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d18311950 con 0x564d18300f40 2023-08-11T14:14:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.371+0000 7f457ef0ab80 10 client.0 _ll_register_callbacks cb 0x7f4560005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.372+0000 7f4569b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.372+0000 7f4569b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53332/0 (socket says 172.21.15.3:53332) 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.372+0000 7f4569b9c700 1 -- 172.21.15.3:0/3412058576 learned_addr learned my addr 172.21.15.3:0/3412058576 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.373+0000 7f4569b9c700 1 -- 172.21.15.3:0/3412058576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d18300f40 msgr2=0x564d1830fb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.373+0000 7f4569b9c700 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d18300f40 0x564d1830fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.373+0000 7f4569b9c700 1 -- 172.21.15.3:0/3412058576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d183018a0 msgr2=0x564d183100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.373+0000 7f4569b9c700 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d183018a0 0x564d183100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.373+0000 7f4569b9c700 1 -- 172.21.15.3:0/3412058576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456410dc60 con 0x564d18302790 2023-08-11T14:14:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.379+0000 7f4569b9c700 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4560021040 tx=0x7f45600147c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68b8a038bf86789 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.379+0000 7f454cff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:14:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.381+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456002d070 con 0x564d18302790 2023-08-11T14:14:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.381+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4560011e80 con 0x564d18302790 2023-08-11T14:14:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.381+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4560003680 con 0x564d18302790 2023-08-11T14:14:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.410+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560014b40 con 0x564d18302790 2023-08-11T14:14:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.410+0000 7f457ef0ab80 10 client.26453 fetch_fsmap learned FSMap version 47 2023-08-11T14:14:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.410+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x564d180a61f0 con 0x564d18302790 2023-08-11T14:14:14.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.426+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4560003680 con 0x564d18302790 2023-08-11T14:14:14.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.426+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f456000f930 con 0x564d18302790 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 10 client.26453 fetch_fsmap finished waiting for FSMap version 47 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 20 client.26453 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d1836b250 0x564d1836d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 4 client.26453 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:14:14.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.427+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564d1824c7c0 con 0x564d1836b250 2023-08-11T14:14:14.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.431+0000 7f456939b700 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d1836b250 0x564d1836d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.474+0000 7f456939b700 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d1836b250 0x564d1836d670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26a41d1d268f570 server_cookie=bbf3c7b193aa9d70 in_seq=0 out_seq=0 2023-08-11T14:14:14.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:14.475+0000 7f454cff9700 10 client.26453 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:14.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:14 smithi103 ceph-mon[130434]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:14:14.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:14 smithi003 ceph-mon[97195]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:14:14.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:14 smithi187 ceph-mon[130973]: pgmap v656: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:14:15.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:15 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:15.805 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:15 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:15.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:15 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:14:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.078+0000 7f454cff9700 1 -- 172.21.15.3:0/3412058576 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564d1824c7c0 con 0x564d1836b250 2023-08-11T14:14:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.078+0000 7f454cff9700 10 client.26453 handle_command_reply: tid=0 2023-08-11T14:14:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.080+0000 7f457ef0ab80 1 client.26453 shutdown 2023-08-11T14:14:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 msgr2=0x564d183110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4560021040 tx=0x7f45600147c0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 shutdown_connections 2023-08-11T14:14:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x564d1836b250 0x564d1836d670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d18302790 0x564d183110e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d18300f40 0x564d1830fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 --2- 172.21.15.3:0/3412058576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564d183018a0 0x564d183100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.083+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 >> 172.21.15.3:0/3412058576 conn(0x564d1822e8f0 msgr2=0x564d18305610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.085+0000 7f454cff9700 0 client.26453 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.089+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 shutdown_connections 2023-08-11T14:14:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.095+0000 7f457ef0ab80 1 -- 172.21.15.3:0/3412058576 wait complete. 2023-08-11T14:14:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.096+0000 7f457ef0ab80 20 client.26453 trim_cache size 0 max 16384 2023-08-11T14:14:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:14:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:14:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:14:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:14:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 msgr2=0x7f45500a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 0x7f45500a2610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4560020000 tx=0x7f45600148b0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 msgr2=0x7f456419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f455c01d0c0 tx=0x7f455c01d480 comp rx=0 tx=0).stop 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 shutdown_connections 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f45500a0150 0x7f45500a2610 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f456410cc60 0x7f456419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4564107920 0x7f456419cda0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f455c01d0c0 tx=0x7f455c01d480 comp rx=0 tx=0).stop 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 --2- 172.21.15.3:0/2635134591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f456410a290 0x7f456419d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.101+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 >> 172.21.15.3:0/2635134591 conn(0x7f456406d750 msgr2=0x7f456410aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.112+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 shutdown_connections 2023-08-11T14:14:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:16.112+0000 7f456b5ff700 1 -- 172.21.15.3:0/2635134591 wait complete. 2023-08-11T14:14:16.932 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:16 smithi103 ceph-mon[130434]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:14:17.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:16 smithi187 ceph-mon[130973]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:14:17.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:16 smithi003 ceph-mon[97195]: pgmap v657: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:14:17.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:17 smithi103 ceph-mon[130434]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:14:18.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:17 smithi187 ceph-mon[130973]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:14:18.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:17 smithi003 ceph-mon[97195]: pgmap v658: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:14:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 440.0 s, 365.1 tps, lat 278.449 ms stddev 469.588 2023-08-11T14:14:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:20 smithi103 ceph-mon[130434]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:14:20.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:20 smithi187 ceph-mon[130973]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:14:20.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:20 smithi003 ceph-mon[97195]: pgmap v659: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 1.45k op/s 2023-08-11T14:14:22.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:22 smithi103 ceph-mon[130434]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:14:22.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:22 smithi187 ceph-mon[130973]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:14:22.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:22 smithi003 ceph-mon[97195]: pgmap v660: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:14:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 445.0 s, 349.4 tps, lat 283.596 ms stddev 297.862 2023-08-11T14:14:24.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:24 smithi103 ceph-mon[130434]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:24.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:24 smithi187 ceph-mon[130973]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:24.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:24 smithi003 ceph-mon[97195]: pgmap v661: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:14:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:26 smithi103 ceph-mon[130434]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:14:26.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:26 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:14:26.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:26 smithi187 ceph-mon[130973]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:14:26.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:26 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:14:26.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:26 smithi003 ceph-mon[97195]: pgmap v662: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 16 MiB/s wr, 1.15k op/s 2023-08-11T14:14:26.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:26 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:14:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 450.0 s, 336.7 tps, lat 294.330 ms stddev 281.819 2023-08-11T14:14:28.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:28 smithi003 ceph-mon[97195]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:14:28.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:28 smithi187 ceph-mon[130973]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:14:28.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:28 smithi103 ceph-mon[130434]: pgmap v663: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:14:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.418+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/3254306821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f34107920 msgr2=0x7f9f34109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.418+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/3254306821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f34107920 0x7f9f34109d50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9f3001a9a0 tx=0x7f9f3001acb0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.420+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/3254306821 shutdown_connections 2023-08-11T14:14:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.420+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/3254306821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3410cc60 0x7f9f3410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.420+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/3254306821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f34107920 0x7f9f34109d50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.420+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/3254306821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f3410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.420+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/3254306821 >> 172.21.15.3:0/3254306821 conn(0x7f9f3406d750 msgr2=0x7f9f3406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.421+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/3254306821 shutdown_connections 2023-08-11T14:14:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.421+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/3254306821 wait complete. 2023-08-11T14:14:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 Processor -- start 2023-08-11T14:14:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 -- start start 2023-08-11T14:14:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 0x7f9f34117d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f34118270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3410cc60 0x7f9f341c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34118bd0 con 0x7f9f3410cc60 2023-08-11T14:14:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f341c6b30 con 0x7f9f3410a290 2023-08-11T14:14:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.423+0000 7f9f3c5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f341c6cf0 con 0x7f9f34107920 2023-08-11T14:14:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.424+0000 7f9f39b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f34118270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.424+0000 7f9f39b9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f34118270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47008/0 (socket says 172.21.15.3:47008) 2023-08-11T14:14:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.424+0000 7f9f39b9a700 1 -- 172.21.15.3:0/1485072525 learned_addr learned my addr 172.21.15.3:0/1485072525 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.424+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3410cc60 0x7f9f341c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 0x7f9f34117d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 -- 172.21.15.3:0/1485072525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 msgr2=0x7f9f34118270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f34118270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 -- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3410cc60 msgr2=0x7f9f341c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3410cc60 0x7f9f341c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 -- 172.21.15.3:0/1485072525 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3001a600 con 0x7f9f34107920 2023-08-11T14:14:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.425+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 0x7f9f34117d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f3001f620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.426+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/1485072525 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3001fc10 con 0x7f9f34107920 2023-08-11T14:14:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.426+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/1485072525 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f30003e20 con 0x7f9f34107920 2023-08-11T14:14:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.426+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/1485072525 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000e3e0 con 0x7f9f34107920 2023-08-11T14:14:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.426+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f341c6f70 con 0x7f9f34107920 2023-08-11T14:14:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.426+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f341c72e0 con 0x7f9f34107920 2023-08-11T14:14:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.433+0000 7f9f4fe94b80 1 Processor -- start 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 -- start start 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed8b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed8bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ece35d0 con 0x55848ed8a8a0 2023-08-11T14:14:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ece0ef0 con 0x55848ed89f40 2023-08-11T14:14:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.436+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ecdc7e0 con 0x55848ed8b790 2023-08-11T14:14:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.439+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/1485072525 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9f3000e540 con 0x7f9f34107920 2023-08-11T14:14:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47020/0 (socket says 172.21.15.3:47020) 2023-08-11T14:14:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/4044705611 learned_addr learned my addr 172.21.15.3:0/4044705611 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/4044705611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 msgr2=0x55848ed8bc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed8bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/4044705611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 msgr2=0x55848ed8b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.440+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed8b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.441+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/4044705611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55848eb5c5d0 con 0x55848ed89f40 2023-08-11T14:14:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.441+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098f0 tx=0x7f9f2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ccbf804931876a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.441+0000 7f9f28bfe700 1 -- 172.21.15.3:0/4044705611 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c019070 con 0x55848ed89f40 2023-08-11T14:14:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.442+0000 7f9f28bfe700 1 -- 172.21.15.3:0/4044705611 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f2c00f040 con 0x55848ed89f40 2023-08-11T14:14:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.442+0000 7f9f28bfe700 1 -- 172.21.15.3:0/4044705611 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c007c70 con 0x55848ed89f40 2023-08-11T14:14:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.443+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/4044705611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 msgr2=0x55848ed8a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.443+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098f0 tx=0x7f9f2c009c00 comp rx=0 tx=0).stop 2023-08-11T14:14:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.446+0000 7f9f2b7fe700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 0x7f9f200a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.446+0000 7f9f2b7fe700 1 -- 172.21.15.3:0/1485072525 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f300cd6d0 con 0x7f9f34107920 2023-08-11T14:14:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.447+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/4044705611 shutdown_connections 2023-08-11T14:14:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.447+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed8bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.447+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.447+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/4044705611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed8b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.447+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/4044705611 >> 172.21.15.3:0/4044705611 conn(0x55848ecb78f0 msgr2=0x55848ed86920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.449+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 0x7f9f200a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.453+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/4044705611 shutdown_connections 2023-08-11T14:14:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.456+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/4044705611 wait complete. 2023-08-11T14:14:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.457+0000 7f9f4fe94b80 1 -- start start 2023-08-11T14:14:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.458+0000 7f9f4fe94b80 1 Processor -- start 2023-08-11T14:14:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.463+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 0x7f9f200a2610 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000f80 tx=0x7f9f2c007500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed89f40 0x55848ed93cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed8a8a0 0x55848ed94200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ec7aa60 con 0x55848ed89f40 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ec7a760 con 0x55848ed8a8a0 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848eb5e8c0 con 0x55848ed8b790 2023-08-11T14:14:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.473+0000 7f9f4fe94b80 10 client.0 _ll_register_callbacks cb 0x55848ec7da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.480+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.480+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47070/0 (socket says 172.21.15.3:47070) 2023-08-11T14:14:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.480+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/106587996 learned_addr learned my addr 172.21.15.3:0/106587996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.480+0000 7f9f39b9a700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed8a8a0 0x55848ed94200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.481+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed8a8a0 msgr2=0x55848ed94200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.481+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed8a8a0 0x55848ed94200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.481+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/106587996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed89f40 msgr2=0x55848ed93cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.481+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed89f40 0x55848ed93cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.481+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/106587996 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c003c50 con 0x55848ed8b790 2023-08-11T14:14:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.482+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c018000 tx=0x7f9f2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c38f8eb4fdbe90 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.482+0000 7f9f1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:14:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.489+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c019070 con 0x55848ed8b790 2023-08-11T14:14:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.489+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f2c007e30 con 0x55848ed8b790 2023-08-11T14:14:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.490+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55848ed7d7e0 con 0x55848ed8b790 2023-08-11T14:14:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.501+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2c015c20 con 0x55848ed8b790 2023-08-11T14:14:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.502+0000 7f9f4fe94b80 10 client.26476 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.502+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55848eb08d80 con 0x55848ed8b790 2023-08-11T14:14:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.513+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55848ed7d7e0 con 0x55848ed8b790 2023-08-11T14:14:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.513+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9f2c029440 con 0x55848ed8b790 2023-08-11T14:14:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 10 client.26476 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:29.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 10 client.26476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 20 client.26476 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848ed9fc10 0x55848eda2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 4 client.26476 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:14:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.514+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55848ed59480 con 0x55848ed9fc10 2023-08-11T14:14:29.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.516+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848ed9fc10 0x55848eda2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.525+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848ed9fc10 0x55848eda2030 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f8ae2e9307dede server_cookie=6cf11b61deb6c852 in_seq=0 out_seq=0 2023-08-11T14:14:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.531+0000 7f9f1dffb700 10 client.26476 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.534+0000 7f9f1dffb700 1 -- 172.21.15.3:0/106587996 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x7f9f300b8670 con 0x55848ed9fc10 2023-08-11T14:14:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.534+0000 7f9f1dffb700 10 client.26476 handle_command_reply: tid=0 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.534+0000 7f9f4fe94b80 1 client.26476 shutdown 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 msgr2=0x55848ed9b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c018000 tx=0x7f9f2c009c00 comp rx=0 tx=0).stop 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 shutdown_connections 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9b240 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed89f40 0x55848ed93cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848ed9fc10 0x55848eda2030 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/106587996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed8a8a0 0x55848ed94200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 >> 172.21.15.3:0/106587996 conn(0x55848ecb78f0 msgr2=0x55848ed85710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.535+0000 7f9f1dffb700 0 client.26476 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.546+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 shutdown_connections 2023-08-11T14:14:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.555+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/106587996 wait complete. 2023-08-11T14:14:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.555+0000 7f9f4fe94b80 20 client.26476 trim_cache size 0 max 16384 2023-08-11T14:14:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.581+0000 7f9f4fe94b80 1 Processor -- start 2023-08-11T14:14:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 -- start start 2023-08-11T14:14:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed9b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed8afd0 con 0x55848ed8a8a0 2023-08-11T14:14:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed919c0 con 0x55848ed89f40 2023-08-11T14:14:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.586+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed8f2d0 con 0x55848ed8b790 2023-08-11T14:14:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.591+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.591+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed91c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47084/0 (socket says 172.21.15.3:47084) 2023-08-11T14:14:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.591+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/524512697 learned_addr learned my addr 172.21.15.3:0/524512697 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.591+0000 7f9f39b9a700 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed9b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 -- 172.21.15.3:0/524512697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 msgr2=0x55848ed91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 -- 172.21.15.3:0/524512697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 msgr2=0x55848ed8a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 -- 172.21.15.3:0/524512697 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55848eaf1c00 con 0x55848ed8a8a0 2023-08-11T14:14:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.592+0000 7f9f39b9a700 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed9b160 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55848ed8af70 tx=0x7f9f24003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ddd153343d4941 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.603+0000 7f9f293ff700 1 -- 172.21.15.3:0/524512697 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009800 con 0x55848ed8a8a0 2023-08-11T14:14:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.603+0000 7f9f293ff700 1 -- 172.21.15.3:0/524512697 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f24009e20 con 0x55848ed8a8a0 2023-08-11T14:14:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.603+0000 7f9f293ff700 1 -- 172.21.15.3:0/524512697 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24012450 con 0x55848ed8a8a0 2023-08-11T14:14:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.604+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/524512697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 msgr2=0x55848ed9b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.604+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed9b160 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55848ed8af70 tx=0x7f9f24003960 comp rx=0 tx=0).stop 2023-08-11T14:14:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.618+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/524512697 shutdown_connections 2023-08-11T14:14:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.618+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.618+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed8a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.618+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/524512697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed9b160 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.618+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/524512697 >> 172.21.15.3:0/524512697 conn(0x55848ecb78f0 msgr2=0x55848ed929f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.642+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/524512697 shutdown_connections 2023-08-11T14:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.652+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/524512697 wait complete. 2023-08-11T14:14:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.652+0000 7f9f4fe94b80 1 -- start start 2023-08-11T14:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 Processor -- start 2023-08-11T14:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed98bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed9a640 con 0x55848ed8a8a0 2023-08-11T14:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed9a7b0 con 0x55848ed89f40 2023-08-11T14:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55848ed9a970 con 0x55848ed8b790 2023-08-11T14:14:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.653+0000 7f9f4fe94b80 10 client.0 _ll_register_callbacks cb 0x7f9f2c004890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.654+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.654+0000 7f9f3ab9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47088/0 (socket says 172.21.15.3:47088) 2023-08-11T14:14:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.654+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/1934582548 learned_addr learned my addr 172.21.15.3:0/1934582548 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.655+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 msgr2=0x55848ed98bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.655+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed98bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.655+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/1934582548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 msgr2=0x55848ed990f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.655+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed990f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.655+0000 7f9f3ab9c700 1 -- 172.21.15.3:0/1934582548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24004310 con 0x55848ed8b790 2023-08-11T14:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.656+0000 7f9f3ab9c700 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0298a0 tx=0x7f9f2c007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810ea493cc755d0f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.656+0000 7f9f1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.657+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c040070 con 0x55848ed8b790 2023-08-11T14:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.657+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f2c029a10 con 0x55848ed8b790 2023-08-11T14:14:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.658+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f2c00f040 con 0x55848ed8b790 2023-08-11T14:14:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.670+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2c029bd0 con 0x55848ed8b790 2023-08-11T14:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.680+0000 7f9f4fe94b80 10 client.26482 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.680+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55848eb2f1f0 con 0x55848ed8b790 2023-08-11T14:14:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f2c00f040 con 0x55848ed8b790 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f9f2c03b3f0 con 0x55848ed8b790 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 10 client.26482 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 10 client.26482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 20 client.26482 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848edf4230 0x55848edf6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 4 client.26482 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:14:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.694+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55848ecd57c0 con 0x55848edf4230 2023-08-11T14:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.700+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848edf4230 0x55848edf6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.741+0000 7f9f3a39b700 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848edf4230 0x55848edf6650 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c15f67dfb9278c server_cookie=edf06686cbf87e3a in_seq=0 out_seq=0 2023-08-11T14:14:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:29.741+0000 7f9f1dffb700 10 client.26482 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.063+0000 7f9f1dffb700 1 -- 172.21.15.3:0/1934582548 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55848ecd57c0 con 0x55848edf4230 2023-08-11T14:14:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.063+0000 7f9f1dffb700 10 client.26482 handle_command_reply: tid=0 2023-08-11T14:14:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.068+0000 7f9f4fe94b80 1 client.26482 shutdown 2023-08-11T14:14:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.069+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 msgr2=0x55848ed9a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.069+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0298a0 tx=0x7f9f2c007980 comp rx=0 tx=0).stop 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 shutdown_connections 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55848ed8b790 0x55848ed9a100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55848edf4230 0x55848edf6650 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55848ed89f40 0x55848ed98bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 --2- 172.21.15.3:0/1934582548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55848ed8a8a0 0x55848ed990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 >> 172.21.15.3:0/1934582548 conn(0x55848ecb78f0 msgr2=0x55848ed91c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.072+0000 7f9f1dffb700 0 client.26482 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.080+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 shutdown_connections 2023-08-11T14:14:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.081+0000 7f9f4fe94b80 1 -- 172.21.15.3:0/1934582548 wait complete. 2023-08-11T14:14:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.081+0000 7f9f4fe94b80 20 client.26482 trim_cache size 0 max 16384 2023-08-11T14:14:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:14:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:14:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:14:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:14:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.083+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 msgr2=0x7f9f200a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.083+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 0x7f9f200a2610 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000f80 tx=0x7f9f2c007500 comp rx=0 tx=0).stop 2023-08-11T14:14:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.083+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 msgr2=0x7f9f34117d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.083+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 0x7f9f34117d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f3001f620 comp rx=0 tx=0).stop 2023-08-11T14:14:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 shutdown_connections 2023-08-11T14:14:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f3410cc60 0x7f9f341c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f34107920 0x7f9f34117d10 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f3001f620 comp rx=0 tx=0).stop 2023-08-11T14:14:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f200a0150 0x7f9f200a2610 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 --2- 172.21.15.3:0/1485072525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f3410a290 0x7f9f34118270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.086+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 >> 172.21.15.3:0/1485072525 conn(0x7f9f3406d750 msgr2=0x7f9f34108390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.090+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 shutdown_connections 2023-08-11T14:14:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:30.091+0000 7f9f3c5ff700 1 -- 172.21.15.3:0/1485072525 wait complete. 2023-08-11T14:14:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:30 smithi103 ceph-mon[130434]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 950 op/s 2023-08-11T14:14:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:30 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:30 smithi187 ceph-mon[130973]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 950 op/s 2023-08-11T14:14:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:30 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:30 smithi003 ceph-mon[97195]: pgmap v664: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 950 op/s 2023-08-11T14:14:30.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:30.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:30 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:32.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:32 smithi003 ceph-mon[97195]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 718 op/s 2023-08-11T14:14:32.802 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:32 smithi187 ceph-mon[130973]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 718 op/s 2023-08-11T14:14:32.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:32 smithi103 ceph-mon[130434]: pgmap v665: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 718 op/s 2023-08-11T14:14:33.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 455.0 s, 724.0 tps, lat 138.751 ms stddev 114.913 2023-08-11T14:14:34.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:34 smithi187 ceph-mon[130973]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-08-11T14:14:34.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:34.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:34 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:34.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:34 smithi003 ceph-mon[97195]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-08-11T14:14:34.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:34.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:34 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:34 smithi103 ceph-mon[130434]: pgmap v666: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-08-11T14:14:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:34.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:34 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:36 smithi103 ceph-mon[130434]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 724 op/s 2023-08-11T14:14:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:14:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:14:36.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:36 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:37.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:36 smithi003 ceph-mon[97195]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 724 op/s 2023-08-11T14:14:37.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:14:37.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:14:37.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:36 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:37.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:36 smithi187 ceph-mon[130973]: pgmap v667: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 10 MiB/s wr, 724 op/s 2023-08-11T14:14:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:14:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:14:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:36 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:14:38.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:37 smithi003 ceph-mon[97195]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 834 op/s 2023-08-11T14:14:38.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:37 smithi187 ceph-mon[130973]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 834 op/s 2023-08-11T14:14:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 460.0 s, 477.0 tps, lat 210.704 ms stddev 480.516 2023-08-11T14:14:38.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:37 smithi103 ceph-mon[130434]: pgmap v668: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 834 op/s 2023-08-11T14:14:40.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:40 smithi103 ceph-mon[130434]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-08-11T14:14:40.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:40 smithi003 ceph-mon[97195]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-08-11T14:14:40.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:40 smithi187 ceph-mon[130973]: pgmap v669: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 891 op/s 2023-08-11T14:14:42.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:42 smithi003 ceph-mon[97195]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 764 op/s 2023-08-11T14:14:42.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:42 smithi187 ceph-mon[130973]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 764 op/s 2023-08-11T14:14:42.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:42 smithi103 ceph-mon[130434]: pgmap v670: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 764 op/s 2023-08-11T14:14:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 465.0 s, 691.4 tps, lat 145.811 ms stddev 272.060 2023-08-11T14:14:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.361+0000 7fa768bff700 1 -- 172.21.15.3:0/3159013040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa764107920 msgr2=0x7fa764109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.361+0000 7fa768bff700 1 --2- 172.21.15.3:0/3159013040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa764107920 0x7fa764109d50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa758009990 tx=0x7fa758009ca0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.367+0000 7fa768bff700 1 -- 172.21.15.3:0/3159013040 shutdown_connections 2023-08-11T14:14:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.367+0000 7fa768bff700 1 --2- 172.21.15.3:0/3159013040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa76410cc60 0x7fa76410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.367+0000 7fa768bff700 1 --2- 172.21.15.3:0/3159013040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa764107920 0x7fa764109d50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.367+0000 7fa768bff700 1 --2- 172.21.15.3:0/3159013040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 0x7fa76410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.367+0000 7fa768bff700 1 -- 172.21.15.3:0/3159013040 >> 172.21.15.3:0/3159013040 conn(0x7fa76406d750 msgr2=0x7fa76406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.377+0000 7fa768bff700 1 -- 172.21.15.3:0/3159013040 shutdown_connections 2023-08-11T14:14:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.387+0000 7fa768bff700 1 -- 172.21.15.3:0/3159013040 wait complete. 2023-08-11T14:14:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.388+0000 7fa768bff700 1 Processor -- start 2023-08-11T14:14:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.388+0000 7fa768bff700 1 -- start start 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764107920 0x7fa7641a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 0x7fa7641a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641a6fb0 con 0x7fa76410cc60 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641a7120 con 0x7fa76410a290 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.389+0000 7fa768bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641a72e0 con 0x7fa764107920 2023-08-11T14:14:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.392+0000 7fa761d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 0x7fa7641a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.392+0000 7fa762d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.392+0000 7fa762d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37086/0 (socket says 172.21.15.3:37086) 2023-08-11T14:14:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.392+0000 7fa762d9d700 1 -- 172.21.15.3:0/2586180095 learned_addr learned my addr 172.21.15.3:0/2586180095 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.399+0000 7fa762d9d700 1 -- 172.21.15.3:0/2586180095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764107920 msgr2=0x7fa7641a5520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:14:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.399+0000 7fa762d9d700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764107920 0x7fa7641a5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.399+0000 7fa762d9d700 1 -- 172.21.15.3:0/2586180095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 msgr2=0x7fa7641a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.399+0000 7fa762d9d700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 0x7fa7641a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.399+0000 7fa762d9d700 1 -- 172.21.15.3:0/2586180095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7580095f0 con 0x7fa76410cc60 2023-08-11T14:14:43.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.414+0000 7fa762d9d700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007ee0 tx=0x7fa75c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.415+0000 7fa7537fe700 1 -- 172.21.15.3:0/2586180095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c017070 con 0x7fa76410cc60 2023-08-11T14:14:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.416+0000 7fa7537fe700 1 -- 172.21.15.3:0/2586180095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75c00f040 con 0x7fa76410cc60 2023-08-11T14:14:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.416+0000 7fa7537fe700 1 -- 172.21.15.3:0/2586180095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c005740 con 0x7fa76410cc60 2023-08-11T14:14:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.423+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7641a75c0 con 0x7fa76410cc60 2023-08-11T14:14:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.423+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa764078de0 con 0x7fa76410cc60 2023-08-11T14:14:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.429+0000 7fa77c4f4b80 1 Processor -- start 2023-08-11T14:14:43.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.443+0000 7fa77c4f4b80 1 -- start start 2023-08-11T14:14:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.443+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d6ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33cc25d0 con 0x55be33d68f40 2023-08-11T14:14:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33cbfef0 con 0x55be33d698a0 2023-08-11T14:14:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33cbb7e0 con 0x55be33d6a790 2023-08-11T14:14:43.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.444+0000 7fa7537fe700 1 -- 172.21.15.3:0/2586180095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa75c0058a0 con 0x7fa76410cc60 2023-08-11T14:14:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.446+0000 7fa7537fe700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 0x7fa74c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.446+0000 7fa7537fe700 1 -- 172.21.15.3:0/2586180095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa75c0ca470 con 0x7fa76410cc60 2023-08-11T14:14:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.450+0000 7fa76259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.450+0000 7fa76259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47236/0 (socket says 172.21.15.3:47236) 2023-08-11T14:14:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.450+0000 7fa76259c700 1 -- 172.21.15.3:0/1741313788 learned_addr learned my addr 172.21.15.3:0/1741313788 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.452+0000 7fa76259c700 1 -- 172.21.15.3:0/1741313788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 msgr2=0x55be33d6ac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.452+0000 7fa76259c700 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d6ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.452+0000 7fa76259c700 1 -- 172.21.15.3:0/1741313788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 msgr2=0x55be33d69360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.452+0000 7fa76259c700 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d69360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.452+0000 7fa76259c700 1 -- 172.21.15.3:0/1741313788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be33b3b5d0 con 0x55be33d698a0 2023-08-11T14:14:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.453+0000 7fa76259c700 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa758005950 tx=0x7fa758005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3311770cb320c9a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.457+0000 7fa750ff9700 1 -- 172.21.15.3:0/1741313788 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801d070 con 0x55be33d698a0 2023-08-11T14:14:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.457+0000 7fa750ff9700 1 -- 172.21.15.3:0/1741313788 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75800be00 con 0x55be33d698a0 2023-08-11T14:14:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.457+0000 7fa750ff9700 1 -- 172.21.15.3:0/1741313788 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758005430 con 0x55be33d698a0 2023-08-11T14:14:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.458+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/1741313788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 msgr2=0x55be33d6a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.458+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa758005950 tx=0x7fa758005dc0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.464+0000 7fa762d9d700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 0x7fa74c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa762d9d700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 0x7fa74c0a2610 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa75c09f420 tx=0x7fa75c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/1741313788 shutdown_connections 2023-08-11T14:14:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d6ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d69360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/1741313788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d6a1b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.466+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/1741313788 >> 172.21.15.3:0/1741313788 conn(0x55be33c968f0 msgr2=0x55be33d65920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.475+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/1741313788 shutdown_connections 2023-08-11T14:14:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.475+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/1741313788 wait complete. 2023-08-11T14:14:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.476+0000 7fa77c4f4b80 1 -- start start 2023-08-11T14:14:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.476+0000 7fa77c4f4b80 1 Processor -- start 2023-08-11T14:14:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d83c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33c59a60 con 0x55be33d68f40 2023-08-11T14:14:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33c59760 con 0x55be33d698a0 2023-08-11T14:14:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33b3d8c0 con 0x55be33d6a790 2023-08-11T14:14:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.477+0000 7fa77c4f4b80 10 client.0 _ll_register_callbacks cb 0x55be33c5ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.478+0000 7fa76259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.478+0000 7fa76259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47246/0 (socket says 172.21.15.3:47246) 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.478+0000 7fa76259c700 1 -- 172.21.15.3:0/2771889969 learned_addr learned my addr 172.21.15.3:0/2771889969 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d83c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa76259c700 1 -- 172.21.15.3:0/2771889969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 msgr2=0x55be33d8b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa76259c700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d8b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa76259c700 1 -- 172.21.15.3:0/2771889969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 msgr2=0x55be33d83c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa76259c700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d83c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.479+0000 7fa76259c700 1 -- 172.21.15.3:0/2771889969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76410dc60 con 0x55be33d698a0 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.480+0000 7fa76259c700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa758005e50 tx=0x7fa75800ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b187dc65d710aac7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801d070 con 0x55be33d698a0 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa758005600 con 0x55be33d698a0 2023-08-11T14:14:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be33d5c7e0 con 0x55be33d698a0 2023-08-11T14:14:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa75800f460 con 0x55be33d698a0 2023-08-11T14:14:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.481+0000 7fa77c4f4b80 10 client.26468 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55be33ae7d80 con 0x55be33d698a0 2023-08-11T14:14:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be33d5c7e0 con 0x55be33d698a0 2023-08-11T14:14:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa758022660 con 0x55be33d698a0 2023-08-11T14:14:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 10 client.26468 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 10 client.26468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 20 client.26468 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33d8f500 0x55be33d91920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 4 client.26468 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:14:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.482+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be33d38480 con 0x55be33d8f500 2023-08-11T14:14:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.483+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33d8f500 0x55be33d91920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.484+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33d8f500 0x55be33d91920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7e4890d1a4f3d5 server_cookie=67e68f0103d48c1c in_seq=0 out_seq=0 2023-08-11T14:14:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.485+0000 7fa749ffb700 10 client.26468 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.486+0000 7fa749ffb700 1 -- 172.21.15.3:0/2771889969 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55be33d38480 con 0x55be33d8f500 2023-08-11T14:14:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.486+0000 7fa749ffb700 10 client.26468 handle_command_reply: tid=0 2023-08-11T14:14:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.486+0000 7fa77c4f4b80 1 client.26468 shutdown 2023-08-11T14:14:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.489+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 msgr2=0x55be33d84190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.489+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa758005e50 tx=0x7fa75800ba00 comp rx=0 tx=0).stop 2023-08-11T14:14:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 shutdown_connections 2023-08-11T14:14:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d6a790 0x55be33d8b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d68f40 0x55be33d83c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33d8f500 0x55be33d91920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2771889969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d84190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 >> 172.21.15.3:0/2771889969 conn(0x55be33c968f0 msgr2=0x55be33d64710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.490+0000 7fa749ffb700 0 client.26468 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.494+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 shutdown_connections 2023-08-11T14:14:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.494+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2771889969 wait complete. 2023-08-11T14:14:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.495+0000 7fa77c4f4b80 20 client.26468 trim_cache size 0 max 16384 2023-08-11T14:14:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.521+0000 7fa77c4f4b80 1 Processor -- start 2023-08-11T14:14:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 -- start start 2023-08-11T14:14:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d68f40 0x55be33d6ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d84e70 con 0x55be33d6a790 2023-08-11T14:14:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d6a640 con 0x55be33d68f40 2023-08-11T14:14:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.522+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d5c7c0 con 0x55be33d698a0 2023-08-11T14:14:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.533+0000 7fa76259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.533+0000 7fa76259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38060/0 (socket says 172.21.15.3:38060) 2023-08-11T14:14:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.533+0000 7fa76259c700 1 -- 172.21.15.3:0/3969859111 learned_addr learned my addr 172.21.15.3:0/3969859111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.533+0000 7fa761d9b700 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d68f40 0x55be33d6ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.534+0000 7fa761d9b700 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.545+0000 7fa76259c700 1 -- 172.21.15.3:0/3969859111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d68f40 msgr2=0x55be33d6ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.545+0000 7fa76259c700 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d68f40 0x55be33d6ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.545+0000 7fa76259c700 1 -- 172.21.15.3:0/3969859111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 msgr2=0x55be33d83c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.545+0000 7fa76259c700 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d83c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.545+0000 7fa76259c700 1 -- 172.21.15.3:0/3969859111 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be33ad0c00 con 0x55be33d698a0 2023-08-11T14:14:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.565+0000 7fa76259c700 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa758022fb0 tx=0x7fa758022ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d536420e6599d56 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.571+0000 7fa7517fa700 1 -- 172.21.15.3:0/3969859111 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801d070 con 0x55be33d698a0 2023-08-11T14:14:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.571+0000 7fa7517fa700 1 -- 172.21.15.3:0/3969859111 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75801e040 con 0x55be33d698a0 2023-08-11T14:14:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.571+0000 7fa7517fa700 1 -- 172.21.15.3:0/3969859111 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75800f6c0 con 0x55be33d698a0 2023-08-11T14:14:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/3969859111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 msgr2=0x55be33d8b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa758022fb0 tx=0x7fa758022ce0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/3969859111 shutdown_connections 2023-08-11T14:14:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d83c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d68f40 0x55be33d6ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/3969859111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d698a0 0x55be33d8b190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.573+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/3969859111 >> 172.21.15.3:0/3969859111 conn(0x55be33c968f0 msgr2=0x55be33d70990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.584+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/3969859111 shutdown_connections 2023-08-11T14:14:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.584+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/3969859111 wait complete. 2023-08-11T14:14:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.585+0000 7fa77c4f4b80 1 -- start start 2023-08-11T14:14:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.585+0000 7fa77c4f4b80 1 Processor -- start 2023-08-11T14:14:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 0x55be33d88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d89160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d8a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d8a6b0 con 0x55be33d6a790 2023-08-11T14:14:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d8a820 con 0x55be33d698a0 2023-08-11T14:14:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be33d8a9e0 con 0x55be33d68f40 2023-08-11T14:14:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.589+0000 7fa77c4f4b80 10 client.0 _ll_register_callbacks cb 0x7fa758009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.597+0000 7fa761d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 0x55be33d88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.597+0000 7fa761d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 0x55be33d88c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:38076/0 (socket says 172.21.15.3:38076) 2023-08-11T14:14:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.597+0000 7fa761d9b700 1 -- 172.21.15.3:0/2058844639 learned_addr learned my addr 172.21.15.3:0/2058844639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.597+0000 7fa76259c700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.598+0000 7fa76259c700 1 -- 172.21.15.3:0/2058844639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 msgr2=0x55be33d88c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.598+0000 7fa76259c700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 0x55be33d88c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.598+0000 7fa76259c700 1 -- 172.21.15.3:0/2058844639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 msgr2=0x55be33d8a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.598+0000 7fa76259c700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.598+0000 7fa76259c700 1 -- 172.21.15.3:0/2058844639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be33ae7d80 con 0x55be33d698a0 2023-08-11T14:14:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.604+0000 7fa76259c700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d89160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa758022fb0 tx=0x7fa758005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b26abe14346687 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.605+0000 7fa749ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:14:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.609+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801d040 con 0x55be33d698a0 2023-08-11T14:14:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.609+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa75800f460 con 0x55be33d698a0 2023-08-11T14:14:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.609+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa75800be00 con 0x55be33d698a0 2023-08-11T14:14:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.613+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7580227f0 con 0x55be33d698a0 2023-08-11T14:14:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.614+0000 7fa77c4f4b80 10 client.26474 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.614+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55be33b0e1f0 con 0x55be33d698a0 2023-08-11T14:14:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75800be00 con 0x55be33d698a0 2023-08-11T14:14:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fa758034730 con 0x55be33d698a0 2023-08-11T14:14:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 10 client.26474 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 20 client.26474 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33de3960 0x55be33de5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 4 client.26474 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:14:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be33cb47c0 con 0x55be33de3960 2023-08-11T14:14:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.616+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33de3960 0x55be33de5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.622+0000 7fa761d9b700 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33de3960 0x55be33de5d80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f58b1875297ecc server_cookie=948817249330c1f3 in_seq=0 out_seq=0 2023-08-11T14:14:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.622+0000 7fa749ffb700 10 client.26474 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.653+0000 7fa749ffb700 1 -- 172.21.15.3:0/2058844639 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be33cb47c0 con 0x55be33de3960 2023-08-11T14:14:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.653+0000 7fa749ffb700 10 client.26474 handle_command_reply: tid=0 2023-08-11T14:14:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 client.26474 shutdown 2023-08-11T14:14:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 msgr2=0x55be33d89160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d89160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa758022fb0 tx=0x7fa758005fb0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 shutdown_connections 2023-08-11T14:14:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be33d6a790 0x55be33d8a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be33d68f40 0x55be33d88c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55be33de3960 0x55be33de5d80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 --2- 172.21.15.3:0/2058844639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55be33d698a0 0x55be33d89160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.654+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 >> 172.21.15.3:0/2058844639 conn(0x55be33c968f0 msgr2=0x55be33d6d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.656+0000 7fa749ffb700 0 client.26474 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.656+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 shutdown_connections 2023-08-11T14:14:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.656+0000 7fa77c4f4b80 1 -- 172.21.15.3:0/2058844639 wait complete. 2023-08-11T14:14:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.656+0000 7fa77c4f4b80 20 client.26474 trim_cache size 0 max 16384 2023-08-11T14:14:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:14:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:14:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:14:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:14:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.660+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 msgr2=0x7fa74c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.660+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 0x7fa74c0a2610 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa75c09f420 tx=0x7fa75c00b040 comp rx=0 tx=0).stop 2023-08-11T14:14:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.660+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 msgr2=0x7fa7641a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.660+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007ee0 tx=0x7fa75c00d3b0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 shutdown_connections 2023-08-11T14:14:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa76410cc60 0x7fa7641a6920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa764107920 0x7fa7641a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa74c0a0150 0x7fa74c0a2610 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 --2- 172.21.15.3:0/2586180095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76410a290 0x7fa7641a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.661+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 >> 172.21.15.3:0/2586180095 conn(0x7fa76406d750 msgr2=0x7fa76410bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.662+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 shutdown_connections 2023-08-11T14:14:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:43.662+0000 7fa768bff700 1 -- 172.21.15.3:0/2586180095 wait complete. 2023-08-11T14:14:44.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:44 smithi003 ceph-mon[97195]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-08-11T14:14:44.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:44 smithi187 ceph-mon[130973]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-08-11T14:14:44.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:44 smithi103 ceph-mon[130434]: pgmap v671: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.01k op/s 2023-08-11T14:14:46.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:46 smithi103 ceph-mon[130434]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:14:47.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:46 smithi003 ceph-mon[97195]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:14:47.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:46 smithi187 ceph-mon[130973]: pgmap v672: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:14:48.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 470.0 s, 624.0 tps, lat 157.439 ms stddev 394.788 2023-08-11T14:14:48.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:48 smithi103 ceph-mon[130434]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:14:49.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:48 smithi187 ceph-mon[130973]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:14:49.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:48 smithi003 ceph-mon[97195]: pgmap v673: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:14:50.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:49 smithi187 ceph-mon[130973]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:14:50.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:49 smithi003 ceph-mon[97195]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:14:50.192 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:49 smithi103 ceph-mon[130434]: pgmap v674: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:14:52.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:52 smithi103 ceph-mon[130434]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:14:52.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:52 smithi187 ceph-mon[130973]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:14:52.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:52 smithi003 ceph-mon[97195]: pgmap v675: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:14:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 475.0 s, 789.6 tps, lat 83.533 ms stddev 128.876 2023-08-11T14:14:54.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:54 smithi103 ceph-mon[130434]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:14:54.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:54 smithi187 ceph-mon[130973]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:14:54.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:54 smithi003 ceph-mon[97195]: pgmap v676: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-08-11T14:14:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:56 smithi103 ceph-mon[130434]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-08-11T14:14:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:56 smithi003 ceph-mon[97195]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-08-11T14:14:56.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:56 smithi187 ceph-mon[130973]: pgmap v677: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 18 MiB/s wr, 1.36k op/s 2023-08-11T14:14:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 -- 172.21.15.3:0/1187048451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab4010a290 msgr2=0x7fab4010c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:56.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 --2- 172.21.15.3:0/1187048451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab4010a290 0x7fab4010c720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b0a0 tx=0x7fab3800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 -- 172.21.15.3:0/1187048451 shutdown_connections 2023-08-11T14:14:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 --2- 172.21.15.3:0/1187048451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab4010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 --2- 172.21.15.3:0/1187048451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab40107920 0x7fab40109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 --2- 172.21.15.3:0/1187048451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab4010a290 0x7fab4010c720 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 -- 172.21.15.3:0/1187048451 >> 172.21.15.3:0/1187048451 conn(0x7fab4006d750 msgr2=0x7fab4006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.936+0000 7fab46dff700 1 -- 172.21.15.3:0/1187048451 shutdown_connections 2023-08-11T14:14:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.937+0000 7fab46dff700 1 -- 172.21.15.3:0/1187048451 wait complete. 2023-08-11T14:14:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.938+0000 7fab46dff700 1 Processor -- start 2023-08-11T14:14:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 -- start start 2023-08-11T14:14:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 0x7fab401a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 0x7fab401a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab401a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab401a7070 con 0x7fab4010cc60 2023-08-11T14:14:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab401a71e0 con 0x7fab40107920 2023-08-11T14:14:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.939+0000 7fab46dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab401a73a0 con 0x7fab4010a290 2023-08-11T14:14:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.940+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 0x7fab401a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.940+0000 7fab4539c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab401a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.940+0000 7fab4539c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab401a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37338/0 (socket says 172.21.15.3:37338) 2023-08-11T14:14:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.940+0000 7fab4539c700 1 -- 172.21.15.3:0/3223251634 learned_addr learned my addr 172.21.15.3:0/3223251634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.940+0000 7fab44b9b700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 0x7fab401a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab44b9b700 1 -- 172.21.15.3:0/3223251634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 msgr2=0x7fab401a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab44b9b700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 0x7fab401a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab44b9b700 1 -- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 msgr2=0x7fab401a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab44b9b700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab401a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab44b9b700 1 -- 172.21.15.3:0/3223251634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009d00 con 0x7fab40107920 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.941+0000 7fab3ffff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 0x7fab401a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.942+0000 7fab44b9b700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 0x7fab401a5650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e9f0 tx=0x7fab3400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.942+0000 7fab3dffb700 1 -- 172.21.15.3:0/3223251634 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340096d0 con 0x7fab40107920 2023-08-11T14:14:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.942+0000 7fab3dffb700 1 -- 172.21.15.3:0/3223251634 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab3400cbd0 con 0x7fab40107920 2023-08-11T14:14:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.942+0000 7fab3dffb700 1 -- 172.21.15.3:0/3223251634 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34018680 con 0x7fab40107920 2023-08-11T14:14:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.943+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab401a7680 con 0x7fab40107920 2023-08-11T14:14:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.943+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4010f230 con 0x7fab40107920 2023-08-11T14:14:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.952+0000 7fab5a68bb80 1 Processor -- start 2023-08-11T14:14:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.954+0000 7fab3dffb700 1 -- 172.21.15.3:0/3223251634 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fab3400cd40 con 0x7fab40107920 2023-08-11T14:14:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 -- start start 2023-08-11T14:14:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722065d0 con 0x561d722ad8a0 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d72203ef0 con 0x561d722ae790 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.966+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d721ff7e0 con 0x561d722acf40 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.967+0000 7fab44b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.967+0000 7fab44b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37346/0 (socket says 172.21.15.3:37346) 2023-08-11T14:14:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.967+0000 7fab44b9b700 1 -- 172.21.15.3:0/2410284229 learned_addr learned my addr 172.21.15.3:0/2410284229 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.969+0000 7fab3dffb700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 0x7fab280a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.970+0000 7fab3dffb700 1 -- 172.21.15.3:0/3223251634 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fab34014070 con 0x7fab40107920 2023-08-11T14:14:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.971+0000 7fab44b9b700 1 -- 172.21.15.3:0/2410284229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 msgr2=0x561d722ad360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.971+0000 7fab44b9b700 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722ad360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.971+0000 7fab44b9b700 1 -- 172.21.15.3:0/2410284229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 msgr2=0x561d722aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.971+0000 7fab44b9b700 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.971+0000 7fab44b9b700 1 -- 172.21.15.3:0/2410284229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d7207f5d0 con 0x561d722ad8a0 2023-08-11T14:14:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.987+0000 7fab44b9b700 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab34098a60 tx=0x7fab34010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2660d175232c157c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.988+0000 7fab26ffd700 1 -- 172.21.15.3:0/2410284229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340986c0 con 0x561d722ad8a0 2023-08-11T14:14:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.988+0000 7fab26ffd700 1 -- 172.21.15.3:0/2410284229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab340cea60 con 0x561d722ad8a0 2023-08-11T14:14:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.988+0000 7fab26ffd700 1 -- 172.21.15.3:0/2410284229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340b80d0 con 0x561d722ad8a0 2023-08-11T14:14:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.989+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2410284229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 msgr2=0x561d722ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.989+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab34098a60 tx=0x7fab34010ae0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab4539c700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 0x7fab280a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2410284229 shutdown_connections 2023-08-11T14:14:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2410284229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722ae1b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:56.995+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2410284229 >> 172.21.15.3:0/2410284229 conn(0x561d721da8f0 msgr2=0x561d722a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.024+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2410284229 shutdown_connections 2023-08-11T14:14:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.034+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2410284229 wait complete. 2023-08-11T14:14:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.035+0000 7fab5a68bb80 1 -- start start 2023-08-11T14:14:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.035+0000 7fab5a68bb80 1 Processor -- start 2023-08-11T14:14:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d7219da60 con 0x561d722ad8a0 2023-08-11T14:14:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d7219d760 con 0x561d722ae790 2023-08-11T14:14:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d720818c0 con 0x561d722acf40 2023-08-11T14:14:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.036+0000 7fab5a68bb80 10 client.0 _ll_register_callbacks cb 0x561d721a0a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.037+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.037+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37350/0 (socket says 172.21.15.3:37350) 2023-08-11T14:14:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.037+0000 7fab3ffff700 1 -- 172.21.15.3:0/2460833727 learned_addr learned my addr 172.21.15.3:0/2460833727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.038+0000 7fab3ffff700 1 -- 172.21.15.3:0/2460833727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 msgr2=0x561d722b6c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:14:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.038+0000 7fab3ffff700 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722b6c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.038+0000 7fab3ffff700 1 -- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 msgr2=0x561d722be180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.038+0000 7fab3ffff700 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.038+0000 7fab3ffff700 1 -- 172.21.15.3:0/2460833727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4010f230 con 0x561d722ad8a0 2023-08-11T14:14:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.039+0000 7fab3ffff700 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fab38000f80 tx=0x7fab3800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55cb6c25e56298a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.042+0000 7fab1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:14:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.042+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800e040 con 0x561d722ad8a0 2023-08-11T14:14:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.042+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fab38003c00 con 0x561d722ad8a0 2023-08-11T14:14:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.042+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d722a07e0 con 0x561d722ad8a0 2023-08-11T14:14:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.044+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab38003dc0 con 0x561d722ad8a0 2023-08-11T14:14:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.045+0000 7fab4539c700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 0x7fab280a2610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fab30009b50 tx=0x7fab30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.049+0000 7fab5a68bb80 10 client.16359 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.049+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x561d7202bd80 con 0x561d722ad8a0 2023-08-11T14:14:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab3802b030 con 0x561d722ad8a0 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fab38022020 con 0x561d722ad8a0 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 20 client.16359 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d722c2ad0 0x561d722c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 4 client.16359 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:14:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.060+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d7227c480 con 0x561d722c2ad0 2023-08-11T14:14:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.081+0000 7fab44b9b700 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d722c2ad0 0x561d722c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.083+0000 7fab44b9b700 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d722c2ad0 0x561d722c4ef0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d26845a42de0838 server_cookie=54bdb56812ac0595 in_seq=0 out_seq=0 2023-08-11T14:14:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.084+0000 7fab1ffff700 10 client.16359 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.085+0000 7fab1ffff700 1 -- 172.21.15.3:0/2460833727 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561d7227c480 con 0x561d722c2ad0 2023-08-11T14:14:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.085+0000 7fab1ffff700 10 client.16359 handle_command_reply: tid=0 2023-08-11T14:14:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.088+0000 7fab5a68bb80 1 client.16359 shutdown 2023-08-11T14:14:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.102+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 msgr2=0x561d722b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.102+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fab38000f80 tx=0x7fab3800bab0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 shutdown_connections 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d722c2ad0 0x561d722c4ef0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722ae790 0x561d722be180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722acf40 0x561d722b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/2460833727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ad8a0 0x561d722b7140 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fab38000f80 tx=0x7fab3800bab0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 >> 172.21.15.3:0/2460833727 conn(0x561d721da8f0 msgr2=0x561d722a8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.103+0000 7fab1ffff700 0 client.16359 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.104+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 shutdown_connections 2023-08-11T14:14:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.115+0000 7fab5a68bb80 1 -- 172.21.15.3:0/2460833727 wait complete. 2023-08-11T14:14:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.117+0000 7fab5a68bb80 20 client.16359 trim_cache size 0 max 16384 2023-08-11T14:14:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 Processor -- start 2023-08-11T14:14:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 -- start start 2023-08-11T14:14:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722acf40 0x561d722aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ae790 0x561d722be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722b2080 con 0x561d722ae790 2023-08-11T14:14:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722c9d20 con 0x561d722acf40 2023-08-11T14:14:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.150+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722a0680 con 0x561d722b16b0 2023-08-11T14:14:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.155+0000 7fab4539c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.155+0000 7fab4539c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40624/0 (socket says 172.21.15.3:40624) 2023-08-11T14:14:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.155+0000 7fab4539c700 1 -- 172.21.15.3:0/147219209 learned_addr learned my addr 172.21.15.3:0/147219209 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 -- 172.21.15.3:0/147219209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722acf40 msgr2=0x561d722aeb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722acf40 0x561d722aeb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 -- 172.21.15.3:0/147219209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ae790 msgr2=0x561d722be140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:14:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ae790 0x561d722be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 -- 172.21.15.3:0/147219209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d72014c00 con 0x561d722b16b0 2023-08-11T14:14:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.156+0000 7fab4539c700 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab30011d80 tx=0x7fab30014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9912d57e62b3daa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.160+0000 7fab277fe700 1 -- 172.21.15.3:0/147219209 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004830 con 0x561d722b16b0 2023-08-11T14:14:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.161+0000 7fab277fe700 1 -- 172.21.15.3:0/147219209 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab30016070 con 0x561d722b16b0 2023-08-11T14:14:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.161+0000 7fab277fe700 1 -- 172.21.15.3:0/147219209 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3001f680 con 0x561d722b16b0 2023-08-11T14:14:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 -- 172.21.15.3:0/147219209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 msgr2=0x561d722b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab30011d80 tx=0x7fab30014070 comp rx=0 tx=0).stop 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 -- 172.21.15.3:0/147219209 shutdown_connections 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722b16b0 0x561d722b1b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722acf40 0x561d722aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/147219209 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722ae790 0x561d722be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.162+0000 7fab5a68bb80 1 -- 172.21.15.3:0/147219209 >> 172.21.15.3:0/147219209 conn(0x561d721da8f0 msgr2=0x561d722c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.163+0000 7fab5a68bb80 1 -- 172.21.15.3:0/147219209 shutdown_connections 2023-08-11T14:14:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.163+0000 7fab5a68bb80 1 -- 172.21.15.3:0/147219209 wait complete. 2023-08-11T14:14:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.164+0000 7fab5a68bb80 1 -- start start 2023-08-11T14:14:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.164+0000 7fab5a68bb80 1 Processor -- start 2023-08-11T14:14:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722acf40 0x561d722b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 0x561d722ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 0x561d722bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722bd950 con 0x561d722acf40 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722bdac0 con 0x561d722b16b0 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d722bdc80 con 0x561d722ae790 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.165+0000 7fab5a68bb80 10 client.0 _ll_register_callbacks cb 0x7fab38009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.166+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 0x561d722ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.166+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 0x561d722ba3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40634/0 (socket says 172.21.15.3:40634) 2023-08-11T14:14:57.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.166+0000 7fab3ffff700 1 -- 172.21.15.3:0/1026560804 learned_addr learned my addr 172.21.15.3:0/1026560804 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:14:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.166+0000 7fab4539c700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 0x561d722bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 -- 172.21.15.3:0/1026560804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 msgr2=0x561d722ba3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 0x561d722ba3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 -- 172.21.15.3:0/1026560804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722acf40 msgr2=0x561d722b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722acf40 0x561d722b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 -- 172.21.15.3:0/1026560804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab30011a60 con 0x561d722b16b0 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.167+0000 7fab4539c700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 0x561d722bd410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab30012ac0 tx=0x7fab30004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb17d2c245c0476b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.168+0000 7fab1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.168+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30004e00 con 0x561d722b16b0 2023-08-11T14:14:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.168+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fab30018070 con 0x561d722b16b0 2023-08-11T14:14:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.168+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab340cea60 con 0x561d722b16b0 2023-08-11T14:14:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.169+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab30027440 con 0x561d722b16b0 2023-08-11T14:14:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.170+0000 7fab5a68bb80 10 client.26483 fetch_fsmap learned FSMap version 48 2023-08-11T14:14:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.170+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x561d720521f0 con 0x561d722b16b0 2023-08-11T14:14:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab340cea60 con 0x561d722b16b0 2023-08-11T14:14:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7fab30025070 con 0x561d722b16b0 2023-08-11T14:14:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 10 client.26483 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:14:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:14:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 20 client.26483 populate_metadata read hostname 'smithi003' 2023-08-11T14:14:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d72317790 0x561d72319bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:14:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 4 client.26483 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:14:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.172+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d721f87c0 con 0x561d72317790 2023-08-11T14:14:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.174+0000 7fab44b9b700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d72317790 0x561d72319bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:14:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.179+0000 7fab44b9b700 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d72317790 0x561d72319bb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d808b1fe659c1d27 server_cookie=797fe08e8ee19990 in_seq=0 out_seq=0 2023-08-11T14:14:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.179+0000 7fab1ffff700 10 client.26483 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab1ffff700 1 -- 172.21.15.3:0/1026560804 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d721f87c0 con 0x561d72317790 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab1ffff700 10 client.26483 handle_command_reply: tid=0 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 client.26483 shutdown 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 msgr2=0x561d722bd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 0x561d722bd410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fab30012ac0 tx=0x7fab30004710 comp rx=0 tx=0).stop 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 shutdown_connections 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561d72317790 0x561d72319bb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d722b16b0 0x561d722bd410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561d722acf40 0x561d722b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 --2- 172.21.15.3:0/1026560804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d722ae790 0x561d722ba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab1ffff700 0 client.26483 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:14:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.200+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 >> 172.21.15.3:0/1026560804 conn(0x561d721da8f0 msgr2=0x561d722a90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.201+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 shutdown_connections 2023-08-11T14:14:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.201+0000 7fab5a68bb80 1 -- 172.21.15.3:0/1026560804 wait complete. 2023-08-11T14:14:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:14:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:14:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:14:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:14:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.201+0000 7fab5a68bb80 20 client.26483 trim_cache size 0 max 16384 2023-08-11T14:14:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 msgr2=0x7fab280a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 0x7fab280a2610 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fab30009b50 tx=0x7fab30009260 comp rx=0 tx=0).stop 2023-08-11T14:14:57.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 msgr2=0x7fab401a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:14:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 0x7fab401a5650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e9f0 tx=0x7fab3400ed00 comp rx=0 tx=0).stop 2023-08-11T14:14:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 shutdown_connections 2023-08-11T14:14:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4010cc60 0x7fab401a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab280a0150 0x7fab280a2610 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab40107920 0x7fab401a5650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 --2- 172.21.15.3:0/3223251634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab4010a290 0x7fab401a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.207+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 >> 172.21.15.3:0/3223251634 conn(0x7fab4006d750 msgr2=0x7fab40109160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.208+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 shutdown_connections 2023-08-11T14:14:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:14:57.208+0000 7fab46dff700 1 -- 172.21.15.3:0/3223251634 wait complete. 2023-08-11T14:14:58.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 480.0 s, 898.7 tps, lat 151.381 ms stddev 364.467 2023-08-11T14:14:58.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:14:58 smithi103 ceph-mon[130434]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:14:58.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:14:58 smithi003 ceph-mon[97195]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:14:58.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:14:58 smithi187 ceph-mon[130973]: pgmap v678: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:15:00.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:00 smithi003 ceph-mon[97195]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-08-11T14:15:00.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:00 smithi187 ceph-mon[130973]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-08-11T14:15:00.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:00 smithi103 ceph-mon[130434]: pgmap v679: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.31k op/s 2023-08-11T14:15:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:02 smithi003 ceph-mon[97195]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2023-08-11T14:15:02.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:02 smithi187 ceph-mon[130973]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2023-08-11T14:15:02.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:02 smithi103 ceph-mon[130434]: pgmap v680: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2023-08-11T14:15:03.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 485.0 s, 691.0 tps, lat 143.555 ms stddev 405.874 2023-08-11T14:15:04.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:04 smithi003 ceph-mon[97195]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2023-08-11T14:15:04.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:04 smithi187 ceph-mon[130973]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2023-08-11T14:15:04.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:04 smithi103 ceph-mon[130434]: pgmap v681: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 19 MiB/s wr, 1.40k op/s 2023-08-11T14:15:06.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:06 smithi103 ceph-mon[130434]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:15:07.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:06 smithi003 ceph-mon[97195]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:15:07.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:06 smithi187 ceph-mon[130973]: pgmap v682: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:15:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 490.0 s, 585.5 tps, lat 170.226 ms stddev 444.807 2023-08-11T14:15:08.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:08 smithi103 ceph-mon[130434]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:15:09.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:08 smithi003 ceph-mon[97195]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:15:09.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:08 smithi187 ceph-mon[130973]: pgmap v683: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:15:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.529+0000 7f519d1ff700 1 -- 172.21.15.3:0/1957103651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5198107920 msgr2=0x7f5198109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.529+0000 7f519d1ff700 1 --2- 172.21.15.3:0/1957103651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5198107920 0x7f5198109d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5190009f90 tx=0x7f519000b330 comp rx=0 tx=0).stop 2023-08-11T14:15:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.553+0000 7f519d1ff700 1 -- 172.21.15.3:0/1957103651 shutdown_connections 2023-08-11T14:15:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.553+0000 7f519d1ff700 1 --2- 172.21.15.3:0/1957103651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519810cc60 0x7f519810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.553+0000 7f519d1ff700 1 --2- 172.21.15.3:0/1957103651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5198107920 0x7f5198109d50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.553+0000 7f519d1ff700 1 --2- 172.21.15.3:0/1957103651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810a290 0x7f519810c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.553+0000 7f519d1ff700 1 -- 172.21.15.3:0/1957103651 >> 172.21.15.3:0/1957103651 conn(0x7f519806d750 msgr2=0x7f519806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.566+0000 7f519d1ff700 1 -- 172.21.15.3:0/1957103651 shutdown_connections 2023-08-11T14:15:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.567+0000 7f519d1ff700 1 -- 172.21.15.3:0/1957103651 wait complete. 2023-08-11T14:15:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.568+0000 7f519d1ff700 1 Processor -- start 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 -- start start 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 0x7f51981a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 0x7f51981a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810cc60 0x7f51981a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a7070 con 0x7f519810a290 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a71e0 con 0x7f519810cc60 2023-08-11T14:15:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.569+0000 7f519d1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981a73a0 con 0x7f5198107920 2023-08-11T14:15:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.570+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 0x7f51981a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.573+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 0x7f51981a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58628/0 (socket says 172.21.15.3:58628) 2023-08-11T14:15:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.573+0000 7f5196d9d700 1 -- 172.21.15.3:0/963900260 learned_addr learned my addr 172.21.15.3:0/963900260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.573+0000 7f519759e700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810cc60 0x7f51981a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.573+0000 7f519659c700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 0x7f51981a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.574+0000 7f519659c700 1 -- 172.21.15.3:0/963900260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 msgr2=0x7f51981a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.574+0000 7f519659c700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 0x7f51981a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.574+0000 7f519659c700 1 -- 172.21.15.3:0/963900260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810cc60 msgr2=0x7f51981a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.574+0000 7f519659c700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810cc60 0x7f51981a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.574+0000 7f519659c700 1 -- 172.21.15.3:0/963900260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009c40 con 0x7f519810a290 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.575+0000 7f519659c700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 0x7f51981a5b90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d950 tx=0x7f518c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.576+0000 7f517ffff700 1 -- 172.21.15.3:0/963900260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c009b40 con 0x7f519810a290 2023-08-11T14:15:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.576+0000 7f517ffff700 1 -- 172.21.15.3:0/963900260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518c00be10 con 0x7f519810a290 2023-08-11T14:15:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.576+0000 7f517ffff700 1 -- 172.21.15.3:0/963900260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00f620 con 0x7f519810a290 2023-08-11T14:15:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.576+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51981a7680 con 0x7f519810a290 2023-08-11T14:15:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.576+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519810f230 con 0x7f519810a290 2023-08-11T14:15:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.583+0000 7f51b0a29b80 1 Processor -- start 2023-08-11T14:15:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 -- start start 2023-08-11T14:15:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58df40 0x56152b58e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58e8a0 0x56152b58f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b4e75d0 con 0x56152b58df40 2023-08-11T14:15:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b4e4ef0 con 0x56152b58f790 2023-08-11T14:15:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.596+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b4e07e0 con 0x56152b58e8a0 2023-08-11T14:15:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.597+0000 7f517ffff700 1 -- 172.21.15.3:0/963900260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f518c010460 con 0x7f519810a290 2023-08-11T14:15:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.600+0000 7f517ffff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 0x7f51800a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.600+0000 7f517ffff700 1 -- 172.21.15.3:0/963900260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f518c0cb260 con 0x7f519810a290 2023-08-11T14:15:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.601+0000 7f519659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.601+0000 7f519659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50560/0 (socket says 172.21.15.3:50560) 2023-08-11T14:15:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.601+0000 7f519659c700 1 -- 172.21.15.3:0/1753922714 learned_addr learned my addr 172.21.15.3:0/1753922714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.602+0000 7f519659c700 1 -- 172.21.15.3:0/1753922714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58e8a0 msgr2=0x56152b58f1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:15:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.602+0000 7f519659c700 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58e8a0 0x56152b58f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.602+0000 7f519659c700 1 -- 172.21.15.3:0/1753922714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58df40 msgr2=0x56152b58e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.602+0000 7f519659c700 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58df40 0x56152b58e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.602+0000 7f519659c700 1 -- 172.21.15.3:0/1753922714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152b3605d0 con 0x56152b58f790 2023-08-11T14:15:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.604+0000 7f519759e700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 0x7f51800a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.604+0000 7f519659c700 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f518c095420 tx=0x7f518c00df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ed9d6dee402a89 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.608+0000 7f517d7fa700 1 -- 172.21.15.3:0/1753922714 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0c8ea0 con 0x56152b58f790 2023-08-11T14:15:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.609+0000 7f517d7fa700 1 -- 172.21.15.3:0/1753922714 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518c0990b0 con 0x56152b58f790 2023-08-11T14:15:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.609+0000 7f517d7fa700 1 -- 172.21.15.3:0/1753922714 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c098c10 con 0x56152b58f790 2023-08-11T14:15:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.610+0000 7f51b0a29b80 1 -- 172.21.15.3:0/1753922714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 msgr2=0x56152b58fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.610+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f518c095420 tx=0x7f518c00df50 comp rx=0 tx=0).stop 2023-08-11T14:15:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.613+0000 7f51b0a29b80 1 -- 172.21.15.3:0/1753922714 shutdown_connections 2023-08-11T14:15:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.613+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b58fc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.613+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58df40 0x56152b58e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.613+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/1753922714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58e8a0 0x56152b58f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.613+0000 7f51b0a29b80 1 -- 172.21.15.3:0/1753922714 >> 172.21.15.3:0/1753922714 conn(0x56152b4bb8f0 msgr2=0x56152b58a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.629+0000 7f51b0a29b80 1 -- 172.21.15.3:0/1753922714 shutdown_connections 2023-08-11T14:15:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.629+0000 7f51b0a29b80 1 -- 172.21.15.3:0/1753922714 wait complete. 2023-08-11T14:15:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.630+0000 7f51b0a29b80 1 -- start start 2023-08-11T14:15:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.630+0000 7f51b0a29b80 1 Processor -- start 2023-08-11T14:15:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b598130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58f790 0x56152b59f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b47ea60 con 0x56152b58e8a0 2023-08-11T14:15:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b47e760 con 0x56152b58df40 2023-08-11T14:15:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b3628c0 con 0x56152b58f790 2023-08-11T14:15:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.631+0000 7f51b0a29b80 10 client.0 _ll_register_callbacks cb 0x56152b481a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.641+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.641+0000 7f519759e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58f790 0x56152b59f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.642+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50570/0 (socket says 172.21.15.3:50570) 2023-08-11T14:15:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.642+0000 7f5196d9d700 1 -- 172.21.15.3:0/4094606431 learned_addr learned my addr 172.21.15.3:0/4094606431 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.647+0000 7f519659c700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b598130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.648+0000 7f5196d9d700 1 -- 172.21.15.3:0/4094606431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58f790 msgr2=0x56152b59f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.648+0000 7f5196d9d700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58f790 0x56152b59f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.648+0000 7f5196d9d700 1 -- 172.21.15.3:0/4094606431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 msgr2=0x56152b598130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.648+0000 7f5196d9d700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b598130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.648+0000 7f5196d9d700 1 -- 172.21.15.3:0/4094606431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519810f230 con 0x56152b58df40 2023-08-11T14:15:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.657+0000 7f5196d9d700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5190007d00 tx=0x7f5190007de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806b00c66055da80 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.657+0000 7f519759e700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 0x7f51800a2610 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f51880094f0 tx=0x7f5188013460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.658+0000 7f517aff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:15:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.669+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000e070 con 0x56152b58df40 2023-08-11T14:15:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.669+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f5190003e10 con 0x56152b58df40 2023-08-11T14:15:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.669+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56152b5817e0 con 0x56152b58df40 2023-08-11T14:15:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.673+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f519000b9f0 con 0x56152b58df40 2023-08-11T14:15:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.673+0000 7f51b0a29b80 10 client.26492 fetch_fsmap learned FSMap version 48 2023-08-11T14:15:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.673+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56152b30cd80 con 0x56152b58df40 2023-08-11T14:15:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.700+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5190028030 con 0x56152b58df40 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.700+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f519000b050 con 0x56152b58df40 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 10 client.26492 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 20 client.26492 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5a3b20 0x56152b5a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 4 client.26492 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:15:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.701+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56152b55d480 con 0x56152b5a3b20 2023-08-11T14:15:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.718+0000 7f519659c700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5a3b20 0x56152b5a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.730+0000 7f517aff5700 10 client.26492 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.730+0000 7f519659c700 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5a3b20 0x56152b5a5f40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882f37c0072fb4bc server_cookie=1cf3d112a27787d8 in_seq=0 out_seq=0 2023-08-11T14:15:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.757+0000 7f517aff5700 1 -- 172.21.15.3:0/4094606431 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56152b55d480 con 0x56152b5a3b20 2023-08-11T14:15:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.757+0000 7f517aff5700 10 client.26492 handle_command_reply: tid=0 2023-08-11T14:15:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.767+0000 7f51b0a29b80 1 client.26492 shutdown 2023-08-11T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.770+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 msgr2=0x56152b597bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.770+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5190007d00 tx=0x7f5190007de0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 shutdown_connections 2023-08-11T14:15:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58f790 0x56152b59f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5a3b20 0x56152b5a5f40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58df40 0x56152b597bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/4094606431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b598130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 >> 172.21.15.3:0/4094606431 conn(0x56152b4bb8f0 msgr2=0x56152b589710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.797+0000 7f517aff5700 0 client.26492 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.801+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 shutdown_connections 2023-08-11T14:15:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.805+0000 7f51b0a29b80 1 -- 172.21.15.3:0/4094606431 wait complete. 2023-08-11T14:15:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.811+0000 7f51b0a29b80 20 client.26492 trim_cache size 0 max 16384 2023-08-11T14:15:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.851+0000 7f51b0a29b80 1 Processor -- start 2023-08-11T14:15:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.851+0000 7f51b0a29b80 1 -- start start 2023-08-11T14:15:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.851+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.851+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58f790 0x56152b592cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b5aad70 con 0x56152b58f790 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b598da0 con 0x56152b58e8a0 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b58db70 con 0x56152b58df40 2023-08-11T14:15:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f519659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f519659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50580/0 (socket says 172.21.15.3:50580) 2023-08-11T14:15:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.852+0000 7f519659c700 1 -- 172.21.15.3:0/2771959275 learned_addr learned my addr 172.21.15.3:0/2771959275 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.863+0000 7f5196d9d700 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.867+0000 7f519659c700 1 -- 172.21.15.3:0/2771959275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 msgr2=0x56152b59d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.867+0000 7f519659c700 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.867+0000 7f519659c700 1 -- 172.21.15.3:0/2771959275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58f790 msgr2=0x56152b592cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.867+0000 7f519659c700 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58f790 0x56152b592cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.867+0000 7f519659c700 1 -- 172.21.15.3:0/2771959275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152b585a10 con 0x56152b58e8a0 2023-08-11T14:15:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.871+0000 7f519659c700 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f518c098940 tx=0x7f518c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c6aae972e3911f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.875+0000 7f517dffb700 1 -- 172.21.15.3:0/2771959275 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c014040 con 0x56152b58e8a0 2023-08-11T14:15:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.875+0000 7f517dffb700 1 -- 172.21.15.3:0/2771959275 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518c09c860 con 0x56152b58e8a0 2023-08-11T14:15:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.875+0000 7f517dffb700 1 -- 172.21.15.3:0/2771959275 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0c8ea0 con 0x56152b58e8a0 2023-08-11T14:15:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.876+0000 7f51b0a29b80 1 -- 172.21.15.3:0/2771959275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 msgr2=0x56152b5927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.876+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f518c098940 tx=0x7f518c000bc0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.888+0000 7f51b0a29b80 1 -- 172.21.15.3:0/2771959275 shutdown_connections 2023-08-11T14:15:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.888+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58f790 0x56152b592cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.888+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.888+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/2771959275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58e8a0 0x56152b5927b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.888+0000 7f51b0a29b80 1 -- 172.21.15.3:0/2771959275 >> 172.21.15.3:0/2771959275 conn(0x56152b4bb8f0 msgr2=0x56152b5a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.896+0000 7f51b0a29b80 1 -- 172.21.15.3:0/2771959275 shutdown_connections 2023-08-11T14:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.896+0000 7f51b0a29b80 1 -- 172.21.15.3:0/2771959275 wait complete. 2023-08-11T14:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.896+0000 7f51b0a29b80 1 -- start start 2023-08-11T14:15:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.897+0000 7f51b0a29b80 1 Processor -- start 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b59b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b59c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b59c5b0 con 0x56152b58e8a0 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b59c720 con 0x56152b58f790 2023-08-11T14:15:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152b59c8e0 con 0x56152b58df40 2023-08-11T14:15:10.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.898+0000 7f51b0a29b80 10 client.0 _ll_register_callbacks cb 0x7f5188005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.903+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.903+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58676/0 (socket says 172.21.15.3:58676) 2023-08-11T14:15:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.903+0000 7f5196d9d700 1 -- 172.21.15.3:0/3884042990 learned_addr learned my addr 172.21.15.3:0/3884042990 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.903+0000 7f519759e700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b59c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.907+0000 7f519659c700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b59b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.912+0000 7f519759e700 1 -- 172.21.15.3:0/3884042990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 msgr2=0x56152b59ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.912+0000 7f519759e700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.912+0000 7f519759e700 1 -- 172.21.15.3:0/3884042990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 msgr2=0x56152b59b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.912+0000 7f519759e700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b59b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.912+0000 7f519759e700 1 -- 172.21.15.3:0/3884042990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56152b3605d0 con 0x56152b58f790 2023-08-11T14:15:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.923+0000 7f519759e700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b59c070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5188013860 tx=0x7f5188009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9adcec35d0be5401 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.923+0000 7f517aff5700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:15:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.924+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188014e90 con 0x56152b58f790 2023-08-11T14:15:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.924+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f518801cbc0 con 0x56152b58f790 2023-08-11T14:15:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.924+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f518c0990b0 con 0x56152b58f790 2023-08-11T14:15:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.932+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518801cd80 con 0x56152b58f790 2023-08-11T14:15:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.937+0000 7f51b0a29b80 10 client.26498 fetch_fsmap learned FSMap version 48 2023-08-11T14:15:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.937+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x56152b2f5c00 con 0x56152b58f790 2023-08-11T14:15:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518c0990b0 con 0x56152b58f790 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) v1 ==== 7930+0+0 (secure 0 0 0) 0x7f5188003ce0 con 0x56152b58f790 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 10 client.26498 fetch_fsmap finished waiting for FSMap version 48 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 10 client.26498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 20 client.26498 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5f84c0 0x56152b5fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 4 client.26498 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:15:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.950+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56152b4d97c0 con 0x56152b5f84c0 2023-08-11T14:15:10.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:10.961+0000 7f5196d9d700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5f84c0 0x56152b5fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:11.007+0000 7f5196d9d700 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5f84c0 0x56152b5fa8e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd06abff8dfcfd8 server_cookie=35f63c0299e35adf in_seq=0 out_seq=0 2023-08-11T14:15:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:11.007+0000 7f517aff5700 10 client.26498 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:11.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:10 smithi003 ceph-mon[97195]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 1.56k op/s 2023-08-11T14:15:11.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:10 smithi187 ceph-mon[130973]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 1.56k op/s 2023-08-11T14:15:11.185 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:10 smithi103 ceph-mon[130434]: pgmap v684: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 21 MiB/s wr, 1.56k op/s 2023-08-11T14:15:12.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:11 smithi103 ceph-mon[130434]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:15:12.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:11 smithi003 ceph-mon[97195]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:15:12.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:11 smithi187 ceph-mon[130973]: pgmap v685: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:15:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 495.0 s, 723.4 tps, lat 87.663 ms stddev 174.509 2023-08-11T14:15:14.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:14 smithi103 ceph-mon[130434]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:15:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.754+0000 7f517aff5700 1 -- 172.21.15.3:0/3884042990 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56152b4d97c0 con 0x56152b5f84c0 2023-08-11T14:15:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.754+0000 7f517aff5700 10 client.26498 handle_command_reply: tid=0 2023-08-11T14:15:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.754+0000 7f51b0a29b80 1 client.26498 shutdown 2023-08-11T14:15:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.765+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 msgr2=0x56152b59c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.765+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b59c070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5188013860 tx=0x7f5188009260 comp rx=0 tx=0).stop 2023-08-11T14:15:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 shutdown_connections 2023-08-11T14:15:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56152b5f84c0 0x56152b5fa8e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56152b58f790 0x56152b59c070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56152b58df40 0x56152b59ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 --2- 172.21.15.3:0/3884042990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56152b58e8a0 0x56152b59b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 >> 172.21.15.3:0/3884042990 conn(0x56152b4bb8f0 msgr2=0x56152b5927b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f517aff5700 0 client.26498 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 shutdown_connections 2023-08-11T14:15:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.767+0000 7f51b0a29b80 1 -- 172.21.15.3:0/3884042990 wait complete. 2023-08-11T14:15:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.770+0000 7f51b0a29b80 20 client.26498 trim_cache size 0 max 16384 2023-08-11T14:15:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:15:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:15:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:15:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:15:14.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:14 smithi187 ceph-mon[130973]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:15:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.790+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 msgr2=0x7f51800a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.790+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 0x7f51800a2610 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f51880094f0 tx=0x7f5188013460 comp rx=0 tx=0).stop 2023-08-11T14:15:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.790+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 msgr2=0x7f51981a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.790+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 0x7f51981a5b90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d950 tx=0x7f518c00dd10 comp rx=0 tx=0).stop 2023-08-11T14:15:14.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:14 smithi003 ceph-mon[97195]: pgmap v686: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:15:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 shutdown_connections 2023-08-11T14:15:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f51800a0150 0x7f51800a2610 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f519810cc60 0x7f51981a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5198107920 0x7f51981a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 --2- 172.21.15.3:0/963900260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f519810a290 0x7f51981a5b90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:14.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.802+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 >> 172.21.15.3:0/963900260 conn(0x7f519806d750 msgr2=0x7f519810baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.808+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 shutdown_connections 2023-08-11T14:15:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:14.814+0000 7f519d1ff700 1 -- 172.21.15.3:0/963900260 wait complete. 2023-08-11T14:15:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:16 smithi187 ceph-mon[130973]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.51k op/s 2023-08-11T14:15:16.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:16 smithi003 ceph-mon[97195]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.51k op/s 2023-08-11T14:15:16.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:16 smithi103 ceph-mon[130434]: pgmap v687: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 20 MiB/s wr, 1.51k op/s 2023-08-11T14:15:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 500.0 s, 292.6 tps, lat 452.802 ms stddev 747.919 2023-08-11T14:15:18.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:18 smithi003 ceph-mon[97195]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:15:18.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:18 smithi187 ceph-mon[130973]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:15:18.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:18 smithi103 ceph-mon[130434]: pgmap v688: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:15:20.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:20 smithi103 ceph-mon[130434]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:15:21.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:20 smithi187 ceph-mon[130973]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:15:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:20 smithi003 ceph-mon[97195]: pgmap v689: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:15:22.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:22 smithi103 ceph-mon[130434]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 975 op/s 2023-08-11T14:15:23.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:22 smithi003 ceph-mon[97195]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 975 op/s 2023-08-11T14:15:23.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:22 smithi187 ceph-mon[130973]: pgmap v690: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 975 op/s 2023-08-11T14:15:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 505.0 s, 530.8 tps, lat 195.943 ms stddev 168.729 2023-08-11T14:15:25.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:24 smithi187 ceph-mon[130973]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:15:25.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:24 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:25.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:24 smithi003 ceph-mon[97195]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:15:25.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:24 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:25.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:24 smithi103 ceph-mon[130434]: pgmap v691: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:15:25.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:24 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:27.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:26 smithi003 ceph-mon[97195]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:15:27.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:26 smithi187 ceph-mon[130973]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:15:27.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:26 smithi103 ceph-mon[130434]: pgmap v692: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:15:28.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 510.0 s, 941.7 tps, lat 107.026 ms stddev 258.822 2023-08-11T14:15:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.150+0000 7fbcb8fff700 1 -- 172.21.15.3:0/4198330036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 msgr2=0x7fbcb410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.150+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/4198330036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb410d0e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbcac01c0d0 tx=0x7fbcac01c3e0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.153+0000 7fbcb8fff700 1 -- 172.21.15.3:0/4198330036 shutdown_connections 2023-08-11T14:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.153+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/4198330036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb410d0e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.153+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/4198330036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb4107920 0x7fbcb4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.153+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/4198330036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb410a290 0x7fbcb410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.153+0000 7fbcb8fff700 1 -- 172.21.15.3:0/4198330036 >> 172.21.15.3:0/4198330036 conn(0x7fbcb406d750 msgr2=0x7fbcb406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.158+0000 7fbcb8fff700 1 -- 172.21.15.3:0/4198330036 shutdown_connections 2023-08-11T14:15:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.158+0000 7fbcb8fff700 1 -- 172.21.15.3:0/4198330036 wait complete. 2023-08-11T14:15:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.163+0000 7fbcb8fff700 1 Processor -- start 2023-08-11T14:15:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.163+0000 7fbcb8fff700 1 -- start start 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb4107920 0x7fbcb419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb410a290 0x7fbcb419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb419e760 con 0x7fbcb410a290 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb419e8d0 con 0x7fbcb410cc60 2023-08-11T14:15:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb8fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb419ea90 con 0x7fbcb4107920 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51372/0 (socket says 172.21.15.3:51372) 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.164+0000 7fbcb2d9d700 1 -- 172.21.15.3:0/475038210 learned_addr learned my addr 172.21.15.3:0/475038210 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb259c700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb4107920 0x7fbcb419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb2d9d700 1 -- 172.21.15.3:0/475038210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb4107920 msgr2=0x7fbcb419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb2d9d700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb4107920 0x7fbcb419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb2d9d700 1 -- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb410a290 msgr2=0x7fbcb419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb2d9d700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb410a290 0x7fbcb419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.165+0000 7fbcb2d9d700 1 -- 172.21.15.3:0/475038210 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac01ad10 con 0x7fbcb410cc60 2023-08-11T14:15:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.166+0000 7fbcb2d9d700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcac003c30 tx=0x7fbcac003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.167+0000 7fbcab7fe700 1 -- 172.21.15.3:0/475038210 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac01f070 con 0x7fbcb410cc60 2023-08-11T14:15:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.167+0000 7fbcab7fe700 1 -- 172.21.15.3:0/475038210 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac01de60 con 0x7fbcb410cc60 2023-08-11T14:15:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.167+0000 7fbcab7fe700 1 -- 172.21.15.3:0/475038210 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac02ba90 con 0x7fbcb410cc60 2023-08-11T14:15:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.167+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb419ed10 con 0x7fbcb410cc60 2023-08-11T14:15:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.167+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb41a5480 con 0x7fbcb410cc60 2023-08-11T14:15:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.181+0000 7fbccc898b80 1 Processor -- start 2023-08-11T14:15:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.184+0000 7fbcab7fe700 1 -- 172.21.15.3:0/475038210 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbcac02bbf0 con 0x7fbcb410cc60 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.185+0000 7fbccc898b80 1 -- start start 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.185+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.185+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf9071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.186+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf907790 0x561dbf907c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.186+0000 7fbccc898b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf85f5d0 con 0x561dbf9068a0 2023-08-11T14:15:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.186+0000 7fbccc898b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf85cef0 con 0x561dbf905f40 2023-08-11T14:15:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.186+0000 7fbccc898b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf8587e0 con 0x561dbf907790 2023-08-11T14:15:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.188+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.189+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51380/0 (socket says 172.21.15.3:51380) 2023-08-11T14:15:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.189+0000 7fbcabfff700 1 -- 172.21.15.3:0/4229820078 learned_addr learned my addr 172.21.15.3:0/4229820078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.189+0000 7fbcabfff700 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf907790 0x561dbf907c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.190+0000 7fbcabfff700 1 -- 172.21.15.3:0/4229820078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf907790 msgr2=0x561dbf907c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.190+0000 7fbcabfff700 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf907790 0x561dbf907c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.190+0000 7fbcabfff700 1 -- 172.21.15.3:0/4229820078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 msgr2=0x561dbf9071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.190+0000 7fbcabfff700 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf9071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.190+0000 7fbcabfff700 1 -- 172.21.15.3:0/4229820078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbf6d85d0 con 0x561dbf905f40 2023-08-11T14:15:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.191+0000 7fbcabfff700 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0094f0 tx=0x7fbc9c009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d2df5a4543465a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.191+0000 7fbca8ff9700 1 -- 172.21.15.3:0/4229820078 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004a30 con 0x561dbf905f40 2023-08-11T14:15:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.191+0000 7fbca8ff9700 1 -- 172.21.15.3:0/4229820078 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9c019070 con 0x561dbf905f40 2023-08-11T14:15:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.191+0000 7fbca8ff9700 1 -- 172.21.15.3:0/4229820078 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00f6e0 con 0x561dbf905f40 2023-08-11T14:15:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.192+0000 7fbccc898b80 1 -- 172.21.15.3:0/4229820078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 msgr2=0x561dbf906360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.192+0000 7fbccc898b80 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c0094f0 tx=0x7fbc9c009710 comp rx=0 tx=0).stop 2023-08-11T14:15:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.193+0000 7fbccc898b80 1 -- 172.21.15.3:0/4229820078 shutdown_connections 2023-08-11T14:15:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.193+0000 7fbccc898b80 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf907790 0x561dbf907c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.193+0000 7fbccc898b80 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf905f40 0x561dbf906360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.193+0000 7fbccc898b80 1 --2- 172.21.15.3:0/4229820078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf9071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.193+0000 7fbccc898b80 1 -- 172.21.15.3:0/4229820078 >> 172.21.15.3:0/4229820078 conn(0x561dbf8338f0 msgr2=0x561dbf902920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.194+0000 7fbccc898b80 1 -- 172.21.15.3:0/4229820078 shutdown_connections 2023-08-11T14:15:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.194+0000 7fbccc898b80 1 -- 172.21.15.3:0/4229820078 wait complete. 2023-08-11T14:15:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.195+0000 7fbccc898b80 1 -- start start 2023-08-11T14:15:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.195+0000 7fbccc898b80 1 Processor -- start 2023-08-11T14:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf910180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf907790 0x561dbf9171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf7f6a60 con 0x561dbf9068a0 2023-08-11T14:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf7f6760 con 0x561dbf907790 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf6da8c0 con 0x561dbf905f40 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.196+0000 7fbccc898b80 10 client.0 _ll_register_callbacks cb 0x561dbf7f9a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.197+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.197+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35042/0 (socket says 172.21.15.3:35042) 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.197+0000 7fbcabfff700 1 -- 172.21.15.3:0/3334080103 learned_addr learned my addr 172.21.15.3:0/3334080103 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.197+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf907790 0x561dbf9171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 -- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf907790 msgr2=0x561dbf9171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf907790 0x561dbf9171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 -- 172.21.15.3:0/3334080103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 msgr2=0x561dbf910180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf910180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 -- 172.21.15.3:0/3334080103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c0135b0 con 0x561dbf905f40 2023-08-11T14:15:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.198+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c004fa0 tx=0x7fbc9c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b3ea0014c0f52c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.199+0000 7fbc99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:15:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.204+0000 7fbcab7fe700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 0x7fbca00a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.204+0000 7fbcab7fe700 1 -- 172.21.15.3:0/475038210 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbcac0dd6b0 con 0x7fbcb410cc60 2023-08-11T14:15:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.205+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004aa0 con 0x561dbf905f40 2023-08-11T14:15:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.205+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbc9c005230 con 0x561dbf905f40 2023-08-11T14:15:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.206+0000 7fbcb259c700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 0x7fbca00a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.208+0000 7fbcb259c700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 0x7fbca00a24a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbca4006fd0 tx=0x7fbca4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.208+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561dbf8f97e0 con 0x561dbf905f40 2023-08-11T14:15:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.208+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc9c0053f0 con 0x561dbf905f40 2023-08-11T14:15:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.209+0000 7fbccc898b80 10 client.26533 fetch_fsmap learned FSMap version 49 2023-08-11T14:15:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.209+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561dbf684d80 con 0x561dbf905f40 2023-08-11T14:15:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.210+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561dbf8f97e0 con 0x561dbf905f40 2023-08-11T14:15:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.210+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 49) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbc9c003b00 con 0x561dbf905f40 2023-08-11T14:15:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 10 client.26533 fetch_fsmap finished waiting for FSMap version 49 2023-08-11T14:15:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 20 client.26533 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf91b980 0x561dbf91dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 4 client.26533 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:15:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.214+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dbf8d5480 con 0x561dbf91b980 2023-08-11T14:15:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.215+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf91b980 0x561dbf91dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.216+0000 7fbcabfff700 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf91b980 0x561dbf91dda0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c2146b888f3931 server_cookie=d51d4e239977ea4 in_seq=0 out_seq=0 2023-08-11T14:15:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.216+0000 7fbc99ffb700 10 client.26533 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.216+0000 7fbc99ffb700 1 -- 172.21.15.3:0/3334080103 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561dbf8d5480 con 0x561dbf91b980 2023-08-11T14:15:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.216+0000 7fbc99ffb700 10 client.26533 handle_command_reply: tid=0 2023-08-11T14:15:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.217+0000 7fbccc898b80 1 client.26533 shutdown 2023-08-11T14:15:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.218+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 msgr2=0x561dbf90fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.218+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c004fa0 tx=0x7fbc9c009260 comp rx=0 tx=0).stop 2023-08-11T14:15:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 shutdown_connections 2023-08-11T14:15:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf91b980 0x561dbf91dda0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf907790 0x561dbf9171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf90fc40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 --2- 172.21.15.3:0/3334080103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf9068a0 0x561dbf910180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbc99ffb700 0 client.26533 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 >> 172.21.15.3:0/3334080103 conn(0x561dbf8338f0 msgr2=0x561dbf901710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.219+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 shutdown_connections 2023-08-11T14:15:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.220+0000 7fbccc898b80 1 -- 172.21.15.3:0/3334080103 wait complete. 2023-08-11T14:15:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.220+0000 7fbccc898b80 20 client.26533 trim_cache size 0 max 16384 2023-08-11T14:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.249+0000 7fbccc898b80 1 Processor -- start 2023-08-11T14:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 -- start start 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf902a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf917150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf905bb0 con 0x561dbf907790 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf90d9f0 con 0x561dbf9068a0 2023-08-11T14:15:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.260+0000 7fbccc898b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf90b300 con 0x561dbf905f40 2023-08-11T14:15:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.267+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.267+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51396/0 (socket says 172.21.15.3:51396) 2023-08-11T14:15:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.267+0000 7fbcabfff700 1 -- 172.21.15.3:0/2550971985 learned_addr learned my addr 172.21.15.3:0/2550971985 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.272+0000 7fbcabfff700 1 -- 172.21.15.3:0/2550971985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 msgr2=0x561dbf902a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.272+0000 7fbcabfff700 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf902a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.272+0000 7fbcabfff700 1 -- 172.21.15.3:0/2550971985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 msgr2=0x561dbf917150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.272+0000 7fbcabfff700 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf917150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.272+0000 7fbcabfff700 1 -- 172.21.15.3:0/2550971985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dbf66dc00 con 0x561dbf9068a0 2023-08-11T14:15:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.279+0000 7fbcabfff700 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02c620 tx=0x7fbc9c024040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d1d65c4359a653 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.286+0000 7fbca97fa700 1 -- 172.21.15.3:0/2550971985 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c003680 con 0x561dbf9068a0 2023-08-11T14:15:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.286+0000 7fbca97fa700 1 -- 172.21.15.3:0/2550971985 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9c023040 con 0x561dbf9068a0 2023-08-11T14:15:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.286+0000 7fbca97fa700 1 -- 172.21.15.3:0/2550971985 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c02d350 con 0x561dbf9068a0 2023-08-11T14:15:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.287+0000 7fbccc898b80 1 -- 172.21.15.3:0/2550971985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 msgr2=0x561dbf906320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.287+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02c620 tx=0x7fbc9c024040 comp rx=0 tx=0).stop 2023-08-11T14:15:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.299+0000 7fbccc898b80 1 -- 172.21.15.3:0/2550971985 shutdown_connections 2023-08-11T14:15:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.299+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf917150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.299+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf902a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.299+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2550971985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf906320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.299+0000 7fbccc898b80 1 -- 172.21.15.3:0/2550971985 >> 172.21.15.3:0/2550971985 conn(0x561dbf8338f0 msgr2=0x561dbf902320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.305+0000 7fbccc898b80 1 -- 172.21.15.3:0/2550971985 shutdown_connections 2023-08-11T14:15:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.316+0000 7fbccc898b80 1 -- 172.21.15.3:0/2550971985 wait complete. 2023-08-11T14:15:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.316+0000 7fbccc898b80 1 -- start start 2023-08-11T14:15:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.316+0000 7fbccc898b80 1 Processor -- start 2023-08-11T14:15:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf914b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf915090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf9165e0 con 0x561dbf907790 2023-08-11T14:15:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf916750 con 0x561dbf9068a0 2023-08-11T14:15:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dbf916910 con 0x561dbf905f40 2023-08-11T14:15:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.317+0000 7fbccc898b80 10 client.0 _ll_register_callbacks cb 0x7fbc9c004830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.318+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.319+0000 7fbcb2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54390/0 (socket says 172.21.15.3:54390) 2023-08-11T14:15:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.319+0000 7fbcb2d9d700 1 -- 172.21.15.3:0/2078085910 learned_addr learned my addr 172.21.15.3:0/2078085910 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf914b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 -- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 msgr2=0x561dbf915090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf915090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 -- 172.21.15.3:0/2078085910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 msgr2=0x561dbf9160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.323+0000 7fbcb259c700 1 -- 172.21.15.3:0/2078085910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dbf684d80 con 0x561dbf905f40 2023-08-11T14:15:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.324+0000 7fbcb2d9d700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:15:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.324+0000 7fbcb259c700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf914b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x561dbf8fc040 tx=0x7fbca4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b2ddcfab36efb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.324+0000 7fbc99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:15:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.324+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40144e0 con 0x561dbf905f40 2023-08-11T14:15:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.325+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbca4012a80 con 0x561dbf905f40 2023-08-11T14:15:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.325+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc9c019070 con 0x561dbf905f40 2023-08-11T14:15:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.326+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca4012c40 con 0x561dbf905f40 2023-08-11T14:15:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.327+0000 7fbccc898b80 10 client.26536 fetch_fsmap learned FSMap version 49 2023-08-11T14:15:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.327+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561dbf6ab1f0 con 0x561dbf905f40 2023-08-11T14:15:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9c019070 con 0x561dbf905f40 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 49) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbca4027070 con 0x561dbf905f40 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 10 client.26536 fetch_fsmap finished waiting for FSMap version 49 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 20 client.26536 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf970370 0x561dbf972790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 4 client.26536 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:15:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.328+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561dbf8517c0 con 0x561dbf970370 2023-08-11T14:15:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.331+0000 7fbcabfff700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf970370 0x561dbf972790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.332+0000 7fbcabfff700 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf970370 0x561dbf972790 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e149108e829b6dea server_cookie=b32f2a03ee24f625 in_seq=0 out_seq=0 2023-08-11T14:15:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.332+0000 7fbc99ffb700 10 client.26536 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.392+0000 7fbc99ffb700 1 -- 172.21.15.3:0/2078085910 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561dbf8517c0 con 0x561dbf970370 2023-08-11T14:15:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.392+0000 7fbc99ffb700 10 client.26536 handle_command_reply: tid=0 2023-08-11T14:15:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.392+0000 7fbccc898b80 1 client.26536 shutdown 2023-08-11T14:15:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 msgr2=0x561dbf914b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf914b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x561dbf8fc040 tx=0x7fbca4004750 comp rx=0 tx=0).stop 2023-08-11T14:15:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 shutdown_connections 2023-08-11T14:15:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561dbf907790 0x561dbf9160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561dbf905f40 0x561dbf914b50 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x561dbf8fc040 tx=0x7fbca4004750 comp rx=0 tx=0).stop 2023-08-11T14:15:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.393+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x561dbf970370 0x561dbf972790 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.394+0000 7fbccc898b80 1 --2- 172.21.15.3:0/2078085910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561dbf9068a0 0x561dbf915090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.394+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 >> 172.21.15.3:0/2078085910 conn(0x561dbf8338f0 msgr2=0x561dbf90dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.394+0000 7fbc99ffb700 0 client.26536 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.396+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 shutdown_connections 2023-08-11T14:15:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.399+0000 7fbccc898b80 1 -- 172.21.15.3:0/2078085910 wait complete. 2023-08-11T14:15:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.407+0000 7fbccc898b80 20 client.26536 trim_cache size 0 max 16384 2023-08-11T14:15:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:15:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:15:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:15:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:15:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.422+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 msgr2=0x7fbca00a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.422+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 0x7fbca00a24a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbca4006fd0 tx=0x7fbca4008040 comp rx=0 tx=0).stop 2023-08-11T14:15:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.422+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 msgr2=0x7fbcb419e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.422+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcac003c30 tx=0x7fbcac003d10 comp rx=0 tx=0).stop 2023-08-11T14:15:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 shutdown_connections 2023-08-11T14:15:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbca009ffe0 0x7fbca00a24a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcb410cc60 0x7fbcb419e050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcb4107920 0x7fbcb419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 --2- 172.21.15.3:0/475038210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcb410a290 0x7fbcb419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.429+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 >> 172.21.15.3:0/475038210 conn(0x7fbcb406d750 msgr2=0x7fbcb41082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.435+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 shutdown_connections 2023-08-11T14:15:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:28.450+0000 7fbcb8fff700 1 -- 172.21.15.3:0/475038210 wait complete. 2023-08-11T14:15:29.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:28 smithi187 ceph-mon[130973]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 846 op/s 2023-08-11T14:15:29.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:28 smithi003 ceph-mon[97195]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 846 op/s 2023-08-11T14:15:29.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:28 smithi103 ceph-mon[130434]: pgmap v693: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 11 MiB/s wr, 846 op/s 2023-08-11T14:15:31.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:30 smithi103 ceph-mon[130434]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 944 op/s 2023-08-11T14:15:31.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:30 smithi187 ceph-mon[130973]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 944 op/s 2023-08-11T14:15:31.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:30 smithi003 ceph-mon[97195]: pgmap v694: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 944 op/s 2023-08-11T14:15:32.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:31 smithi103 ceph-mon[130434]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 768 op/s 2023-08-11T14:15:32.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:31 smithi003 ceph-mon[97195]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 768 op/s 2023-08-11T14:15:32.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:31 smithi187 ceph-mon[130973]: pgmap v695: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 768 op/s 2023-08-11T14:15:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 515.0 s, 418.1 tps, lat 240.075 ms stddev 605.366 2023-08-11T14:15:34.804 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:34 smithi187 ceph-mon[130973]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 976 op/s 2023-08-11T14:15:34.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:34 smithi003 ceph-mon[97195]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 976 op/s 2023-08-11T14:15:34.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:34 smithi103 ceph-mon[130434]: pgmap v696: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 976 op/s 2023-08-11T14:15:36.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:36 smithi003 ceph-mon[97195]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-08-11T14:15:36.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:36 smithi187 ceph-mon[130973]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-08-11T14:15:36.934 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:36 smithi103 ceph-mon[130434]: pgmap v697: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-08-11T14:15:37.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:37 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:15:37.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:37 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:15:37.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:37 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 520.0 s, 511.0 tps, lat 182.691 ms stddev 391.718 2023-08-11T14:15:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:38 smithi103 ceph-mon[130434]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-08-11T14:15:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:38 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:38.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:38.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:38 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:38 smithi003 ceph-mon[97195]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-08-11T14:15:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:38 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:39.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:38 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:39.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:38 smithi187 ceph-mon[130973]: pgmap v698: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-08-11T14:15:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:38 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:15:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:38 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:40.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:40 smithi103 ceph-mon[130434]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-08-11T14:15:40.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:40 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:15:40.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:40 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:15:40.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:40 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:41.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:40 smithi003 ceph-mon[97195]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-08-11T14:15:41.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:40 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:15:41.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:40 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:15:41.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:40 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:40 smithi187 ceph-mon[130973]: pgmap v699: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.23k op/s 2023-08-11T14:15:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:40 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:15:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:40 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:15:41.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:40 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:15:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 -- 172.21.15.3:0/352232700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 msgr2=0x7f23fc110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 --2- 172.21.15.3:0/352232700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc110530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009f90 tx=0x7f23f400b330 comp rx=0 tx=0).stop 2023-08-11T14:15:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 -- 172.21.15.3:0/352232700 shutdown_connections 2023-08-11T14:15:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 --2- 172.21.15.3:0/352232700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc110530 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 --2- 172.21.15.3:0/352232700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23fc107b90 0x7f23fc109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 --2- 172.21.15.3:0/352232700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc069680 0x7f23fc10cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.755+0000 7f24027ff700 1 -- 172.21.15.3:0/352232700 >> 172.21.15.3:0/352232700 conn(0x7f23fc06daf0 msgr2=0x7f23fc06ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.756+0000 7f24027ff700 1 -- 172.21.15.3:0/352232700 shutdown_connections 2023-08-11T14:15:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.756+0000 7f24027ff700 1 -- 172.21.15.3:0/352232700 wait complete. 2023-08-11T14:15:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.758+0000 7f24027ff700 1 Processor -- start 2023-08-11T14:15:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 -- start start 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23fc069680 0x7f23fc1130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc107b90 0x7f23fc113630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc114b60 con 0x7f23fc107b90 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc114cd0 con 0x7f23fc10a5a0 2023-08-11T14:15:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.759+0000 7f24027ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc114e90 con 0x7f23fc069680 2023-08-11T14:15:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.760+0000 7f2400d9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.761+0000 7f2400d9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45788/0 (socket says 172.21.15.3:45788) 2023-08-11T14:15:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.761+0000 7f2400d9c700 1 -- 172.21.15.3:0/2657499306 learned_addr learned my addr 172.21.15.3:0/2657499306 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.761+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc107b90 0x7f23fc113630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.762+0000 7f2400d9c700 1 -- 172.21.15.3:0/2657499306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23fc069680 msgr2=0x7f23fc1130f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:15:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.762+0000 7f2400d9c700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23fc069680 0x7f23fc1130f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.762+0000 7f2400d9c700 1 -- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc107b90 msgr2=0x7f23fc113630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.762+0000 7f2400d9c700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc107b90 0x7f23fc113630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.762+0000 7f2400d9c700 1 -- 172.21.15.3:0/2657499306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f4009c40 con 0x7f23fc10a5a0 2023-08-11T14:15:41.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.764+0000 7f2400d9c700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009f60 tx=0x7f23f4004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.765+0000 7f23f97fa700 1 -- 172.21.15.3:0/2657499306 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400e040 con 0x7f23fc10a5a0 2023-08-11T14:15:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.765+0000 7f23f97fa700 1 -- 172.21.15.3:0/2657499306 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f400bbf0 con 0x7f23fc10a5a0 2023-08-11T14:15:41.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.765+0000 7f23f97fa700 1 -- 172.21.15.3:0/2657499306 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4005280 con 0x7f23fc10a5a0 2023-08-11T14:15:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.766+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc115110 con 0x7f23fc10a5a0 2023-08-11T14:15:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.766+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc115920 con 0x7f23fc10a5a0 2023-08-11T14:15:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.773+0000 7f23f97fa700 1 -- 172.21.15.3:0/2657499306 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f23f40053e0 con 0x7f23fc10a5a0 2023-08-11T14:15:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.774+0000 7f241607bb80 1 Processor -- start 2023-08-11T14:15:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.775+0000 7f241607bb80 1 -- start start 2023-08-11T14:15:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d08a0 0x558d0f1d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1295d0 con 0x558d0f1cff40 2023-08-11T14:15:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f126ef0 con 0x558d0f1d1790 2023-08-11T14:15:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.776+0000 7f241607bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1227e0 con 0x558d0f1d08a0 2023-08-11T14:15:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.777+0000 7f23f97fa700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 0x7f23e40a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.777+0000 7f23f97fa700 1 -- 172.21.15.3:0/2657499306 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23f40cc460 con 0x7f23fc10a5a0 2023-08-11T14:15:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.777+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56892/0 (socket says 172.21.15.3:56892) 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 -- 172.21.15.3:0/594681756 learned_addr learned my addr 172.21.15.3:0/594681756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 -- 172.21.15.3:0/594681756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d08a0 msgr2=0x558d0f1d11b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d08a0 0x558d0f1d11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 -- 172.21.15.3:0/594681756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 msgr2=0x558d0f1d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.778+0000 7f23fb7fe700 1 -- 172.21.15.3:0/594681756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d0efa25d0 con 0x558d0f1d1790 2023-08-11T14:15:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.779+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1d1c10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23f00109a0 tx=0x7f23f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6261f05a8e4f76d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.780+0000 7f23e27fc700 1 -- 172.21.15.3:0/594681756 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0004a30 con 0x558d0f1d1790 2023-08-11T14:15:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.780+0000 7f23e27fc700 1 -- 172.21.15.3:0/594681756 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f00054e0 con 0x558d0f1d1790 2023-08-11T14:15:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.780+0000 7f23e27fc700 1 -- 172.21.15.3:0/594681756 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000f6e0 con 0x558d0f1d1790 2023-08-11T14:15:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.780+0000 7f2400d9c700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 0x7f23e40a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.781+0000 7f241607bb80 1 -- 172.21.15.3:0/594681756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 msgr2=0x558d0f1d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.781+0000 7f241607bb80 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1d1c10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23f00109a0 tx=0x7f23f0004750 comp rx=0 tx=0).stop 2023-08-11T14:15:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.781+0000 7f2400d9c700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 0x7f23e40a2560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007900 tx=0x7f23f400bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.782+0000 7f241607bb80 1 -- 172.21.15.3:0/594681756 shutdown_connections 2023-08-11T14:15:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.782+0000 7f241607bb80 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1d1c10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.782+0000 7f241607bb80 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.782+0000 7f241607bb80 1 --2- 172.21.15.3:0/594681756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d08a0 0x558d0f1d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.782+0000 7f241607bb80 1 -- 172.21.15.3:0/594681756 >> 172.21.15.3:0/594681756 conn(0x558d0f0fd8f0 msgr2=0x558d0f1cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.784+0000 7f241607bb80 1 -- 172.21.15.3:0/594681756 shutdown_connections 2023-08-11T14:15:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.785+0000 7f241607bb80 1 -- 172.21.15.3:0/594681756 wait complete. 2023-08-11T14:15:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.786+0000 7f241607bb80 1 -- start start 2023-08-11T14:15:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.786+0000 7f241607bb80 1 Processor -- start 2023-08-11T14:15:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 0x558d0f1e1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f0c0a60 con 0x558d0f1cff40 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f0c0760 con 0x558d0f1d08a0 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0efa48c0 con 0x558d0f1d1790 2023-08-11T14:15:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f241607bb80 10 client.0 _ll_register_callbacks cb 0x558d0f0c3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f23fbfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.788+0000 7f23fbfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45810/0 (socket says 172.21.15.3:45810) 2023-08-11T14:15:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 -- 172.21.15.3:0/1195398670 learned_addr learned my addr 172.21.15.3:0/1195398670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 -- 172.21.15.3:0/1195398670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 msgr2=0x558d0f1e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 0x558d0f1e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 0x558d0f1e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 -- 172.21.15.3:0/1195398670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 msgr2=0x558d0f1d9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fbfff700 1 -- 172.21.15.3:0/1195398670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f0010680 con 0x558d0f1d08a0 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.789+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 0x558d0f1e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:15:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23fbfff700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009ee0 tx=0x7f23ec00e5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=495a009e428797 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004220 con 0x558d0f1d08a0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d0f1c37e0 con 0x558d0f1d08a0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23ec004380 con 0x558d0f1d08a0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ec004540 con 0x558d0f1d08a0 2023-08-11T14:15:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f241607bb80 10 client.26525 fetch_fsmap learned FSMap version 50 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x558d0ef4ed80 con 0x558d0f1d08a0 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.790+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d0f1c37e0 con 0x558d0f1d08a0 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 50) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f23ec00c070 con 0x558d0f1d08a0 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 10 client.26525 fetch_fsmap finished waiting for FSMap version 50 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 20 client.26525 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f1e5b80 0x558d0f1e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 4 client.26525 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:15:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d0f19f480 con 0x558d0f1e5b80 2023-08-11T14:15:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.791+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f1e5b80 0x558d0f1e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.795+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f1e5b80 0x558d0f1e7fa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f54dbf81b193821 server_cookie=66547fb151c62d5f in_seq=0 out_seq=0 2023-08-11T14:15:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.795+0000 7f23db7fe700 10 client.26525 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.819+0000 7f23db7fe700 1 -- 172.21.15.3:0/1195398670 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558d0f19f480 con 0x558d0f1e5b80 2023-08-11T14:15:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.819+0000 7f23db7fe700 10 client.26525 handle_command_reply: tid=0 2023-08-11T14:15:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.824+0000 7f241607bb80 1 client.26525 shutdown 2023-08-11T14:15:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.831+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 msgr2=0x558d0f1da1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.831+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009ee0 tx=0x7f23ec00e5d0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 shutdown_connections 2023-08-11T14:15:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1d1790 0x558d0f1e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1d9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f1e5b80 0x558d0f1e7fa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 --2- 172.21.15.3:0/1195398670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d08a0 0x558d0f1da1e0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009ee0 tx=0x7f23ec00e5d0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.833+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 >> 172.21.15.3:0/1195398670 conn(0x558d0f0fd8f0 msgr2=0x558d0f1cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.837+0000 7f23db7fe700 0 client.26525 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.845+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 shutdown_connections 2023-08-11T14:15:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.849+0000 7f241607bb80 1 -- 172.21.15.3:0/1195398670 wait complete. 2023-08-11T14:15:41.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.866+0000 7f241607bb80 20 client.26525 trim_cache size 0 max 16384 2023-08-11T14:15:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.903+0000 7f241607bb80 1 Processor -- start 2023-08-11T14:15:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.903+0000 7f241607bb80 1 -- start start 2023-08-11T14:15:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.904+0000 7f241607bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.904+0000 7f241607bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 0x558d0f1e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.905+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.905+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:49838/0 (socket says 172.21.15.3:49838) 2023-08-11T14:15:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.909+0000 7f23fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 0x558d0f1e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.909+0000 7f23fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 0x558d0f1e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56912/0 (socket says 172.21.15.3:56912) 2023-08-11T14:15:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f241607bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1ecb70 0x558d0f1d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f241607bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1e2ee0 con 0x558d0f1d1790 2023-08-11T14:15:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f241607bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1d0fa0 con 0x558d0f1ecb70 2023-08-11T14:15:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f241607bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1d7980 con 0x558d0f1cff40 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 -- 172.21.15.3:0/4048448329 learned_addr learned my addr 172.21.15.3:0/4048448329 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1ecb70 0x558d0f1d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 -- 172.21.15.3:0/4048448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1ecb70 msgr2=0x558d0f1d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1ecb70 0x558d0f1d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 -- 172.21.15.3:0/4048448329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 msgr2=0x558d0f1e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 0x558d0f1e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.910+0000 7f23fb7fe700 1 -- 172.21.15.3:0/4048448329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d0ef37c00 con 0x558d0f1cff40 2023-08-11T14:15:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.911+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005950 tx=0x7f23f00058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63deb314d404f01c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.911+0000 7f23e2ffd700 1 -- 172.21.15.3:0/4048448329 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00050e0 con 0x558d0f1cff40 2023-08-11T14:15:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.911+0000 7f23e2ffd700 1 -- 172.21.15.3:0/4048448329 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f0023040 con 0x558d0f1cff40 2023-08-11T14:15:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.911+0000 7f23e2ffd700 1 -- 172.21.15.3:0/4048448329 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000f7b0 con 0x558d0f1cff40 2023-08-11T14:15:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.912+0000 7f241607bb80 1 -- 172.21.15.3:0/4048448329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 msgr2=0x558d0f1d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.912+0000 7f241607bb80 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23f0005950 tx=0x7f23f00058e0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.916+0000 7f241607bb80 1 -- 172.21.15.3:0/4048448329 shutdown_connections 2023-08-11T14:15:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.916+0000 7f241607bb80 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1ecb70 0x558d0f1d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.916+0000 7f241607bb80 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1cff40 0x558d0f1d0320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.916+0000 7f241607bb80 1 --2- 172.21.15.3:0/4048448329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1d1790 0x558d0f1e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.916+0000 7f241607bb80 1 -- 172.21.15.3:0/4048448329 >> 172.21.15.3:0/4048448329 conn(0x558d0f0fd8f0 msgr2=0x558d0f1d89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.918+0000 7f241607bb80 1 -- 172.21.15.3:0/4048448329 shutdown_connections 2023-08-11T14:15:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.919+0000 7f241607bb80 1 -- 172.21.15.3:0/4048448329 wait complete. 2023-08-11T14:15:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.919+0000 7f241607bb80 1 -- start start 2023-08-11T14:15:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.920+0000 7f241607bb80 1 Processor -- start 2023-08-11T14:15:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.920+0000 7f241607bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 0x558d0f1e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1e0b40 con 0x558d0f1cff40 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1e0cb0 con 0x558d0f1d1790 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d0f1e0e70 con 0x558d0f1ecb70 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f241607bb80 10 client.0 _ll_register_callbacks cb 0x558d0f0c3a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 0x558d0f1e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f23fb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 0x558d0f1e0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:49842/0 (socket says 172.21.15.3:49842) 2023-08-11T14:15:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.921+0000 7f23fb7fe700 1 -- 172.21.15.3:0/3677201399 learned_addr learned my addr 172.21.15.3:0/3677201399 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.922+0000 7f23fbfff700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.922+0000 7f23fbfff700 1 -- 172.21.15.3:0/3677201399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 msgr2=0x558d0f1e0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.922+0000 7f23fbfff700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 0x558d0f1e0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.922+0000 7f23fbfff700 1 -- 172.21.15.3:0/3677201399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 msgr2=0x558d0f1df0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.923+0000 7f23fbfff700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.923+0000 7f23fbfff700 1 -- 172.21.15.3:0/3677201399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d0efa25d0 con 0x558d0f1d1790 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.923+0000 7f23fbfff700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1df5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0118a0 tx=0x7f23ec004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68093a62cb8cfaee server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.923+0000 7f23db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:15:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.924+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec011c10 con 0x558d0f1d1790 2023-08-11T14:15:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.924+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23ec011e40 con 0x558d0f1d1790 2023-08-11T14:15:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.924+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23f00054e0 con 0x558d0f1d1790 2023-08-11T14:15:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.925+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ec011680 con 0x558d0f1d1790 2023-08-11T14:15:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.926+0000 7f241607bb80 10 client.26531 fetch_fsmap learned FSMap version 50 2023-08-11T14:15:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.926+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x558d0ef751f0 con 0x558d0f1d1790 2023-08-11T14:15:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.927+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f00054e0 con 0x558d0f1d1790 2023-08-11T14:15:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.927+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 50) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f23ec04d070 con 0x558d0f1d1790 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 10 client.26531 fetch_fsmap finished waiting for FSMap version 50 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 10 client.26531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 20 client.26531 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f23a6c0 0x558d0f23cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 4 client.26531 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:15:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.929+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d0f11b7c0 con 0x558d0f23a6c0 2023-08-11T14:15:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.933+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f23a6c0 0x558d0f23cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.934+0000 7f23fb7fe700 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f23a6c0 0x558d0f23cae0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e59047573106c7 server_cookie=b4124c86ea75f9ec in_seq=0 out_seq=0 2023-08-11T14:15:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.935+0000 7f23db7fe700 10 client.26531 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.966+0000 7f23db7fe700 1 -- 172.21.15.3:0/3677201399 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d0f11b7c0 con 0x558d0f23a6c0 2023-08-11T14:15:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.966+0000 7f23db7fe700 10 client.26531 handle_command_reply: tid=0 2023-08-11T14:15:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.968+0000 7f241607bb80 1 client.26531 shutdown 2023-08-11T14:15:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.968+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 msgr2=0x558d0f1df5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.968+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1df5f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0118a0 tx=0x7f23ec004690 comp rx=0 tx=0).stop 2023-08-11T14:15:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 shutdown_connections 2023-08-11T14:15:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d0f1ecb70 0x558d0f1e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d0f1cff40 0x558d0f1df0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d0f23a6c0 0x558d0f23cae0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 --2- 172.21.15.3:0/3677201399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d0f1d1790 0x558d0f1df5f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 >> 172.21.15.3:0/3677201399 conn(0x558d0f0fd8f0 msgr2=0x558d0f1d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f23db7fe700 0 client.26531 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 shutdown_connections 2023-08-11T14:15:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.969+0000 7f241607bb80 1 -- 172.21.15.3:0/3677201399 wait complete. 2023-08-11T14:15:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.971+0000 7f241607bb80 20 client.26531 trim_cache size 0 max 16384 2023-08-11T14:15:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:15:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:15:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:15:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:15:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 msgr2=0x7f23e40a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 0x7f23e40a2560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007900 tx=0x7f23f400bf60 comp rx=0 tx=0).stop 2023-08-11T14:15:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 msgr2=0x7f23fc114500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009f60 tx=0x7f23f4004c60 comp rx=0 tx=0).stop 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 shutdown_connections 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23e40a00a0 0x7f23e40a2560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23fc10a5a0 0x7f23fc114500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23fc069680 0x7f23fc1130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 --2- 172.21.15.3:0/2657499306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23fc107b90 0x7f23fc113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.972+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 >> 172.21.15.3:0/2657499306 conn(0x7f23fc06daf0 msgr2=0x7f23fc10edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.973+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 shutdown_connections 2023-08-11T14:15:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:41.973+0000 7f24027ff700 1 -- 172.21.15.3:0/2657499306 wait complete. 2023-08-11T14:15:42.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:42 smithi103 ceph-mon[130434]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-08-11T14:15:43.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:42 smithi003 ceph-mon[97195]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-08-11T14:15:43.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:42 smithi187 ceph-mon[130973]: pgmap v700: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-08-11T14:15:43.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 525.0 s, 962.8 tps, lat 110.545 ms stddev 177.832 2023-08-11T14:15:44.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:44 smithi103 ceph-mon[130434]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:15:45.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:44 smithi003 ceph-mon[97195]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:15:45.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:44 smithi187 ceph-mon[130973]: pgmap v701: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:15:47.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:46 smithi003 ceph-mon[97195]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-08-11T14:15:47.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:46 smithi187 ceph-mon[130973]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-08-11T14:15:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:46 smithi103 ceph-mon[130434]: pgmap v702: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-08-11T14:15:48.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 530.0 s, 551.7 tps, lat 121.723 ms stddev 396.786 2023-08-11T14:15:49.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:48 smithi103 ceph-mon[130434]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 883 op/s 2023-08-11T14:15:49.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:48 smithi003 ceph-mon[97195]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 883 op/s 2023-08-11T14:15:49.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:48 smithi187 ceph-mon[130973]: pgmap v703: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 883 op/s 2023-08-11T14:15:51.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:50 smithi103 ceph-mon[130434]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:15:51.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:50 smithi003 ceph-mon[97195]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:15:51.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:50 smithi187 ceph-mon[130973]: pgmap v704: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:15:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 535.0 s, 1085.1 tps, lat 117.249 ms stddev 392.980 2023-08-11T14:15:53.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:52 smithi103 ceph-mon[130434]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-08-11T14:15:53.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:52 smithi003 ceph-mon[97195]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-08-11T14:15:53.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:52 smithi187 ceph-mon[130973]: pgmap v705: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-08-11T14:15:54.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:54 smithi003 ceph-mon[97195]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:15:54.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:54 smithi187 ceph-mon[130973]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:15:54.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:54 smithi103 ceph-mon[130434]: pgmap v706: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:15:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.287+0000 7f9f8abff700 1 -- 172.21.15.3:0/1937135039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f84107920 msgr2=0x7f9f84109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.287+0000 7f9f8abff700 1 --2- 172.21.15.3:0/1937135039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f84107920 0x7f9f84109d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c01a9a0 tx=0x7f9f6c01acb0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.291+0000 7f9f8abff700 1 -- 172.21.15.3:0/1937135039 shutdown_connections 2023-08-11T14:15:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.292+0000 7f9f8abff700 1 --2- 172.21.15.3:0/1937135039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8410cc60 0x7f9f8410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.292+0000 7f9f8abff700 1 --2- 172.21.15.3:0/1937135039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f84107920 0x7f9f84109d50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.292+0000 7f9f8abff700 1 --2- 172.21.15.3:0/1937135039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f8410a290 0x7f9f8410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.292+0000 7f9f8abff700 1 -- 172.21.15.3:0/1937135039 >> 172.21.15.3:0/1937135039 conn(0x7f9f8406d750 msgr2=0x7f9f8406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.296+0000 7f9f8abff700 1 -- 172.21.15.3:0/1937135039 shutdown_connections 2023-08-11T14:15:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.303+0000 7f9f8abff700 1 -- 172.21.15.3:0/1937135039 wait complete. 2023-08-11T14:15:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.305+0000 7f9f8abff700 1 Processor -- start 2023-08-11T14:15:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.314+0000 7f9f8abff700 1 -- start start 2023-08-11T14:15:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f8410a290 0x7f9f8419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8410cc60 0x7f9f8419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8419e7d0 con 0x7f9f84107920 2023-08-11T14:15:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8419e940 con 0x7f9f8410a290 2023-08-11T14:15:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.315+0000 7f9f8abff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8419eb00 con 0x7f9f8410cc60 2023-08-11T14:15:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.316+0000 7f9f8899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.316+0000 7f9f8899b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59890/0 (socket says 172.21.15.3:59890) 2023-08-11T14:15:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.316+0000 7f9f8899b700 1 -- 172.21.15.3:0/2202762588 learned_addr learned my addr 172.21.15.3:0/2202762588 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.317+0000 7f9f8899b700 1 -- 172.21.15.3:0/2202762588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8410cc60 msgr2=0x7f9f8419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.317+0000 7f9f8899b700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8410cc60 0x7f9f8419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.317+0000 7f9f8899b700 1 -- 172.21.15.3:0/2202762588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f8410a290 msgr2=0x7f9f8419d320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:15:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.317+0000 7f9f8899b700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f8410a290 0x7f9f8419d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.317+0000 7f9f8899b700 1 -- 172.21.15.3:0/2202762588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c01a600 con 0x7f9f84107920 2023-08-11T14:15:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.318+0000 7f9f8899b700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c003790 tx=0x7f9f6c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.318+0000 7f9f81ffb700 1 -- 172.21.15.3:0/2202762588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02d070 con 0x7f9f84107920 2023-08-11T14:15:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.318+0000 7f9f81ffb700 1 -- 172.21.15.3:0/2202762588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6c031410 con 0x7f9f84107920 2023-08-11T14:15:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.318+0000 7f9f81ffb700 1 -- 172.21.15.3:0/2202762588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0283f0 con 0x7f9f84107920 2023-08-11T14:15:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.327+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8419ed80 con 0x7f9f84107920 2023-08-11T14:15:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.327+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f841a5520 con 0x7f9f84107920 2023-08-11T14:15:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 Processor -- start 2023-08-11T14:15:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 -- start start 2023-08-11T14:15:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a10790 0x5619f5a10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f59685d0 con 0x5619f5a10790 2023-08-11T14:15:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5965ef0 con 0x5619f5a0ef40 2023-08-11T14:15:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.334+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f59617e0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.335+0000 7f9f8899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.335+0000 7f9f8899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35892/0 (socket says 172.21.15.3:35892) 2023-08-11T14:15:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.335+0000 7f9f8899b700 1 -- 172.21.15.3:0/4169112215 learned_addr learned my addr 172.21.15.3:0/4169112215 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.339+0000 7f9f8919c700 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.339+0000 7f9f81ffb700 1 -- 172.21.15.3:0/2202762588 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9f6c031580 con 0x7f9f84107920 2023-08-11T14:15:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.340+0000 7f9f8899b700 1 -- 172.21.15.3:0/4169112215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0ef40 msgr2=0x5619f5a0f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.340+0000 7f9f8899b700 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.340+0000 7f9f8899b700 1 -- 172.21.15.3:0/4169112215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a10790 msgr2=0x5619f5a10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.340+0000 7f9f8899b700 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a10790 0x5619f5a10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.340+0000 7f9f8899b700 1 -- 172.21.15.3:0/4169112215 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f57e15d0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.342+0000 7f9f81ffb700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 0x7f9f700a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.342+0000 7f9f81ffb700 1 -- 172.21.15.3:0/2202762588 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f6c0e2660 con 0x7f9f84107920 2023-08-11T14:15:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.344+0000 7f9f8899b700 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0dfd80 tx=0x7f9f6c0e4f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e8d5bcd8afe806 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.345+0000 7f9f7affd700 1 -- 172.21.15.3:0/4169112215 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0ac070 con 0x5619f5a0f8a0 2023-08-11T14:15:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.345+0000 7f9f7affd700 1 -- 172.21.15.3:0/4169112215 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6c040040 con 0x5619f5a0f8a0 2023-08-11T14:15:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.345+0000 7f9f7affd700 1 -- 172.21.15.3:0/4169112215 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0b0430 con 0x5619f5a0f8a0 2023-08-11T14:15:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.346+0000 7f9f9e619b80 1 -- 172.21.15.3:0/4169112215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 msgr2=0x5619f5a101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.346+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0dfd80 tx=0x7f9f6c0e4f80 comp rx=0 tx=0).stop 2023-08-11T14:15:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f8919c700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 0x7f9f700a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f9e619b80 1 -- 172.21.15.3:0/4169112215 shutdown_connections 2023-08-11T14:15:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a10790 0x5619f5a10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/4169112215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a101b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.348+0000 7f9f9e619b80 1 -- 172.21.15.3:0/4169112215 >> 172.21.15.3:0/4169112215 conn(0x5619f593c8f0 msgr2=0x5619f5a0b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.349+0000 7f9f9e619b80 1 -- 172.21.15.3:0/4169112215 shutdown_connections 2023-08-11T14:15:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.349+0000 7f9f9e619b80 1 -- 172.21.15.3:0/4169112215 wait complete. 2023-08-11T14:15:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.350+0000 7f9f9e619b80 1 -- start start 2023-08-11T14:15:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.350+0000 7f9f9e619b80 1 Processor -- start 2023-08-11T14:15:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a1fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f58ffa60 con 0x5619f5a0ef40 2023-08-11T14:15:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f58ff760 con 0x5619f5a0f8a0 2023-08-11T14:15:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f57e38c0 con 0x5619f5a10790 2023-08-11T14:15:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.351+0000 7f9f9e619b80 10 client.0 _ll_register_callbacks cb 0x5619f5902a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.352+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.352+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40832/0 (socket says 172.21.15.3:40832) 2023-08-11T14:15:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.352+0000 7f9f83fff700 1 -- 172.21.15.3:0/366500515 learned_addr learned my addr 172.21.15.3:0/366500515 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.352+0000 7f9f8919c700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a1fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 -- 172.21.15.3:0/366500515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 msgr2=0x5619f5a1fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a1fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 -- 172.21.15.3:0/366500515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 msgr2=0x5619f5a18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 -- 172.21.15.3:0/366500515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f841a5520 con 0x5619f5a0f8a0 2023-08-11T14:15:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f83fff700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b840 tx=0x7f9f7400bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84daccec5e54f33 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.353+0000 7f9f63fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:15:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.354+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400f870 con 0x5619f5a0f8a0 2023-08-11T14:15:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.354+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f74006e40 con 0x5619f5a0f8a0 2023-08-11T14:15:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.354+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f5a027e0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.355+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f7400f9d0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.355+0000 7f9f8919c700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 0x7f9f700a2610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00af70 tx=0x7f9f7c00a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.363+0000 7f9f9e619b80 10 client.26540 fetch_fsmap learned FSMap version 50 2023-08-11T14:15:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.363+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5619f578dd80 con 0x5619f5a0f8a0 2023-08-11T14:15:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.367+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619f5a027e0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.368+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 50) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9f74015020 con 0x5619f5a0f8a0 2023-08-11T14:15:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 10 client.26540 fetch_fsmap finished waiting for FSMap version 50 2023-08-11T14:15:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 20 client.26540 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a24790 0x5619f5a26bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 4 client.26540 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:15:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f59de480 con 0x5619f5a24790 2023-08-11T14:15:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.369+0000 7f9f8899b700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a24790 0x5619f5a26bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.376+0000 7f9f8899b700 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a24790 0x5619f5a26bb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3458bd288a732033 server_cookie=ce5d6488f1748ec5 in_seq=0 out_seq=0 2023-08-11T14:15:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.376+0000 7f9f63fff700 10 client.26540 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.383+0000 7f9f63fff700 1 -- 172.21.15.3:0/366500515 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5619f59de480 con 0x5619f5a24790 2023-08-11T14:15:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.383+0000 7f9f63fff700 10 client.26540 handle_command_reply: tid=0 2023-08-11T14:15:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.383+0000 7f9f9e619b80 1 client.26540 shutdown 2023-08-11T14:15:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 msgr2=0x5619f5a18e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400b840 tx=0x7f9f7400bc00 comp rx=0 tx=0).stop 2023-08-11T14:15:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 shutdown_connections 2023-08-11T14:15:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a1fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a24790 0x5619f5a26bb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/366500515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a18e40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 >> 172.21.15.3:0/366500515 conn(0x5619f593c8f0 msgr2=0x5619f5a0a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.384+0000 7f9f63fff700 0 client.26540 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.392+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 shutdown_connections 2023-08-11T14:15:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.392+0000 7f9f9e619b80 1 -- 172.21.15.3:0/366500515 wait complete. 2023-08-11T14:15:55.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.395+0000 7f9f9e619b80 20 client.26540 trim_cache size 0 max 16384 2023-08-11T14:15:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.436+0000 7f9f9e619b80 1 Processor -- start 2023-08-11T14:15:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 -- start start 2023-08-11T14:15:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a04ea0 con 0x5619f5a0ef40 2023-08-11T14:15:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a186d0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.445+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a0fc80 con 0x5619f5a10790 2023-08-11T14:15:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.446+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.446+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:40848/0 (socket says 172.21.15.3:40848) 2023-08-11T14:15:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.446+0000 7f9f83fff700 1 -- 172.21.15.3:0/1929194989 learned_addr learned my addr 172.21.15.3:0/1929194989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.448+0000 7f9f8919c700 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.448+0000 7f9f83fff700 1 -- 172.21.15.3:0/1929194989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 msgr2=0x5619f5a18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.448+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.448+0000 7f9f83fff700 1 -- 172.21.15.3:0/1929194989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 msgr2=0x5619f5a0f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.448+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.449+0000 7f9f83fff700 1 -- 172.21.15.3:0/1929194989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f5776c00 con 0x5619f5a0f8a0 2023-08-11T14:15:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.449+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f74021e70 tx=0x7f9f74017480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555610b2b56dcae0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.450+0000 7f9f7b7fe700 1 -- 172.21.15.3:0/1929194989 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f740159e0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.450+0000 7f9f7b7fe700 1 -- 172.21.15.3:0/1929194989 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f74015320 con 0x5619f5a0f8a0 2023-08-11T14:15:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.450+0000 7f9f7b7fe700 1 -- 172.21.15.3:0/1929194989 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74017b70 con 0x5619f5a0f8a0 2023-08-11T14:15:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.451+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1929194989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 msgr2=0x5619f5a1fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.451+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f74021e70 tx=0x7f9f74017480 comp rx=0 tx=0).stop 2023-08-11T14:15:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.460+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1929194989 shutdown_connections 2023-08-11T14:15:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.460+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a10790 0x5619f5a18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.460+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a0f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.460+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1929194989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1fe40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.460+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1929194989 >> 172.21.15.3:0/1929194989 conn(0x5619f593c8f0 msgr2=0x5619f5a17690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.461+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1929194989 shutdown_connections 2023-08-11T14:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.462+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1929194989 wait complete. 2023-08-11T14:15:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.462+0000 7f9f9e619b80 1 -- start start 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 Processor -- start 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a1bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a10790 0x5619f5a1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a1f810 con 0x5619f5a0ef40 2023-08-11T14:15:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a1f980 con 0x5619f5a10790 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5a1fb40 con 0x5619f5a0f8a0 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.463+0000 7f9f9e619b80 10 client.0 _ll_register_callbacks cb 0x7f9f74009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.464+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.464+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:35936/0 (socket says 172.21.15.3:35936) 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.464+0000 7f9f83fff700 1 -- 172.21.15.3:0/1121688193 learned_addr learned my addr 172.21.15.3:0/1121688193 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.464+0000 7f9f8919c700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a10790 0x5619f5a1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f8899b700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a1bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 -- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a10790 msgr2=0x5619f5a1f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a10790 0x5619f5a1f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 -- 172.21.15.3:0/1121688193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 msgr2=0x5619f5a1bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a1bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 -- 172.21.15.3:0/1121688193 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619f578dd80 con 0x5619f5a0f8a0 2023-08-11T14:15:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.465+0000 7f9f83fff700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009fd0 tx=0x7f9f7400c0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786494f0c54c2467 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:15:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.466+0000 7f9f63fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:15:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.467+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400f670 con 0x5619f5a0f8a0 2023-08-11T14:15:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.467+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9f74026070 con 0x5619f5a0f8a0 2023-08-11T14:15:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.467+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f6c040040 con 0x5619f5a0f8a0 2023-08-11T14:15:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.468+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f7400f7d0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.470+0000 7f9f9e619b80 10 client.26557 fetch_fsmap learned FSMap version 50 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.470+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5619f57b41f0 con 0x5619f5a0f8a0 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6c040040 con 0x5619f5a0f8a0 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 50) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9f7400b920 con 0x5619f5a0f8a0 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 10 client.26557 fetch_fsmap finished waiting for FSMap version 50 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 20 client.26557 populate_metadata read hostname 'smithi003' 2023-08-11T14:15:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a9b080 0x5619f5a9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 4 client.26557 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.471+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619f595a7c0 con 0x5619f5a9b080 2023-08-11T14:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.472+0000 7f9f8899b700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a9b080 0x5619f5a9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.475+0000 7f9f8899b700 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a9b080 0x5619f5a9d4a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8865963f6bdf93 server_cookie=b36e571e547c8f8f in_seq=0 out_seq=0 2023-08-11T14:15:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.475+0000 7f9f63fff700 10 client.26557 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.762+0000 7f9f63fff700 1 -- 172.21.15.3:0/1121688193 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619f595a7c0 con 0x5619f5a9b080 2023-08-11T14:15:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.763+0000 7f9f63fff700 10 client.26557 handle_command_reply: tid=0 2023-08-11T14:15:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.763+0000 7f9f9e619b80 1 client.26557 shutdown 2023-08-11T14:15:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.763+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 msgr2=0x5619f5a1c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.763+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009fd0 tx=0x7f9f7400c0e0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 shutdown_connections 2023-08-11T14:15:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5619f5a9b080 0x5619f5a9d4a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619f5a10790 0x5619f5a1f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5619f5a0ef40 0x5619f5a1bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 --2- 172.21.15.3:0/1121688193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619f5a0f8a0 0x5619f5a1c2b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 >> 172.21.15.3:0/1121688193 conn(0x5619f593c8f0 msgr2=0x5619f5a136b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f63fff700 0 client.26557 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:15:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 shutdown_connections 2023-08-11T14:15:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.764+0000 7f9f9e619b80 1 -- 172.21.15.3:0/1121688193 wait complete. 2023-08-11T14:15:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f9e619b80 20 client.26557 trim_cache size 0 max 16384 2023-08-11T14:15:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:15:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:15:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:15:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:15:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 msgr2=0x7f9f700a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 0x7f9f700a2610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00af70 tx=0x7f9f7c00a8d0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 msgr2=0x7f9f8419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:15:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c003790 tx=0x7f9f6c003c40 comp rx=0 tx=0).stop 2023-08-11T14:15:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.765+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 shutdown_connections 2023-08-11T14:15:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f8410cc60 0x7f9f8419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f84107920 0x7f9f8419cde0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f9f700a0150 0x7f9f700a2610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 --2- 172.21.15.3:0/2202762588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f8410a290 0x7f9f8419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:15:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 >> 172.21.15.3:0/2202762588 conn(0x7f9f8406d750 msgr2=0x7f9f84108370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:15:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 shutdown_connections 2023-08-11T14:15:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:15:55.766+0000 7f9f8abff700 1 -- 172.21.15.3:0/2202762588 wait complete. 2023-08-11T14:15:56.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:56 smithi103 ceph-mon[130434]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:15:57.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:56 smithi003 ceph-mon[97195]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:15:57.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:56 smithi187 ceph-mon[130973]: pgmap v707: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 19 MiB/s wr, 1.48k op/s 2023-08-11T14:15:58.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 540.0 s, 630.1 tps, lat 159.925 ms stddev 491.942 2023-08-11T14:15:58.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:15:58 smithi103 ceph-mon[130434]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-08-11T14:15:59.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:15:58 smithi003 ceph-mon[97195]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-08-11T14:15:59.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:15:58 smithi187 ceph-mon[130973]: pgmap v708: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.59k op/s 2023-08-11T14:16:00.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:00 smithi103 ceph-mon[130434]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-08-11T14:16:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:00 smithi003 ceph-mon[97195]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-08-11T14:16:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:00 smithi187 ceph-mon[130973]: pgmap v709: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 18 MiB/s wr, 1.52k op/s 2023-08-11T14:16:02.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:02 smithi103 ceph-mon[130434]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:16:03.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:02 smithi003 ceph-mon[97195]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:16:03.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:02 smithi187 ceph-mon[130973]: pgmap v710: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:16:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 545.0 s, 721.6 tps, lat 141.418 ms stddev 322.129 2023-08-11T14:16:05.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:04 smithi187 ceph-mon[130973]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:16:05.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:04 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:05.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:04 smithi003 ceph-mon[97195]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:16:05.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:04 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:05.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:04 smithi103 ceph-mon[130434]: pgmap v711: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:16:05.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:04 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:07.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:06 smithi103 ceph-mon[130434]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:16:07.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:06 smithi003 ceph-mon[97195]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:16:07.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:06 smithi187 ceph-mon[130973]: pgmap v712: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-08-11T14:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 550.0 s, 150.4 tps, lat 618.628 ms stddev 631.294 2023-08-11T14:16:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.086+0000 7f7dd0bff700 1 -- 172.21.15.3:0/2997986917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc107920 msgr2=0x7f7dcc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.086+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/2997986917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc107920 0x7f7dcc109d50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b0a0 tx=0x7f7dbc00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 -- 172.21.15.3:0/2997986917 shutdown_connections 2023-08-11T14:16:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/2997986917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/2997986917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc107920 0x7f7dcc109d50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/2997986917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 -- 172.21.15.3:0/2997986917 >> 172.21.15.3:0/2997986917 conn(0x7f7dcc06d750 msgr2=0x7f7dcc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 -- 172.21.15.3:0/2997986917 shutdown_connections 2023-08-11T14:16:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.087+0000 7f7dd0bff700 1 -- 172.21.15.3:0/2997986917 wait complete. 2023-08-11T14:16:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.089+0000 7f7dd0bff700 1 Processor -- start 2023-08-11T14:16:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.089+0000 7f7dd0bff700 1 -- start start 2023-08-11T14:16:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc107920 0x7f7dcc1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc1a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1a70a0 con 0x7f7dcc107920 2023-08-11T14:16:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1a7210 con 0x7f7dcc10cc60 2023-08-11T14:16:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dd0bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1a73d0 con 0x7f7dcc10a290 2023-08-11T14:16:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc1a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc1a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52742/0 (socket says 172.21.15.3:52742) 2023-08-11T14:16:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dcad9d700 1 -- 172.21.15.3:0/3962254850 learned_addr learned my addr 172.21.15.3:0/3962254850 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.090+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dca59c700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc107920 0x7f7dcc1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/3962254850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 msgr2=0x7f7dcc1a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc1a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc107920 msgr2=0x7f7dcc1a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc107920 0x7f7dcc1a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.091+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/3962254850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc009d00 con 0x7f7dcc10a290 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.092+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc1a5b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.093+0000 7f7dc3fff700 1 -- 172.21.15.3:0/3962254850 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400c9c0 con 0x7f7dcc10a290 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.093+0000 7f7dc3fff700 1 -- 172.21.15.3:0/3962254850 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7db4010430 con 0x7f7dcc10a290 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.093+0000 7f7dc3fff700 1 -- 172.21.15.3:0/3962254850 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4018680 con 0x7f7dcc10a290 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.093+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc1a76b0 con 0x7f7dcc10a290 2023-08-11T14:16:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.093+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc078d60 con 0x7f7dcc10a290 2023-08-11T14:16:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.107+0000 7f7dc3fff700 1 -- 172.21.15.3:0/3962254850 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f7db400cb20 con 0x7f7dcc10a290 2023-08-11T14:16:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.108+0000 7f7de45d1b80 1 Processor -- start 2023-08-11T14:16:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.110+0000 7f7dc3fff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 0x7f7db80a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.110+0000 7f7dc3fff700 1 -- 172.21.15.3:0/3962254850 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7db4014070 con 0x7f7dcc10a290 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 -- start start 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017822360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c0178231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01777b5d0 con 0x55c0178228a0 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017778ef0 con 0x55c017823790 2023-08-11T14:16:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0177747e0 con 0x55c017821f40 2023-08-11T14:16:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.112+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 0x7f7db80a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.113+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.113+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52748/0 (socket says 172.21.15.3:52748) 2023-08-11T14:16:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.113+0000 7f7dcad9d700 1 -- 172.21.15.3:0/45647571 learned_addr learned my addr 172.21.15.3:0/45647571 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.113+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c0178231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.113+0000 7f7dca59c700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017822360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 -- 172.21.15.3:0/45647571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 msgr2=0x55c017822360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017822360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 -- 172.21.15.3:0/45647571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 msgr2=0x55c0178231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c0178231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 -- 172.21.15.3:0/45647571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0175f45d0 con 0x55c017823790 2023-08-11T14:16:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c0178231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:16:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.114+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013b40 tx=0x7f7dc4013e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3917773082c70361 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.115+0000 7f7dc0d99700 1 -- 172.21.15.3:0/45647571 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400fdb0 con 0x55c017823790 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.115+0000 7f7dc0d99700 1 -- 172.21.15.3:0/45647571 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc4013180 con 0x55c017823790 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.115+0000 7f7dc0d99700 1 -- 172.21.15.3:0/45647571 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401f400 con 0x55c017823790 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 -- 172.21.15.3:0/45647571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 msgr2=0x55c017823c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013b40 tx=0x7f7dc4013e50 comp rx=0 tx=0).stop 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 0x7f7db80a2610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400f6a0 tx=0x7f7dc4011000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 -- 172.21.15.3:0/45647571 shutdown_connections 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017823c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017822360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/45647571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c0178231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 -- 172.21.15.3:0/45647571 >> 172.21.15.3:0/45647571 conn(0x55c01774f8f0 msgr2=0x55c01781e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.116+0000 7f7de45d1b80 1 -- 172.21.15.3:0/45647571 shutdown_connections 2023-08-11T14:16:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.117+0000 7f7de45d1b80 1 -- 172.21.15.3:0/45647571 wait complete. 2023-08-11T14:16:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.117+0000 7f7de45d1b80 1 -- start start 2023-08-11T14:16:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 Processor -- start 2023-08-11T14:16:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c017821f40 0x55c01783cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017844200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017712a60 con 0x55c017821f40 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017712760 con 0x55c017823790 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0175f68c0 con 0x55c0178228a0 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.118+0000 7f7de45d1b80 10 client.0 _ll_register_callbacks cb 0x55c017715a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.119+0000 7f7dc1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.119+0000 7f7dc1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52880/0 (socket says 172.21.15.3:52880) 2023-08-11T14:16:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.119+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1639557843 learned_addr learned my addr 172.21.15.3:0/1639557843 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dca59c700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c017821f40 0x55c01783cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017844200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 msgr2=0x55c017844200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017844200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1639557843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c017821f40 msgr2=0x55c01783cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c017821f40 0x55c01783cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.120+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1639557843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc401b040 con 0x55c0178228a0 2023-08-11T14:16:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.123+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7db4095690 tx=0x7f7db4099160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5619a5d00344bd0a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.124+0000 7f7db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:16:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.124+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4098ca0 con 0x55c0178228a0 2023-08-11T14:16:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.124+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7db40ca040 con 0x55c0178228a0 2023-08-11T14:16:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.125+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0178157e0 con 0x55c0178228a0 2023-08-11T14:16:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.126+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7db409ca50 con 0x55c0178228a0 2023-08-11T14:16:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.127+0000 7f7de45d1b80 10 client.26569 fetch_fsmap learned FSMap version 52 2023-08-11T14:16:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.127+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c0175a0d80 con 0x55c0178228a0 2023-08-11T14:16:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.127+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0178157e0 con 0x55c0178228a0 2023-08-11T14:16:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 52) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7db40a06c0 con 0x55c0178228a0 2023-08-11T14:16:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 10 client.26569 fetch_fsmap finished waiting for FSMap version 52 2023-08-11T14:16:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 10 client.26569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 20 client.26569 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c017848a20 0x55c01784ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 4 client.26569 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:16:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.128+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0177f1480 con 0x55c017848a20 2023-08-11T14:16:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.129+0000 7f7dca59c700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c017848a20 0x55c01784ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.129+0000 7f7dca59c700 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c017848a20 0x55c01784ae40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e2a4bfa5d2f872 server_cookie=3c50c5a4c47d51dd in_seq=0 out_seq=0 2023-08-11T14:16:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.129+0000 7f7db1ffb700 10 client.26569 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.130+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1639557843 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c0177f1480 con 0x55c017848a20 2023-08-11T14:16:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.130+0000 7f7db1ffb700 10 client.26569 handle_command_reply: tid=0 2023-08-11T14:16:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.130+0000 7f7de45d1b80 1 client.26569 shutdown 2023-08-11T14:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 msgr2=0x55c01783d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7db4095690 tx=0x7f7db4099160 comp rx=0 tx=0).stop 2023-08-11T14:16:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 shutdown_connections 2023-08-11T14:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c017848a20 0x55c01784ae40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017844200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c017821f40 0x55c01783cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1639557843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c0178228a0 0x55c01783d1c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 >> 172.21.15.3:0/1639557843 conn(0x55c01774f8f0 msgr2=0x55c01781d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7db1ffb700 0 client.26569 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 shutdown_connections 2023-08-11T14:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.131+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1639557843 wait complete. 2023-08-11T14:16:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.132+0000 7f7de45d1b80 20 client.26569 trim_cache size 0 max 16384 2023-08-11T14:16:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 Processor -- start 2023-08-11T14:16:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 -- start start 2023-08-11T14:16:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017823b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017845e60 con 0x55c0178228a0 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01783c020 con 0x55c017823790 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.162+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017817ea0 con 0x55c017821f40 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017823b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017823b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52894/0 (socket says 172.21.15.3:52894) 2023-08-11T14:16:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dcad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:52764/0 (socket says 172.21.15.3:52764) 2023-08-11T14:16:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dca59c700 1 -- 172.21.15.3:0/3615883090 learned_addr learned my addr 172.21.15.3:0/3615883090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.163+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.164+0000 7f7dcad9d700 1 -- 172.21.15.3:0/3615883090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 msgr2=0x55c017823b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.164+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017823b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.164+0000 7f7dcad9d700 1 -- 172.21.15.3:0/3615883090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 msgr2=0x55c017842110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.164+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.164+0000 7f7dcad9d700 1 -- 172.21.15.3:0/3615883090 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c017589c00 con 0x55c017823790 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.165+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013b10 tx=0x7f7dc4009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577e766b01285c46 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.165+0000 7f7dc159a700 1 -- 172.21.15.3:0/3615883090 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401f400 con 0x55c017823790 2023-08-11T14:16:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.165+0000 7f7dc159a700 1 -- 172.21.15.3:0/3615883090 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc40099d0 con 0x55c017823790 2023-08-11T14:16:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.166+0000 7f7dc159a700 1 -- 172.21.15.3:0/3615883090 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4010710 con 0x55c017823790 2023-08-11T14:16:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.166+0000 7f7de45d1b80 1 -- 172.21.15.3:0/3615883090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 msgr2=0x55c0178266b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.166+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4013b10 tx=0x7f7dc4009660 comp rx=0 tx=0).stop 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 -- 172.21.15.3:0/3615883090 shutdown_connections 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c0178266b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017823b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/3615883090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 -- 172.21.15.3:0/3615883090 >> 172.21.15.3:0/3615883090 conn(0x55c01774f8f0 msgr2=0x55c01783cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 -- 172.21.15.3:0/3615883090 shutdown_connections 2023-08-11T14:16:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 -- 172.21.15.3:0/3615883090 wait complete. 2023-08-11T14:16:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.167+0000 7f7de45d1b80 1 -- start start 2023-08-11T14:16:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 Processor -- start 2023-08-11T14:16:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017841c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017840240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017840950 con 0x55c0178228a0 2023-08-11T14:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017840ac0 con 0x55c017823790 2023-08-11T14:16:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c017840c80 con 0x55c017821f40 2023-08-11T14:16:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.168+0000 7f7de45d1b80 10 client.0 _ll_register_callbacks cb 0x7f7db40987f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.169+0000 7f7dca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017841c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.169+0000 7f7dca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017841c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52900/0 (socket says 172.21.15.3:52900) 2023-08-11T14:16:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.169+0000 7f7dca59c700 1 -- 172.21.15.3:0/1991498571 learned_addr learned my addr 172.21.15.3:0/1991498571 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.169+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017840240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.169+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1991498571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 msgr2=0x55c017841c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017841c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 msgr2=0x55c017840240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017840240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 -- 172.21.15.3:0/1991498571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0175f45d0 con 0x55c0178228a0 2023-08-11T14:16:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dcad9d700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017840240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:16:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7dc1d9b700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7db4095640 tx=0x7f7db40c8310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafac3d3beb598cd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.170+0000 7f7db1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db409ca50 con 0x55c0178228a0 2023-08-11T14:16:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f7db40ca040 con 0x55c0178228a0 2023-08-11T14:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c01776d7c0 con 0x55c0178228a0 2023-08-11T14:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7db4098c20 con 0x55c0178228a0 2023-08-11T14:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7de45d1b80 10 client.16407 fetch_fsmap learned FSMap version 52 2023-08-11T14:16:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.171+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c0175c71f0 con 0x55c0178228a0 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c01776d7c0 con 0x55c0178228a0 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 52) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7db40995b0 con 0x55c0178228a0 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 52 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 20 client.16407 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:09.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c01789d730 0x55c01789fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 4 client.16407 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c01772dff0 con 0x55c01789d730 2023-08-11T14:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.172+0000 7f7dca59c700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c01789d730 0x55c01789fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.173+0000 7f7dca59c700 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c01789d730 0x55c01789fb50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5058920c0c57b47b server_cookie=ab365eabd02bdce2 in_seq=0 out_seq=0 2023-08-11T14:16:09.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.173+0000 7f7db1ffb700 10 client.16407 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.189+0000 7f7db1ffb700 1 -- 172.21.15.3:0/1991498571 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c01772dff0 con 0x55c01789d730 2023-08-11T14:16:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.189+0000 7f7db1ffb700 10 client.16407 handle_command_reply: tid=0 2023-08-11T14:16:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.189+0000 7f7de45d1b80 1 client.16407 shutdown 2023-08-11T14:16:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 msgr2=0x55c017842150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842150 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7db4095640 tx=0x7f7db40c8310 comp rx=0 tx=0).stop 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 shutdown_connections 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c01789d730 0x55c01789fb50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c017823790 0x55c017840240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c017821f40 0x55c017841c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 --2- 172.21.15.3:0/1991498571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c0178228a0 0x55c017842150 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7db1ffb700 0 client.16407 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 >> 172.21.15.3:0/1991498571 conn(0x55c01774f8f0 msgr2=0x55c01783cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 shutdown_connections 2023-08-11T14:16:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.190+0000 7f7de45d1b80 1 -- 172.21.15.3:0/1991498571 wait complete. 2023-08-11T14:16:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.192+0000 7f7de45d1b80 20 client.16407 trim_cache size 0 max 16384 2023-08-11T14:16:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:16:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:16:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:16:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:16:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.192+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 msgr2=0x7f7db80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.192+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 0x7f7db80a2610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400f6a0 tx=0x7f7dc4011000 comp rx=0 tx=0).stop 2023-08-11T14:16:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.192+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 msgr2=0x7f7dcc1a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.192+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc1a5b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400ed00 comp rx=0 tx=0).stop 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 shutdown_connections 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f7db80a0150 0x7f7db80a2610 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dcc10cc60 0x7f7dcc1a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dcc107920 0x7f7dcc1a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 --2- 172.21.15.3:0/3962254850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7dcc10a290 0x7f7dcc1a5b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 >> 172.21.15.3:0/3962254850 conn(0x7f7dcc06d750 msgr2=0x7f7dcc10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 shutdown_connections 2023-08-11T14:16:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:09.193+0000 7f7dd0bff700 1 -- 172.21.15.3:0/3962254850 wait complete. 2023-08-11T14:16:09.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:08 smithi003 ceph-mon[97195]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:16:09.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:08 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:09.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:08 smithi187 ceph-mon[130973]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:16:09.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:08 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:08 smithi103 ceph-mon[130434]: pgmap v713: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 19 MiB/s wr, 1.35k op/s 2023-08-11T14:16:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:08 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:10.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:10 smithi003 ceph-mon[97195]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-08-11T14:16:10.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:10 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:10.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:10 smithi187 ceph-mon[130973]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-08-11T14:16:10.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:10 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:10 smithi103 ceph-mon[130434]: pgmap v714: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.18k op/s 2023-08-11T14:16:10.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:10 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:12 smithi103 ceph-mon[130434]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:13.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:12 smithi003 ceph-mon[97195]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:13.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:12 smithi187 ceph-mon[130973]: pgmap v715: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:13.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 555.0 s, 394.5 tps, lat 273.963 ms stddev 201.394 2023-08-11T14:16:14.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:14 smithi103 ceph-mon[130434]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:16:15.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:14 smithi003 ceph-mon[97195]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:16:15.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:14 smithi187 ceph-mon[130973]: pgmap v716: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:16:16.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:16 smithi103 ceph-mon[130434]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1k op/s 2023-08-11T14:16:17.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:16 smithi003 ceph-mon[97195]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1k op/s 2023-08-11T14:16:17.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:16 smithi187 ceph-mon[130973]: pgmap v717: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1k op/s 2023-08-11T14:16:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 560.0 s, 985.0 tps, lat 99.069 ms stddev 87.490 2023-08-11T14:16:18.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:18 smithi103 ceph-mon[130434]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-08-11T14:16:19.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:18 smithi003 ceph-mon[97195]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-08-11T14:16:19.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:18 smithi187 ceph-mon[130973]: pgmap v718: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 13 MiB/s wr, 999 op/s 2023-08-11T14:16:21.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:20 smithi187 ceph-mon[130973]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 786 op/s 2023-08-11T14:16:21.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:20 smithi003 ceph-mon[97195]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 786 op/s 2023-08-11T14:16:21.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:20 smithi103 ceph-mon[130434]: pgmap v719: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 786 op/s 2023-08-11T14:16:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.525+0000 7f6146fff700 1 -- 172.21.15.3:0/644383794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 msgr2=0x7f614010f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.525+0000 7f6146fff700 1 --2- 172.21.15.3:0/644383794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f614010f4d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6130008570 tx=0x7f613000af90 comp rx=0 tx=0).stop 2023-08-11T14:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.531+0000 7f6146fff700 1 -- 172.21.15.3:0/644383794 shutdown_connections 2023-08-11T14:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.531+0000 7f6146fff700 1 --2- 172.21.15.3:0/644383794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f614010f4d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.531+0000 7f6146fff700 1 --2- 172.21.15.3:0/644383794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6140107b90 0x7f6140109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.531+0000 7f6146fff700 1 --2- 172.21.15.3:0/644383794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 0x7f614010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.531+0000 7f6146fff700 1 -- 172.21.15.3:0/644383794 >> 172.21.15.3:0/644383794 conn(0x7f614006daf0 msgr2=0x7f614006ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.550+0000 7f6146fff700 1 -- 172.21.15.3:0/644383794 shutdown_connections 2023-08-11T14:16:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.556+0000 7f6146fff700 1 -- 172.21.15.3:0/644383794 wait complete. 2023-08-11T14:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.557+0000 7f6146fff700 1 Processor -- start 2023-08-11T14:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 -- start start 2023-08-11T14:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6140107b90 0x7f61401166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 0x7f6140116be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6140118110 con 0x7f6140107b90 2023-08-11T14:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6140118280 con 0x7f614010d010 2023-08-11T14:16:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.558+0000 7f6146fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6140118440 con 0x7f614010a550 2023-08-11T14:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46574/0 (socket says 172.21.15.3:46574) 2023-08-11T14:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 -- 172.21.15.3:0/2359474983 learned_addr learned my addr 172.21.15.3:0/2359474983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f613ffff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 0x7f6140116be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 -- 172.21.15.3:0/2359474983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 msgr2=0x7f6140116be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 0x7f6140116be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 -- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6140107b90 msgr2=0x7f61401166a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6140107b90 0x7f61401166a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.559+0000 7f614559c700 1 -- 172.21.15.3:0/2359474983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300081d0 con 0x7f614010d010 2023-08-11T14:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.560+0000 7f614559c700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f613000b3a0 tx=0x7f6130003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.560+0000 7f613dffb700 1 -- 172.21.15.3:0/2359474983 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130005240 con 0x7f614010d010 2023-08-11T14:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.561+0000 7f613dffb700 1 -- 172.21.15.3:0/2359474983 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6130004660 con 0x7f614010d010 2023-08-11T14:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.561+0000 7f613dffb700 1 -- 172.21.15.3:0/2359474983 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613000b7b0 con 0x7f614010d010 2023-08-11T14:16:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.561+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61401186c0 con 0x7f614010d010 2023-08-11T14:16:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.561+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6140118f50 con 0x7f614010d010 2023-08-11T14:16:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.567+0000 7f615a8e9b80 1 Processor -- start 2023-08-11T14:16:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 -- start start 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973aa9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aaa790 0x563973aaac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973a025d0 con 0x563973aaa790 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639739ffef0 con 0x563973aa8f40 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.570+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639739fb7e0 con 0x563973aa98a0 2023-08-11T14:16:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.571+0000 7f6144d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.571+0000 7f6144d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36948/0 (socket says 172.21.15.3:36948) 2023-08-11T14:16:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.571+0000 7f6144d9b700 1 -- 172.21.15.3:0/1121704928 learned_addr learned my addr 172.21.15.3:0/1121704928 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.572+0000 7f613ffff700 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973aa9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.572+0000 7f613ffff700 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aaa790 0x563973aaac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 -- 172.21.15.3:0/1121704928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 msgr2=0x563973aa9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973aa9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 -- 172.21.15.3:0/1121704928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aaa790 msgr2=0x563973aaac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aaa790 0x563973aaac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 -- 172.21.15.3:0/1121704928 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56397387b5d0 con 0x563973aa98a0 2023-08-11T14:16:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.573+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6138009d50 tx=0x7f613800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d6dd7c6b741e3d server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.574+0000 7f6136ffd700 1 -- 172.21.15.3:0/1121704928 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800e070 con 0x563973aa98a0 2023-08-11T14:16:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.574+0000 7f6136ffd700 1 -- 172.21.15.3:0/1121704928 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6138012430 con 0x563973aa98a0 2023-08-11T14:16:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.574+0000 7f6136ffd700 1 -- 172.21.15.3:0/1121704928 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138009ea0 con 0x563973aa98a0 2023-08-11T14:16:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.575+0000 7f613dffb700 1 -- 172.21.15.3:0/2359474983 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6130018040 con 0x7f614010d010 2023-08-11T14:16:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.575+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1121704928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 msgr2=0x563973aaa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.575+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6138009d50 tx=0x7f613800b330 comp rx=0 tx=0).stop 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1121704928 shutdown_connections 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aaa790 0x563973aaac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973aa9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1121704928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aa98a0 0x563973aaa1b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1121704928 >> 172.21.15.3:0/1121704928 conn(0x5639739d68f0 msgr2=0x563973aa5920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1121704928 shutdown_connections 2023-08-11T14:16:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.576+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1121704928 wait complete. 2023-08-11T14:16:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.577+0000 7f615a8e9b80 1 -- start start 2023-08-11T14:16:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.578+0000 7f615a8e9b80 1 Processor -- start 2023-08-11T14:16:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973999a60 con 0x563973aa8f40 2023-08-11T14:16:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973999760 con 0x563973aa98a0 2023-08-11T14:16:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56397387d8c0 con 0x563973aaa790 2023-08-11T14:16:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.579+0000 7f615a8e9b80 10 client.0 _ll_register_callbacks cb 0x56397399ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.580+0000 7f6144d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.580+0000 7f6144d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:46580/0 (socket says 172.21.15.3:46580) 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.580+0000 7f6144d9b700 1 -- 172.21.15.3:0/3760608809 learned_addr learned my addr 172.21.15.3:0/3760608809 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.580+0000 7f613ffff700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.580+0000 7f613ffff700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f6144d9b700 1 -- 172.21.15.3:0/3760608809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 msgr2=0x563973aba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f6144d9b700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f6144d9b700 1 -- 172.21.15.3:0/3760608809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 msgr2=0x563973ab2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f6144d9b700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f6144d9b700 1 -- 172.21.15.3:0/3760608809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6138009a00 con 0x563973aa98a0 2023-08-11T14:16:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f613dffb700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 0x7f612c0a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.581+0000 7f613dffb700 1 -- 172.21.15.3:0/2359474983 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f61300cb760 con 0x7f614010d010 2023-08-11T14:16:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f6144d9b700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6138012a10 tx=0x7f613800bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31d93e2093cbdf5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f613ffff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 0x7f612c0a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f611ffff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800e040 con 0x563973aa98a0 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6138006e40 con 0x563973aa98a0 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.582+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563973a9c7e0 con 0x563973aa98a0 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.583+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6138009ea0 con 0x563973aa98a0 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.583+0000 7f615a8e9b80 10 client.26576 fetch_fsmap learned FSMap version 53 2023-08-11T14:16:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.583+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x563973827d80 con 0x563973aa98a0 2023-08-11T14:16:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.583+0000 7f613ffff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 0x7f612c0a2470 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x563973aa9f70 tx=0x7f61280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613802b030 con 0x563973aa98a0 2023-08-11T14:16:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 53) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f613801a870 con 0x563973aa98a0 2023-08-11T14:16:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 10 client.26576 fetch_fsmap finished waiting for FSMap version 53 2023-08-11T14:16:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 20 client.26576 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973abe940 0x563973ac0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 4 client.26576 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:16:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.584+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563973a78480 con 0x563973abe940 2023-08-11T14:16:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.585+0000 7f613ffff700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973abe940 0x563973ac0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.586+0000 7f613ffff700 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973abe940 0x563973ac0d60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5622e76ec2bbec4 server_cookie=e81a6c2b9d01474a in_seq=0 out_seq=0 2023-08-11T14:16:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.587+0000 7f611ffff700 10 client.26576 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.587+0000 7f611ffff700 1 -- 172.21.15.3:0/3760608809 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563973a78480 con 0x563973abe940 2023-08-11T14:16:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.587+0000 7f611ffff700 10 client.26576 handle_command_reply: tid=0 2023-08-11T14:16:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.587+0000 7f615a8e9b80 1 client.26576 shutdown 2023-08-11T14:16:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 msgr2=0x563973ab3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6138012a10 tx=0x7f613800bd50 comp rx=0 tx=0).stop 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 shutdown_connections 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973abe940 0x563973ac0d60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/3760608809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 >> 172.21.15.3:0/3760608809 conn(0x5639739d68f0 msgr2=0x563973aa4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f611ffff700 0 client.26576 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.588+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 shutdown_connections 2023-08-11T14:16:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.589+0000 7f615a8e9b80 1 -- 172.21.15.3:0/3760608809 wait complete. 2023-08-11T14:16:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.589+0000 7f615a8e9b80 20 client.26576 trim_cache size 0 max 16384 2023-08-11T14:16:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.604+0000 7f615a8e9b80 1 Processor -- start 2023-08-11T14:16:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.604+0000 7f615a8e9b80 1 -- start start 2023-08-11T14:16:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.604+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973aa9c80 con 0x563973aa8f40 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973ab0950 con 0x563973aa98a0 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973ab3d90 con 0x563973aaa790 2023-08-11T14:16:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f6144d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36974/0 (socket says 172.21.15.3:36974) 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 -- 172.21.15.3:0/516025811 learned_addr learned my addr 172.21.15.3:0/516025811 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f613ffff700 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 -- 172.21.15.3:0/516025811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 msgr2=0x563973ab3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 -- 172.21.15.3:0/516025811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 msgr2=0x563973ab2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.605+0000 7f614559c700 1 -- 172.21.15.3:0/516025811 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563973810c00 con 0x563973aaa790 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.606+0000 7f613ffff700 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.606+0000 7f614559c700 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6130095930 tx=0x7f6130098920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2f5603c8b66cf0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.606+0000 7f61377fe700 1 -- 172.21.15.3:0/516025811 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130098ec0 con 0x563973aaa790 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.606+0000 7f61377fe700 1 -- 172.21.15.3:0/516025811 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613009ccf0 con 0x563973aaa790 2023-08-11T14:16:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.606+0000 7f61377fe700 1 -- 172.21.15.3:0/516025811 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300983b0 con 0x563973aaa790 2023-08-11T14:16:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 -- 172.21.15.3:0/516025811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 msgr2=0x563973aba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6130095930 tx=0x7f6130098920 comp rx=0 tx=0).stop 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 -- 172.21.15.3:0/516025811 shutdown_connections 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973aba180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa8f40 0x563973ab2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/516025811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa98a0 0x563973ab3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 -- 172.21.15.3:0/516025811 >> 172.21.15.3:0/516025811 conn(0x5639739d68f0 msgr2=0x563973ab0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 -- 172.21.15.3:0/516025811 shutdown_connections 2023-08-11T14:16:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.607+0000 7f615a8e9b80 1 -- 172.21.15.3:0/516025811 wait complete. 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 -- start start 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 Processor -- start 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973ab5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 0x563973ab61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973ab9710 con 0x563973aa98a0 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973ab9880 con 0x563973aa8f40 2023-08-11T14:16:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973ab9a40 con 0x563973aaa790 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.608+0000 7f615a8e9b80 10 client.0 _ll_register_callbacks cb 0x7f612800d930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36978/0 (socket says 172.21.15.3:36978) 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 -- 172.21.15.3:0/1295220002 learned_addr learned my addr 172.21.15.3:0/1295220002 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f613ffff700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 0x563973ab61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973ab5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 -- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 msgr2=0x563973ab5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973ab5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 -- 172.21.15.3:0/1295220002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 msgr2=0x563973ab61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 0x563973ab61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f614559c700 1 -- 172.21.15.3:0/1295220002 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6140118f50 con 0x563973aaa790 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.609+0000 7f613ffff700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 0x563973ab61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f614559c700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61300953c0 tx=0x7f6130011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9063ef74e06e7679 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f611ffff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:16:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61300a09b0 con 0x563973aaa790 2023-08-11T14:16:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6138012430 con 0x563973aaa790 2023-08-11T14:16:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6130098ec0 con 0x563973aaa790 2023-08-11T14:16:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61300c8f60 con 0x563973aaa790 2023-08-11T14:16:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f615a8e9b80 10 client.26590 fetch_fsmap learned FSMap version 53 2023-08-11T14:16:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.610+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x56397384e1f0 con 0x563973aaa790 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.611+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6138012430 con 0x563973aaa790 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 53) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6130093d00 con 0x563973aaa790 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 10 client.26590 fetch_fsmap finished waiting for FSMap version 53 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 10 client.26590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 20 client.26590 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973b12dd0 0x563973b151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 4 client.26590 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:16:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639739f47c0 con 0x563973b12dd0 2023-08-11T14:16:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.612+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973b12dd0 0x563973b151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.614+0000 7f6144d9b700 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973b12dd0 0x563973b151f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b4b004f44e78cb server_cookie=a57e81dd93089979 in_seq=0 out_seq=0 2023-08-11T14:16:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.614+0000 7f611ffff700 10 client.26590 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.657+0000 7f611ffff700 1 -- 172.21.15.3:0/1295220002 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639739f47c0 con 0x563973b12dd0 2023-08-11T14:16:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.657+0000 7f611ffff700 10 client.26590 handle_command_reply: tid=0 2023-08-11T14:16:22.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.657+0000 7f615a8e9b80 1 client.26590 shutdown 2023-08-11T14:16:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.658+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 msgr2=0x563973ab91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.658+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61300953c0 tx=0x7f6130011000 comp rx=0 tx=0).stop 2023-08-11T14:16:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 shutdown_connections 2023-08-11T14:16:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563973aaa790 0x563973ab91d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x563973b12dd0 0x563973b151f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563973aa8f40 0x563973ab5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 --2- 172.21.15.3:0/1295220002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563973aa98a0 0x563973ab61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 >> 172.21.15.3:0/1295220002 conn(0x5639739d68f0 msgr2=0x563973aad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f611ffff700 0 client.26590 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 shutdown_connections 2023-08-11T14:16:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.659+0000 7f615a8e9b80 1 -- 172.21.15.3:0/1295220002 wait complete. 2023-08-11T14:16:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.660+0000 7f615a8e9b80 20 client.26590 trim_cache size 0 max 16384 2023-08-11T14:16:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:16:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:16:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:16:22.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:16:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 msgr2=0x7f612c0a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 0x7f612c0a2470 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x563973aa9f70 tx=0x7f61280058e0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 msgr2=0x7f6140117ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f613000b3a0 tx=0x7f6130003730 comp rx=0 tx=0).stop 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 shutdown_connections 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f612c09ffb0 0x7f612c0a2470 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614010d010 0x7f6140117ab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6140107b90 0x7f61401166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 --2- 172.21.15.3:0/2359474983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f614010a550 0x7f6140116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.665+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 >> 172.21.15.3:0/2359474983 conn(0x7f614006daf0 msgr2=0x7f6140108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.667+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 shutdown_connections 2023-08-11T14:16:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:22.668+0000 7f6146fff700 1 -- 172.21.15.3:0/2359474983 wait complete. 2023-08-11T14:16:23.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:22 smithi003 ceph-mon[97195]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:23.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:22 smithi187 ceph-mon[130973]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 565.0 s, 462.8 tps, lat 224.388 ms stddev 577.580 2023-08-11T14:16:23.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:22 smithi103 ceph-mon[130434]: pgmap v720: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:25.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:24 smithi187 ceph-mon[130973]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:25.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:24 smithi003 ceph-mon[97195]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:25.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:24 smithi103 ceph-mon[130434]: pgmap v721: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.00k op/s 2023-08-11T14:16:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:26 smithi103 ceph-mon[130434]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 968 op/s 2023-08-11T14:16:27.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:26 smithi003 ceph-mon[97195]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 968 op/s 2023-08-11T14:16:27.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:26 smithi187 ceph-mon[130973]: pgmap v722: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 968 op/s 2023-08-11T14:16:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 570.0 s, 802.2 tps, lat 86.749 ms stddev 87.055 2023-08-11T14:16:29.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:28 smithi103 ceph-mon[130434]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 964 op/s 2023-08-11T14:16:29.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:28 smithi003 ceph-mon[97195]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 964 op/s 2023-08-11T14:16:29.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:28 smithi187 ceph-mon[130973]: pgmap v723: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 964 op/s 2023-08-11T14:16:31.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:30 smithi103 ceph-mon[130434]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 965 op/s 2023-08-11T14:16:31.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:30 smithi003 ceph-mon[97195]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 965 op/s 2023-08-11T14:16:31.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:30 smithi187 ceph-mon[130973]: pgmap v724: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 965 op/s 2023-08-11T14:16:33.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 575.0 s, 854.0 tps, lat 152.187 ms stddev 424.666 2023-08-11T14:16:33.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:32 smithi103 ceph-mon[130434]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:16:33.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:32 smithi187 ceph-mon[130973]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:16:33.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:32 smithi003 ceph-mon[97195]: pgmap v725: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 19 MiB/s wr, 1.39k op/s 2023-08-11T14:16:34.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:33 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:34.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:33 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:34.298 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:33 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:35.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:34 smithi003 ceph-mon[97195]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:16:35.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:34 smithi187 ceph-mon[130973]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:16:35.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:34 smithi103 ceph-mon[130434]: pgmap v726: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:16:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.038+0000 7faee77ff700 1 -- 172.21.15.3:0/2550517980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee0107920 msgr2=0x7faee0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.038+0000 7faee77ff700 1 --2- 172.21.15.3:0/2550517980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee0107920 0x7faee0109d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faed800b0a0 tx=0x7faed800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.045+0000 7faee77ff700 1 -- 172.21.15.3:0/2550517980 shutdown_connections 2023-08-11T14:16:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.045+0000 7faee77ff700 1 --2- 172.21.15.3:0/2550517980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010cc60 0x7faee010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.045+0000 7faee77ff700 1 --2- 172.21.15.3:0/2550517980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee0107920 0x7faee0109d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.045+0000 7faee77ff700 1 --2- 172.21.15.3:0/2550517980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee010a290 0x7faee010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.045+0000 7faee77ff700 1 -- 172.21.15.3:0/2550517980 >> 172.21.15.3:0/2550517980 conn(0x7faee006d750 msgr2=0x7faee006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:36.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.060+0000 7faee77ff700 1 -- 172.21.15.3:0/2550517980 shutdown_connections 2023-08-11T14:16:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.075+0000 7faee77ff700 1 -- 172.21.15.3:0/2550517980 wait complete. 2023-08-11T14:16:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.076+0000 7faee77ff700 1 Processor -- start 2023-08-11T14:16:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 -- start start 2023-08-11T14:16:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 0x7faee01182b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 0x7faee01187f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee010cc60 0x7faee0116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0119060 con 0x7faee010a290 2023-08-11T14:16:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0117130 con 0x7faee0107920 2023-08-11T14:16:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.087+0000 7faee77ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01172f0 con 0x7faee010cc60 2023-08-11T14:16:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.089+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 0x7faee01182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.089+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 0x7faee01182b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50922/0 (socket says 172.21.15.3:50922) 2023-08-11T14:16:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.089+0000 7faee559b700 1 -- 172.21.15.3:0/3812447797 learned_addr learned my addr 172.21.15.3:0/3812447797 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.095+0000 7faee4d9a700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 0x7faee01187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 -- 172.21.15.3:0/3812447797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee010cc60 msgr2=0x7faee0116bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:16:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee010cc60 0x7faee0116bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 -- 172.21.15.3:0/3812447797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 msgr2=0x7faee01182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 0x7faee01182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 -- 172.21.15.3:0/3812447797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8009d00 con 0x7faee010a290 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.096+0000 7faee4d9a700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 0x7faee01187f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faed000c970 tx=0x7faed000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.099+0000 7faede7fc700 1 -- 172.21.15.3:0/3812447797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0007b00 con 0x7faee010a290 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.100+0000 7faede7fc700 1 -- 172.21.15.3:0/3812447797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed000ce60 con 0x7faee010a290 2023-08-11T14:16:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.100+0000 7faede7fc700 1 -- 172.21.15.3:0/3812447797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed000e610 con 0x7faee010a290 2023-08-11T14:16:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.100+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee01175d0 con 0x7faee010a290 2023-08-11T14:16:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.100+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01b6620 con 0x7faee010a290 2023-08-11T14:16:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.108+0000 7faefb0f5b80 1 Processor -- start 2023-08-11T14:16:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 -- start start 2023-08-11T14:16:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda484360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 0x55aeda4851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 0x55aeda485c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda3dd5d0 con 0x55aeda485790 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda3daef0 con 0x55aeda483f40 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.117+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda3d67e0 con 0x55aeda4848a0 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.120+0000 7faee5d9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda484360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.122+0000 7faede7fc700 1 -- 172.21.15.3:0/3812447797 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7faed000f450 con 0x7faee010a290 2023-08-11T14:16:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.124+0000 7faede7fc700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 0x7faecc0a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.124+0000 7faede7fc700 1 -- 172.21.15.3:0/3812447797 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7faed00cb4b0 con 0x7faee010a290 2023-08-11T14:16:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.127+0000 7faee559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 0x55aeda4851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.127+0000 7faee559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 0x55aeda4851b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46826/0 (socket says 172.21.15.3:46826) 2023-08-11T14:16:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.127+0000 7faee559b700 1 -- 172.21.15.3:0/2268103655 learned_addr learned my addr 172.21.15.3:0/2268103655 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 0x55aeda485c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 -- 172.21.15.3:0/2268103655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 msgr2=0x55aeda4851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 0x55aeda4851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 -- 172.21.15.3:0/2268103655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 msgr2=0x55aeda484360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda484360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.130+0000 7faee4d9a700 1 -- 172.21.15.3:0/2268103655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeda2565d0 con 0x55aeda485790 2023-08-11T14:16:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.131+0000 7faee4d9a700 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 0x55aeda485c10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7faed0098b10 tx=0x7faed0098a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dd5c47263761a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.136+0000 7faee5d9c700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 0x7faecc0a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.137+0000 7faee5d9c700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 0x7faecc0a2560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faec8009fd0 tx=0x7faec8004750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.138+0000 7faed77fe700 1 -- 172.21.15.3:0/2268103655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed009c9b0 con 0x55aeda485790 2023-08-11T14:16:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.138+0000 7faed77fe700 1 -- 172.21.15.3:0/2268103655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed00a0990 con 0x55aeda485790 2023-08-11T14:16:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.138+0000 7faed77fe700 1 -- 172.21.15.3:0/2268103655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed00c8d40 con 0x55aeda485790 2023-08-11T14:16:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.154+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2268103655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 msgr2=0x55aeda485c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.154+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 0x55aeda485c10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7faed0098b10 tx=0x7faed0098a80 comp rx=0 tx=0).stop 2023-08-11T14:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.155+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2268103655 shutdown_connections 2023-08-11T14:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.155+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda485790 0x55aeda485c10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.155+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda484360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.155+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2268103655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda4848a0 0x55aeda4851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.155+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2268103655 >> 172.21.15.3:0/2268103655 conn(0x55aeda3b18f0 msgr2=0x55aeda480920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.168+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2268103655 shutdown_connections 2023-08-11T14:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.169+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2268103655 wait complete. 2023-08-11T14:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.169+0000 7faefb0f5b80 1 -- start start 2023-08-11T14:16:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.170+0000 7faefb0f5b80 1 Processor -- start 2023-08-11T14:16:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.170+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda483f40 0x55aeda48dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda4848a0 0x55aeda48e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda374a60 con 0x55aeda483f40 2023-08-11T14:16:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda374760 con 0x55aeda4848a0 2023-08-11T14:16:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda2588c0 con 0x55aeda485790 2023-08-11T14:16:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.171+0000 7faefb0f5b80 10 client.0 _ll_register_callbacks cb 0x55aeda377a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.176+0000 7faee559b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda483f40 0x55aeda48dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.176+0000 7faee5d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.176+0000 7faee5d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:46838/0 (socket says 172.21.15.3:46838) 2023-08-11T14:16:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.176+0000 7faee5d9c700 1 -- 172.21.15.3:0/2104646562 learned_addr learned my addr 172.21.15.3:0/2104646562 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.177+0000 7faee5d9c700 1 -- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda4848a0 msgr2=0x55aeda48e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.177+0000 7faee5d9c700 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda4848a0 0x55aeda48e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.177+0000 7faee5d9c700 1 -- 172.21.15.3:0/2104646562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda483f40 msgr2=0x55aeda48dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.177+0000 7faee5d9c700 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda483f40 0x55aeda48dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.177+0000 7faee5d9c700 1 -- 172.21.15.3:0/2104646562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01b6620 con 0x55aeda485790 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.178+0000 7faee5d9c700 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faec800f7c0 tx=0x7faec80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc0abb51195b8b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.178+0000 7faed4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:16:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.178+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8004250 con 0x55aeda485790 2023-08-11T14:16:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.179+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7faec80043b0 con 0x55aeda485790 2023-08-11T14:16:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.179+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeda4777e0 con 0x55aeda485790 2023-08-11T14:16:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.183+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7faec8004570 con 0x55aeda485790 2023-08-11T14:16:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.188+0000 7faefb0f5b80 10 client.26599 fetch_fsmap learned FSMap version 54 2023-08-11T14:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.188+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55aeda202d80 con 0x55aeda485790 2023-08-11T14:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aeda4777e0 con 0x55aeda485790 2023-08-11T14:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 54) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faec801e7f0 con 0x55aeda485790 2023-08-11T14:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 10 client.26599 fetch_fsmap finished waiting for FSMap version 54 2023-08-11T14:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 20 client.26599 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda499bc0 0x55aeda49bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 4 client.26599 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:16:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.194+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeda453480 con 0x55aeda499bc0 2023-08-11T14:16:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.211+0000 7faee559b700 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda499bc0 0x55aeda49bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.221+0000 7faee559b700 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda499bc0 0x55aeda49bfe0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a1a67cfaed6214 server_cookie=136b23b3703d81f5 in_seq=0 out_seq=0 2023-08-11T14:16:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.221+0000 7faed4ff9700 10 client.26599 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.223+0000 7faed4ff9700 1 -- 172.21.15.3:0/2104646562 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55aeda453480 con 0x55aeda499bc0 2023-08-11T14:16:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.224+0000 7faed4ff9700 10 client.26599 handle_command_reply: tid=0 2023-08-11T14:16:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.225+0000 7faefb0f5b80 1 client.26599 shutdown 2023-08-11T14:16:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 msgr2=0x55aeda495230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faec800f7c0 tx=0x7faec80047c0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 shutdown_connections 2023-08-11T14:16:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda495230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda483f40 0x55aeda48dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda499bc0 0x55aeda49bfe0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2104646562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda4848a0 0x55aeda48e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.238+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 >> 172.21.15.3:0/2104646562 conn(0x55aeda3b18f0 msgr2=0x55aeda47f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.245+0000 7faed4ff9700 0 client.26599 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.245+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 shutdown_connections 2023-08-11T14:16:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.250+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2104646562 wait complete. 2023-08-11T14:16:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.260+0000 7faefb0f5b80 20 client.26599 trim_cache size 0 max 16384 2023-08-11T14:16:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.290+0000 7faefb0f5b80 1 Processor -- start 2023-08-11T14:16:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.290+0000 7faefb0f5b80 1 -- start start 2023-08-11T14:16:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda488bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda479ed0 con 0x55aeda4848a0 2023-08-11T14:16:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda47fd50 con 0x55aeda483f40 2023-08-11T14:16:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.291+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda48e930 con 0x55aeda485790 2023-08-11T14:16:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.292+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.292+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50936/0 (socket says 172.21.15.3:50936) 2023-08-11T14:16:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.292+0000 7faee559b700 1 -- 172.21.15.3:0/2309550660 learned_addr learned my addr 172.21.15.3:0/2309550660 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.292+0000 7faee4d9a700 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda488bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 -- 172.21.15.3:0/2309550660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 msgr2=0x55aeda4951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 -- 172.21.15.3:0/2309550660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 msgr2=0x55aeda488bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda488bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 -- 172.21.15.3:0/2309550660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeda1ebc00 con 0x55aeda483f40 2023-08-11T14:16:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.293+0000 7faee559b700 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faed8009fd0 tx=0x7faed8007950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa32716822c2ee2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.305+0000 7faed7fff700 1 -- 172.21.15.3:0/2309550660 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800e040 con 0x55aeda483f40 2023-08-11T14:16:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.305+0000 7faed7fff700 1 -- 172.21.15.3:0/2309550660 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed800bab0 con 0x55aeda483f40 2023-08-11T14:16:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.305+0000 7faed7fff700 1 -- 172.21.15.3:0/2309550660 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8003a00 con 0x55aeda483f40 2023-08-11T14:16:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.307+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2309550660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 msgr2=0x55aeda4886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.307+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faed8009fd0 tx=0x7faed8007950 comp rx=0 tx=0).stop 2023-08-11T14:16:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.314+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2309550660 shutdown_connections 2023-08-11T14:16:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.314+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.314+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda4886b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.314+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/2309550660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda488bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.314+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2309550660 >> 172.21.15.3:0/2309550660 conn(0x55aeda3b18f0 msgr2=0x55aeda48c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.316+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2309550660 shutdown_connections 2023-08-11T14:16:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.316+0000 7faefb0f5b80 1 -- 172.21.15.3:0/2309550660 wait complete. 2023-08-11T14:16:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.316+0000 7faefb0f5b80 1 -- start start 2023-08-11T14:16:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 Processor -- start 2023-08-11T14:16:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda4934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda494a20 con 0x55aeda4848a0 2023-08-11T14:16:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda494b90 con 0x55aeda483f40 2023-08-11T14:16:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.317+0000 7faefb0f5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeda494d50 con 0x55aeda485790 2023-08-11T14:16:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.318+0000 7faefb0f5b80 10 client.0 _ll_register_callbacks cb 0x55aeda377a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.318+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.319+0000 7faee559b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50940/0 (socket says 172.21.15.3:50940) 2023-08-11T14:16:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.319+0000 7faee559b700 1 -- 172.21.15.3:0/613781724 learned_addr learned my addr 172.21.15.3:0/613781724 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.320+0000 7faee559b700 1 -- 172.21.15.3:0/613781724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 msgr2=0x55aeda4944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.320+0000 7faee559b700 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.320+0000 7faee559b700 1 -- 172.21.15.3:0/613781724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 msgr2=0x55aeda4934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.320+0000 7faee559b700 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda4934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.320+0000 7faee559b700 1 -- 172.21.15.3:0/613781724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee010dc60 con 0x55aeda483f40 2023-08-11T14:16:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.331+0000 7faee559b700 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faed8020c60 tx=0x7faed8009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d212058bf250ca server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.331+0000 7faed4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:16:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.332+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800e030 con 0x55aeda483f40 2023-08-11T14:16:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.332+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7faed8020370 con 0x55aeda483f40 2023-08-11T14:16:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.332+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faed00a0990 con 0x55aeda483f40 2023-08-11T14:16:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.335+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed8003a00 con 0x55aeda483f40 2023-08-11T14:16:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.335+0000 7faefb0f5b80 10 client.26594 fetch_fsmap learned FSMap version 54 2023-08-11T14:16:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.335+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55aeda2291f0 con 0x55aeda483f40 2023-08-11T14:16:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed00a0990 con 0x55aeda483f40 2023-08-11T14:16:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) v1 ==== 7898+0+0 (secure 0 0 0) 0x7faed800b0d0 con 0x55aeda483f40 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 10 client.26594 fetch_fsmap finished waiting for FSMap version 54 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 20 client.26594 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda4ee3b0 0x55aeda4f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 4 client.26594 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:16:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.337+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aeda3cf7c0 con 0x55aeda4ee3b0 2023-08-11T14:16:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.338+0000 7faee4d9a700 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda4ee3b0 0x55aeda4f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.346+0000 7faee4d9a700 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda4ee3b0 0x55aeda4f07d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfeb95d3a7dd384 server_cookie=c6266bba703eef27 in_seq=0 out_seq=0 2023-08-11T14:16:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:36.346+0000 7faed4ff9700 10 client.26594 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:37.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:36 smithi003 ceph-mon[97195]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:16:37.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:36 smithi187 ceph-mon[130973]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:16:37.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:36 smithi103 ceph-mon[130434]: pgmap v727: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.13k op/s 2023-08-11T14:16:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.495+0000 7faed4ff9700 1 -- 172.21.15.3:0/613781724 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aeda3cf7c0 con 0x55aeda4ee3b0 2023-08-11T14:16:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.495+0000 7faed4ff9700 10 client.26594 handle_command_reply: tid=0 2023-08-11T14:16:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.495+0000 7faefb0f5b80 1 client.26594 shutdown 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 msgr2=0x55aeda492f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faed8020c60 tx=0x7faed8009530 comp rx=0 tx=0).stop 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 shutdown_connections 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aeda485790 0x55aeda4944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55aeda4ee3b0 0x55aeda4f07d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeda483f40 0x55aeda492f90 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faed8020c60 tx=0x7faed8009530 comp rx=0 tx=0).stop 2023-08-11T14:16:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 --2- 172.21.15.3:0/613781724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55aeda4848a0 0x55aeda4934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 >> 172.21.15.3:0/613781724 conn(0x55aeda3b18f0 msgr2=0x55aeda4886b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.496+0000 7faed4ff9700 0 client.26594 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.497+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 shutdown_connections 2023-08-11T14:16:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.497+0000 7faefb0f5b80 1 -- 172.21.15.3:0/613781724 wait complete. 2023-08-11T14:16:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.503+0000 7faefb0f5b80 20 client.26594 trim_cache size 0 max 16384 2023-08-11T14:16:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:16:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:16:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:16:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:16:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.511+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 msgr2=0x7faecc0a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.511+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 0x7faecc0a2560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faec8009fd0 tx=0x7faec8004750 comp rx=0 tx=0).stop 2023-08-11T14:16:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.511+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 msgr2=0x7faee01187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.511+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 0x7faee01187f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faed000c970 tx=0x7faed000cc80 comp rx=0 tx=0).stop 2023-08-11T14:16:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 shutdown_connections 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee010cc60 0x7faee0116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7faecc0a00a0 0x7faecc0a2560 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faee0107920 0x7faee01182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 --2- 172.21.15.3:0/3812447797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee010a290 0x7faee01187f0 secure :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faed000c970 tx=0x7faed000cc80 comp rx=0 tx=0).stop 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 >> 172.21.15.3:0/3812447797 conn(0x7faee006d750 msgr2=0x7faee01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.512+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 shutdown_connections 2023-08-11T14:16:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:37.513+0000 7faee77ff700 1 -- 172.21.15.3:0/3812447797 wait complete. 2023-08-11T14:16:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 580.0 s, 641.5 tps, lat 157.393 ms stddev 399.210 2023-08-11T14:16:39.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:39 smithi103 ceph-mon[130434]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-08-11T14:16:39.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:39 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:39.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:39 smithi003 ceph-mon[97195]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-08-11T14:16:39.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:39 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:39.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:39 smithi187 ceph-mon[130973]: pgmap v728: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.21k op/s 2023-08-11T14:16:39.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:39 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-08-11T14:16:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:16:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:41 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-08-11T14:16:41.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:16:41.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:41 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.580 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: pgmap v729: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-08-11T14:16:41.581 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:16:41.581 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.581 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.582 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:41.582 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:41 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:42.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:16:42.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:16:42.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:16:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:16:42.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:42.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:16:42.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:16:42.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:16:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 585.0 s, 781.4 tps, lat 126.528 ms stddev 241.070 2023-08-11T14:16:43.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:43 smithi103 ceph-mon[130434]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 1.50k op/s 2023-08-11T14:16:43.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:43 smithi003 ceph-mon[97195]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 1.50k op/s 2023-08-11T14:16:43.552 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:43 smithi187 ceph-mon[130973]: pgmap v730: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 1.50k op/s 2023-08-11T14:16:44.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:44 smithi103 ceph-mon[130434]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.27k op/s 2023-08-11T14:16:45.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:44 smithi003 ceph-mon[97195]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.27k op/s 2023-08-11T14:16:45.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:44 smithi187 ceph-mon[130973]: pgmap v731: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 1.27k op/s 2023-08-11T14:16:46.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:46 smithi103 ceph-mon[130434]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:16:47.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:46 smithi003 ceph-mon[97195]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:16:47.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:46 smithi187 ceph-mon[130973]: pgmap v732: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:16:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 590.0 s, 777.8 tps, lat 129.635 ms stddev 304.967 2023-08-11T14:16:49.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:48 smithi003 ceph-mon[97195]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-08-11T14:16:49.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:48 smithi187 ceph-mon[130973]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-08-11T14:16:49.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:48 smithi103 ceph-mon[130434]: pgmap v733: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 1.42k op/s 2023-08-11T14:16:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.794+0000 7fcb35dff700 1 -- 172.21.15.3:0/1830256023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb3010d010 msgr2=0x7fcb3010f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.794+0000 7fcb35dff700 1 --2- 172.21.15.3:0/1830256023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb3010d010 0x7fcb3010f4d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b910 tx=0x7fcb2800bc20 comp rx=0 tx=0).stop 2023-08-11T14:16:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.801+0000 7fcb35dff700 1 -- 172.21.15.3:0/1830256023 shutdown_connections 2023-08-11T14:16:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.801+0000 7fcb35dff700 1 --2- 172.21.15.3:0/1830256023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb3010d010 0x7fcb3010f4d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.801+0000 7fcb35dff700 1 --2- 172.21.15.3:0/1830256023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb30107b90 0x7fcb30109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.801+0000 7fcb35dff700 1 --2- 172.21.15.3:0/1830256023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 0x7fcb3010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.801+0000 7fcb35dff700 1 -- 172.21.15.3:0/1830256023 >> 172.21.15.3:0/1830256023 conn(0x7fcb3006daf0 msgr2=0x7fcb3006ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.808+0000 7fcb35dff700 1 -- 172.21.15.3:0/1830256023 shutdown_connections 2023-08-11T14:16:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.812+0000 7fcb35dff700 1 -- 172.21.15.3:0/1830256023 wait complete. 2023-08-11T14:16:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.813+0000 7fcb35dff700 1 Processor -- start 2023-08-11T14:16:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 -- start start 2023-08-11T14:16:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb30107b90 0x7fcb301a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 0x7fcb301a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a6f90 con 0x7fcb3010d010 2023-08-11T14:16:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a7100 con 0x7fcb30107b90 2023-08-11T14:16:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.814+0000 7fcb35dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301a72c0 con 0x7fcb3010a550 2023-08-11T14:16:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.815+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.815+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34088/0 (socket says 172.21.15.3:34088) 2023-08-11T14:16:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.815+0000 7fcb2ffff700 1 -- 172.21.15.3:0/960149613 learned_addr learned my addr 172.21.15.3:0/960149613 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.815+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 0x7fcb301a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 -- 172.21.15.3:0/960149613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 msgr2=0x7fcb301a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 0x7fcb301a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 -- 172.21.15.3:0/960149613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb30107b90 msgr2=0x7fcb301a5570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:16:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb30107b90 0x7fcb301a5570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 -- 172.21.15.3:0/960149613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28009c40 con 0x7fcb3010d010 2023-08-11T14:16:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.816+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009040 tx=0x7fcb28003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.821+0000 7fcb2d7fa700 1 -- 172.21.15.3:0/960149613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800b400 con 0x7fcb3010d010 2023-08-11T14:16:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.821+0000 7fcb2d7fa700 1 -- 172.21.15.3:0/960149613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb28007c50 con 0x7fcb3010d010 2023-08-11T14:16:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.821+0000 7fcb2d7fa700 1 -- 172.21.15.3:0/960149613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28014440 con 0x7fcb3010d010 2023-08-11T14:16:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.821+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb301a7540 con 0x7fcb3010d010 2023-08-11T14:16:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.821+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301add60 con 0x7fcb3010d010 2023-08-11T14:16:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.827+0000 7fcb4970bb80 1 Processor -- start 2023-08-11T14:16:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.828+0000 7fcb4970bb80 1 -- start start 2023-08-11T14:16:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ade31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ad3b5d0 con 0x558d5ade1f40 2023-08-11T14:16:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ad38ef0 con 0x558d5ade3790 2023-08-11T14:16:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ad347e0 con 0x558d5ade28a0 2023-08-11T14:16:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:58182/0 (socket says 172.21.15.3:58182) 2023-08-11T14:16:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb26ffd700 1 -- 172.21.15.3:0/4122596961 learned_addr learned my addr 172.21.15.3:0/4122596961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.829+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ade31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 -- 172.21.15.3:0/4122596961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 msgr2=0x558d5ade31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ade31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 -- 172.21.15.3:0/4122596961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 msgr2=0x558d5ade2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 -- 172.21.15.3:0/4122596961 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5abb45d0 con 0x558d5ade3790 2023-08-11T14:16:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.830+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009d20 tx=0x7fcb1800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145b9902c1c71a67 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.831+0000 7fcb25ffb700 1 -- 172.21.15.3:0/4122596961 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18004230 con 0x558d5ade3790 2023-08-11T14:16:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.831+0000 7fcb25ffb700 1 -- 172.21.15.3:0/4122596961 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb18009e70 con 0x558d5ade3790 2023-08-11T14:16:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.831+0000 7fcb25ffb700 1 -- 172.21.15.3:0/4122596961 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18005560 con 0x558d5ade3790 2023-08-11T14:16:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.842+0000 7fcb4970bb80 1 -- 172.21.15.3:0/4122596961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 msgr2=0x558d5ade3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.842+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009d20 tx=0x7fcb1800f740 comp rx=0 tx=0).stop 2023-08-11T14:16:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.857+0000 7fcb4970bb80 1 -- 172.21.15.3:0/4122596961 shutdown_connections 2023-08-11T14:16:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.857+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade3c10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.857+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.857+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/4122596961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ade31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.857+0000 7fcb4970bb80 1 -- 172.21.15.3:0/4122596961 >> 172.21.15.3:0/4122596961 conn(0x558d5ad0f8f0 msgr2=0x558d5adde920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.858+0000 7fcb2d7fa700 1 -- 172.21.15.3:0/960149613 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcb28005230 con 0x7fcb3010d010 2023-08-11T14:16:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.860+0000 7fcb2d7fa700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 0x7fcb1c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.860+0000 7fcb2d7fa700 1 -- 172.21.15.3:0/960149613 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcb280cbd40 con 0x7fcb3010d010 2023-08-11T14:16:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.862+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 0x7fcb1c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.863+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 0x7fcb1c0a2610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x558d5adde2f0 tx=0x7fcb20009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.877+0000 7fcb4970bb80 1 -- 172.21.15.3:0/4122596961 shutdown_connections 2023-08-11T14:16:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.883+0000 7fcb4970bb80 1 -- 172.21.15.3:0/4122596961 wait complete. 2023-08-11T14:16:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.883+0000 7fcb4970bb80 1 -- start start 2023-08-11T14:16:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.884+0000 7fcb4970bb80 1 Processor -- start 2023-08-11T14:16:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5adfcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5acd2a60 con 0x558d5ade1f40 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5acd2760 con 0x558d5ade28a0 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5abb68c0 con 0x558d5ade3790 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb4970bb80 10 client.0 _ll_register_callbacks cb 0x558d5acd5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.885+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:58194/0 (socket says 172.21.15.3:58194) 2023-08-11T14:16:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2743351567 learned_addr learned my addr 172.21.15.3:0/2743351567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2743351567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 msgr2=0x558d5ae04210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:16:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae04210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2743351567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 msgr2=0x558d5adfcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5adfcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.886+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2743351567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb18009a90 con 0x558d5ade28a0 2023-08-11T14:16:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.887+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb18004820 tx=0x7fcb18004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6466a6488cc4d9b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.891+0000 7fcb0effd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:16:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.904+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18004de0 con 0x558d5ade28a0 2023-08-11T14:16:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.904+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcb18000e40 con 0x558d5ade28a0 2023-08-11T14:16:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.904+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d5add57e0 con 0x558d5ade28a0 2023-08-11T14:16:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.910+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb18026430 con 0x558d5ade28a0 2023-08-11T14:16:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.911+0000 7fcb4970bb80 10 client.26603 fetch_fsmap learned FSMap version 56 2023-08-11T14:16:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.911+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558d5ab60d80 con 0x558d5ade28a0 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d5add57e0 con 0x558d5ade28a0 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb1802d020 con 0x558d5ade28a0 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 10 client.26603 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 20 client.26603 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae08a70 0x558d5ae0ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 4 client.26603 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:16:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.920+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d5adb1480 con 0x558d5ae08a70 2023-08-11T14:16:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:50 smithi103 ceph-mon[130434]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:50.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:50 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.928+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae08a70 0x558d5ae0ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.929+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae08a70 0x558d5ae0ae90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9562361273bd16 server_cookie=b293a6a0acf51042 in_seq=0 out_seq=0 2023-08-11T14:16:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.930+0000 7fcb0effd700 10 client.26603 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.930+0000 7fcb0effd700 1 -- 172.21.15.3:0/2743351567 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558d5adb1480 con 0x558d5ae08a70 2023-08-11T14:16:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.930+0000 7fcb0effd700 10 client.26603 handle_command_reply: tid=0 2023-08-11T14:16:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.930+0000 7fcb4970bb80 1 client.26603 shutdown 2023-08-11T14:16:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.935+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 msgr2=0x558d5adfd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.935+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb18004820 tx=0x7fcb18004900 comp rx=0 tx=0).stop 2023-08-11T14:16:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 shutdown_connections 2023-08-11T14:16:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5adfcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae08a70 0x558d5ae0ae90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2743351567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade28a0 0x558d5adfd1d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.939+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 >> 172.21.15.3:0/2743351567 conn(0x558d5ad0f8f0 msgr2=0x558d5addd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.941+0000 7fcb0effd700 0 client.26603 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.945+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 shutdown_connections 2023-08-11T14:16:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.951+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2743351567 wait complete. 2023-08-11T14:16:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.953+0000 7fcb4970bb80 20 client.26603 trim_cache size 0 max 16384 2023-08-11T14:16:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.984+0000 7fcb4970bb80 1 Processor -- start 2023-08-11T14:16:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.986+0000 7fcb4970bb80 1 -- start start 2023-08-11T14:16:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ae041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ade9980 con 0x558d5ade1f40 2023-08-11T14:16:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ade2fa0 con 0x558d5ade3790 2023-08-11T14:16:50.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5adfdde0 con 0x558d5ade28a0 2023-08-11T14:16:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:58196/0 (socket says 172.21.15.3:58196) 2023-08-11T14:16:50.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.987+0000 7fcb2ffff700 1 -- 172.21.15.3:0/1406307585 learned_addr learned my addr 172.21.15.3:0/1406307585 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.991+0000 7fcb2ffff700 1 -- 172.21.15.3:0/1406307585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 msgr2=0x558d5ae041d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:16:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.991+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ae041d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.991+0000 7fcb2ffff700 1 -- 172.21.15.3:0/1406307585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 msgr2=0x558d5ade2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.991+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.991+0000 7fcb2ffff700 1 -- 172.21.15.3:0/1406307585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5ab49c00 con 0x558d5ade3790 2023-08-11T14:16:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.995+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb280a0cf0 tx=0x7fcb28006070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9d93be0829ad2c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.995+0000 7fcb267fc700 1 -- 172.21.15.3:0/1406307585 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28098fe0 con 0x558d5ade3790 2023-08-11T14:16:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.995+0000 7fcb267fc700 1 -- 172.21.15.3:0/1406307585 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb28098e00 con 0x558d5ade3790 2023-08-11T14:16:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.995+0000 7fcb267fc700 1 -- 172.21.15.3:0/1406307585 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb280a12b0 con 0x558d5ade3790 2023-08-11T14:16:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.996+0000 7fcb4970bb80 1 -- 172.21.15.3:0/1406307585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 msgr2=0x558d5ade66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.996+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb280a0cf0 tx=0x7fcb28006070 comp rx=0 tx=0).stop 2023-08-11T14:16:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.998+0000 7fcb4970bb80 1 -- 172.21.15.3:0/1406307585 shutdown_connections 2023-08-11T14:16:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.998+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade3790 0x558d5ade66e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.998+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade1f40 0x558d5ade2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.998+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/1406307585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade28a0 0x558d5ae041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:50.998+0000 7fcb4970bb80 1 -- 172.21.15.3:0/1406307585 >> 172.21.15.3:0/1406307585 conn(0x558d5ad0f8f0 msgr2=0x558d5ae0aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.000+0000 7fcb4970bb80 1 -- 172.21.15.3:0/1406307585 shutdown_connections 2023-08-11T14:16:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.003+0000 7fcb4970bb80 1 -- 172.21.15.3:0/1406307585 wait complete. 2023-08-11T14:16:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.004+0000 7fcb4970bb80 1 -- start start 2023-08-11T14:16:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.004+0000 7fcb4970bb80 1 Processor -- start 2023-08-11T14:16:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 0x558d5ae038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ae5b340 con 0x558d5ade28a0 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ae5b4b0 con 0x558d5ade1f40 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5ae5b670 con 0x558d5ade3790 2023-08-11T14:16:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.005+0000 7fcb4970bb80 10 client.0 _ll_register_callbacks cb 0x7fcb20006f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:16:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.007+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.007+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34122/0 (socket says 172.21.15.3:34122) 2023-08-11T14:16:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.007+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2247072964 learned_addr learned my addr 172.21.15.3:0/2247072964 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:16:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.007+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 0x558d5ae038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb2ffff700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae5ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2247072964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 msgr2=0x558d5ae5ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae5ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 msgr2=0x558d5ae038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:16:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 0x558d5ae038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 -- 172.21.15.3:0/2247072964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301add60 con 0x558d5ade28a0 2023-08-11T14:16:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 0x558d5ae038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:16:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.008+0000 7fcb26ffd700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb1801a6c0 tx=0x7fcb180055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498b9d61b3e5b4b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:16:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.009+0000 7fcb0effd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:16:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.009+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18024ef0 con 0x558d5ade28a0 2023-08-11T14:16:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.009+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcb18000e40 con 0x558d5ade28a0 2023-08-11T14:16:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.010+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb18009e70 con 0x558d5ade28a0 2023-08-11T14:16:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.016+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb1801aa30 con 0x558d5ade28a0 2023-08-11T14:16:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.017+0000 7fcb4970bb80 10 client.16437 fetch_fsmap learned FSMap version 56 2023-08-11T14:16:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.017+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558d5ab871f0 con 0x558d5ade28a0 2023-08-11T14:16:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb18009e70 con 0x558d5ade28a0 2023-08-11T14:16:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fcb1800f7b0 con 0x558d5ade28a0 2023-08-11T14:16:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 10 client.16437 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:16:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:16:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 20 client.16437 populate_metadata read hostname 'smithi003' 2023-08-11T14:16:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae5ceb0 0x558d5ae5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:16:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 4 client.16437 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:16:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.020+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d5ad2d7c0 con 0x558d5ae5ceb0 2023-08-11T14:16:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.026+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae5ceb0 0x558d5ae5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:16:51.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:50 smithi187 ceph-mon[130973]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:51.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:50 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.042+0000 7fcb2f7fe700 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae5ceb0 0x558d5ae5f2d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15199d9ab50e371a server_cookie=b1ba8415d18433db in_seq=0 out_seq=0 2023-08-11T14:16:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.043+0000 7fcb0effd700 10 client.16437 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:51.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:50 smithi003 ceph-mon[97195]: pgmap v734: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-08-11T14:16:51.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:50 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.363+0000 7fcb0effd700 1 -- 172.21.15.3:0/2247072964 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d5ad2d7c0 con 0x558d5ae5ceb0 2023-08-11T14:16:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.363+0000 7fcb0effd700 10 client.16437 handle_command_reply: tid=0 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.364+0000 7fcb4970bb80 1 client.16437 shutdown 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.364+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 msgr2=0x558d5ae03df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.364+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb1801a6c0 tx=0x7fcb180055f0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 shutdown_connections 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d5ade3790 0x558d5ae5ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558d5ae5ceb0 0x558d5ae5f2d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5ade1f40 0x558d5ae038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 --2- 172.21.15.3:0/2247072964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558d5ade28a0 0x558d5ae03df0 secure :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcb1801a6c0 tx=0x7fcb180055f0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 >> 172.21.15.3:0/2247072964 conn(0x558d5ad0f8f0 msgr2=0x558d5ade66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.365+0000 7fcb0effd700 0 client.16437 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.366+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 shutdown_connections 2023-08-11T14:16:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.367+0000 7fcb4970bb80 1 -- 172.21.15.3:0/2247072964 wait complete. 2023-08-11T14:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.376+0000 7fcb4970bb80 20 client.16437 trim_cache size 0 max 16384 2023-08-11T14:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:16:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:16:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:16:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:16:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.384+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 msgr2=0x7fcb1c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.384+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 0x7fcb1c0a2610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x558d5adde2f0 tx=0x7fcb20009110 comp rx=0 tx=0).stop 2023-08-11T14:16:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.384+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 msgr2=0x7fcb301a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:16:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.384+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009040 tx=0x7fcb28003e20 comp rx=0 tx=0).stop 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 shutdown_connections 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3010d010 0x7fcb301a6930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb1c0a0150 0x7fcb1c0a2610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb30107b90 0x7fcb301a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 --2- 172.21.15.3:0/960149613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb3010a550 0x7fcb301a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:16:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.389+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 >> 172.21.15.3:0/960149613 conn(0x7fcb3006daf0 msgr2=0x7fcb30108800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:16:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.395+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 shutdown_connections 2023-08-11T14:16:51.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:16:51.398+0000 7fcb35dff700 1 -- 172.21.15.3:0/960149613 wait complete. 2023-08-11T14:16:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:52 smithi103 ceph-mon[130434]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:16:53.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:52 smithi003 ceph-mon[97195]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:16:53.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:52 smithi187 ceph-mon[130973]: pgmap v735: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-08-11T14:16:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 595.0 s, 598.1 tps, lat 163.857 ms stddev 439.736 2023-08-11T14:16:55.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:54 smithi003 ceph-mon[97195]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:16:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:54 smithi187 ceph-mon[130973]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:16:55.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:54 smithi103 ceph-mon[130434]: pgmap v736: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.38k op/s 2023-08-11T14:16:57.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:56 smithi187 ceph-mon[130973]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:16:57.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:56 smithi103 ceph-mon[130434]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:16:57.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:56 smithi003 ceph-mon[97195]: pgmap v737: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 1.34k op/s 2023-08-11T14:16:58.084 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 600.0 s, 285.5 tps, lat 352.553 ms stddev 526.261 2023-08-11T14:16:59.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:16:58 smithi103 ceph-mon[130434]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:16:59.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:16:58 smithi003 ceph-mon[97195]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:16:59.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:16:58 smithi187 ceph-mon[130973]: pgmap v738: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 18 MiB/s wr, 1.39k op/s 2023-08-11T14:17:01.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:00 smithi003 ceph-mon[97195]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:17:01.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:00 smithi187 ceph-mon[130973]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:17:01.430 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:00 smithi103 ceph-mon[130434]: pgmap v739: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:17:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 605.0 s, 473.6 tps, lat 210.225 ms stddev 287.653 2023-08-11T14:17:03.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:03 smithi187 ceph-mon[130973]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:17:03.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:03 smithi003 ceph-mon[97195]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:17:03.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:03 smithi103 ceph-mon[130434]: pgmap v740: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:17:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.693+0000 7f6f523ff700 1 -- 172.21.15.3:0/3378044751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10cc60 msgr2=0x7f6f4c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.693+0000 7f6f523ff700 1 --2- 172.21.15.3:0/3378044751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10cc60 0x7f6f4c10d0e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009990 tx=0x7f6f40009ca0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.701+0000 7f6f523ff700 1 -- 172.21.15.3:0/3378044751 shutdown_connections 2023-08-11T14:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.701+0000 7f6f523ff700 1 --2- 172.21.15.3:0/3378044751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10cc60 0x7f6f4c10d0e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.701+0000 7f6f523ff700 1 --2- 172.21.15.3:0/3378044751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f4c107920 0x7f6f4c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.701+0000 7f6f523ff700 1 --2- 172.21.15.3:0/3378044751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4c10a290 0x7f6f4c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.701+0000 7f6f523ff700 1 -- 172.21.15.3:0/3378044751 >> 172.21.15.3:0/3378044751 conn(0x7f6f4c06d750 msgr2=0x7f6f4c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.715+0000 7f6f523ff700 1 -- 172.21.15.3:0/3378044751 shutdown_connections 2023-08-11T14:17:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.716+0000 7f6f523ff700 1 -- 172.21.15.3:0/3378044751 wait complete. 2023-08-11T14:17:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.717+0000 7f6f523ff700 1 Processor -- start 2023-08-11T14:17:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 -- start start 2023-08-11T14:17:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4c107920 0x7f6f4c118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f4c10cc60 0x7f6f4c116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c1172b0 con 0x7f6f4c107920 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c117420 con 0x7f6f4c10cc60 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.727+0000 7f6f523ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c1175e0 con 0x7f6f4c10a290 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:55840/0 (socket says 172.21.15.3:55840) 2023-08-11T14:17:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/337020173 learned_addr learned my addr 172.21.15.3:0/337020173 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/337020173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f4c10cc60 msgr2=0x7f6f4c116d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:17:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f4c10cc60 0x7f6f4c116d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4c107920 msgr2=0x7f6f4c118650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.729+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4c107920 0x7f6f4c118650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.730+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/337020173 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400095f0 con 0x7f6f4c10a290 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.735+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9a0 tx=0x7f6f3c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.737+0000 7f6f497fa700 1 -- 172.21.15.3:0/337020173 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00cbd0 con 0x7f6f4c10a290 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.737+0000 7f6f497fa700 1 -- 172.21.15.3:0/337020173 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3c00cd30 con 0x7f6f4c10a290 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.737+0000 7f6f497fa700 1 -- 172.21.15.3:0/337020173 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c018710 con 0x7f6f4c10a290 2023-08-11T14:17:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.737+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c1178c0 con 0x7f6f4c10a290 2023-08-11T14:17:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.737+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c1b6600 con 0x7f6f4c10a290 2023-08-11T14:17:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.747+0000 7f6f497fa700 1 -- 172.21.15.3:0/337020173 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6f3c018870 con 0x7f6f4c10a290 2023-08-11T14:17:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.749+0000 7f6f497fa700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 0x7f6f340a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.749+0000 7f6f497fa700 1 -- 172.21.15.3:0/337020173 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6f3c014070 con 0x7f6f4c10a290 2023-08-11T14:17:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.760+0000 7f6f65c23b80 1 Processor -- start 2023-08-11T14:17:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.762+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 0x7f6f340a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.770+0000 7f6f65c23b80 1 -- start start 2023-08-11T14:17:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5551f40 0x5648a5552360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a55528a0 0x5648a55531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a54ab5d0 con 0x5648a5551f40 2023-08-11T14:17:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a54a8ef0 con 0x5648a5553790 2023-08-11T14:17:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.771+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a54a47e0 con 0x5648a55528a0 2023-08-11T14:17:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.772+0000 7f6f5099c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.772+0000 7f6f5099c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45480/0 (socket says 172.21.15.3:45480) 2023-08-11T14:17:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.772+0000 7f6f5099c700 1 -- 172.21.15.3:0/3294049885 learned_addr learned my addr 172.21.15.3:0/3294049885 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 -- 172.21.15.3:0/3294049885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a55528a0 msgr2=0x5648a55531b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a55528a0 0x5648a55531b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 -- 172.21.15.3:0/3294049885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5551f40 msgr2=0x5648a5552360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5551f40 0x5648a5552360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 -- 172.21.15.3:0/3294049885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a53245d0 con 0x5648a5553790 2023-08-11T14:17:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.773+0000 7f6f5099c700 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005da0 tx=0x7f6f40005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70839dfbdfe61f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.780+0000 7f6f327fc700 1 -- 172.21.15.3:0/3294049885 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4001d070 con 0x5648a5553790 2023-08-11T14:17:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.780+0000 7f6f327fc700 1 -- 172.21.15.3:0/3294049885 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4000be00 con 0x5648a5553790 2023-08-11T14:17:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.780+0000 7f6f327fc700 1 -- 172.21.15.3:0/3294049885 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f40021410 con 0x5648a5553790 2023-08-11T14:17:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.782+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 0x7f6f340a2610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c119690 tx=0x7f6f44009230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3294049885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 msgr2=0x5648a5553c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005da0 tx=0x7f6f40005dd0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3294049885 shutdown_connections 2023-08-11T14:17:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a5553c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5551f40 0x5648a5552360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3294049885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a55528a0 0x5648a55531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.793+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3294049885 >> 172.21.15.3:0/3294049885 conn(0x5648a547f8f0 msgr2=0x5648a554e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.818+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3294049885 shutdown_connections 2023-08-11T14:17:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.829+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3294049885 wait complete. 2023-08-11T14:17:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.829+0000 7f6f65c23b80 1 -- start start 2023-08-11T14:17:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.830+0000 7f6f65c23b80 1 Processor -- start 2023-08-11T14:17:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.830+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a555c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a55631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a5442a60 con 0x5648a55528a0 2023-08-11T14:17:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a5442760 con 0x5648a5553790 2023-08-11T14:17:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a53268c0 con 0x5648a5551f40 2023-08-11T14:17:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f65c23b80 10 client.0 _ll_register_callbacks cb 0x5648a5445a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:55850/0 (socket says 172.21.15.3:55850) 2023-08-11T14:17:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.831+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2417817326 learned_addr learned my addr 172.21.15.3:0/2417817326 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 msgr2=0x5648a55631a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:17:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a55631a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2417817326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 msgr2=0x5648a555c160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:17:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a555c160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2417817326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c10dc60 con 0x5648a5551f40 2023-08-11T14:17:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.832+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f44018aa0 tx=0x7f6f44004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8cfb05db34bdfef server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.834+0000 7f6f2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:17:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.834+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44009c20 con 0x5648a5551f40 2023-08-11T14:17:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.834+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6f44009d80 con 0x5648a5551f40 2023-08-11T14:17:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.836+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a55457e0 con 0x5648a5551f40 2023-08-11T14:17:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.848+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f44019420 con 0x5648a5551f40 2023-08-11T14:17:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.857+0000 7f6f65c23b80 10 client.26617 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.857+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5648a52d0d80 con 0x5648a5551f40 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648a55457e0 con 0x5648a5551f40 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6f440165b0 con 0x5648a5551f40 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 10 client.26617 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 10 client.26617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 20 client.26617 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a5567c20 0x5648a556a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 4 client.26617 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:17:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.868+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a5521480 con 0x5648a5567c20 2023-08-11T14:17:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.869+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a5567c20 0x5648a556a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.877+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a5567c20 0x5648a556a040 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf409463f4a213b server_cookie=24cfc994f8382d21 in_seq=0 out_seq=0 2023-08-11T14:17:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.877+0000 7f6f2b7fe700 10 client.26617 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.880+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/2417817326 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5648a5521480 con 0x5648a5567c20 2023-08-11T14:17:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.880+0000 7f6f2b7fe700 10 client.26617 handle_command_reply: tid=0 2023-08-11T14:17:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.880+0000 7f6f65c23b80 1 client.26617 shutdown 2023-08-11T14:17:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 msgr2=0x5648a555bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f44018aa0 tx=0x7f6f44004b10 comp rx=0 tx=0).stop 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 shutdown_connections 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a5567c20 0x5648a556a040 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a55631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a555bc20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2417817326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a555c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.883+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 >> 172.21.15.3:0/2417817326 conn(0x5648a547f8f0 msgr2=0x5648a554d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.884+0000 7f6f2b7fe700 0 client.26617 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.884+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 shutdown_connections 2023-08-11T14:17:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.884+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2417817326 wait complete. 2023-08-11T14:17:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.885+0000 7f6f65c23b80 20 client.26617 trim_cache size 0 max 16384 2023-08-11T14:17:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.923+0000 7f6f65c23b80 1 Processor -- start 2023-08-11T14:17:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.923+0000 7f6f65c23b80 1 -- start start 2023-08-11T14:17:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.923+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:55866/0 (socket says 172.21.15.3:55866) 2023-08-11T14:17:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a554e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a554e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a5551a00 con 0x5648a55528a0 2023-08-11T14:17:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a55457b0 con 0x5648a5553790 2023-08-11T14:17:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.924+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a5564e40 con 0x5648a5551f40 2023-08-11T14:17:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2908010099 learned_addr learned my addr 172.21.15.3:0/2908010099 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f5099c700 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a554e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2908010099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 msgr2=0x5648a554e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a554e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2908010099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 msgr2=0x5648a554e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a554e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.925+0000 7f6f4bfff700 1 -- 172.21.15.3:0/2908010099 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a52b9c00 con 0x5648a5551f40 2023-08-11T14:17:04.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.928+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000f80 tx=0x7f6f440070f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc64afb6bb08e275 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.933+0000 7f6f32ffd700 1 -- 172.21.15.3:0/2908010099 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44039070 con 0x5648a5551f40 2023-08-11T14:17:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.933+0000 7f6f32ffd700 1 -- 172.21.15.3:0/2908010099 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4400c6f0 con 0x5648a5551f40 2023-08-11T14:17:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.933+0000 7f6f32ffd700 1 -- 172.21.15.3:0/2908010099 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44009a20 con 0x5648a5551f40 2023-08-11T14:17:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2908010099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 msgr2=0x5648a5561120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000f80 tx=0x7f6f440070f0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2908010099 shutdown_connections 2023-08-11T14:17:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a5553790 0x5648a554e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a5561120 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/2908010099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a55528a0 0x5648a554e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.934+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2908010099 >> 172.21.15.3:0/2908010099 conn(0x5648a547f8f0 msgr2=0x5648a5556750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.935+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2908010099 shutdown_connections 2023-08-11T14:17:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.935+0000 7f6f65c23b80 1 -- 172.21.15.3:0/2908010099 wait complete. 2023-08-11T14:17:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.935+0000 7f6f65c23b80 1 -- start start 2023-08-11T14:17:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.936+0000 7f6f65c23b80 1 Processor -- start 2023-08-11T14:17:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a55b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5553790 0x5648a55ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a55baca0 con 0x5648a5553790 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a55bae10 con 0x5648a55528a0 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a55bafd0 con 0x5648a5551f40 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f65c23b80 10 client.0 _ll_register_callbacks cb 0x7f6f440183b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:04.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.937+0000 7f6f4b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.938+0000 7f6f4bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a55b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.938+0000 7f6f4b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45508/0 (socket says 172.21.15.3:45508) 2023-08-11T14:17:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.938+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/3044513069 learned_addr learned my addr 172.21.15.3:0/3044513069 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:04.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.941+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/3044513069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 msgr2=0x5648a55b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.941+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a55b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.941+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/3044513069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5553790 msgr2=0x5648a55ba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.941+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5553790 0x5648a55ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.941+0000 7f6f4b7fe700 1 -- 172.21.15.3:0/3044513069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648a52d0d80 con 0x5648a55528a0 2023-08-11T14:17:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.943+0000 7f6f4b7fe700 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0c9ed0 tx=0x7f6f3c099890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708ea469875f3eb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.949+0000 7f6f2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:17:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.949+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c09dc50 con 0x5648a55528a0 2023-08-11T14:17:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.949+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6f3c0a1c30 con 0x5648a55528a0 2023-08-11T14:17:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.949+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f4000be00 con 0x5648a55528a0 2023-08-11T14:17:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.950+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f3c0b9070 con 0x5648a55528a0 2023-08-11T14:17:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.950+0000 7f6f65c23b80 10 client.26615 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.950+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5648a52f71f0 con 0x5648a55528a0 2023-08-11T14:17:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.951+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4000be00 con 0x5648a55528a0 2023-08-11T14:17:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.951+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6f3c0a17a0 con 0x5648a55528a0 2023-08-11T14:17:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 10 client.26615 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 10 client.26615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 20 client.26615 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a55bca90 0x5648a55beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 4 client.26615 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:17:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648a549d7c0 con 0x5648a55bca90 2023-08-11T14:17:04.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.952+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a55bca90 0x5648a55beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.953+0000 7f6f4bfff700 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a55bca90 0x5648a55beeb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf936b7330cd696 server_cookie=859d521a207f5e1d in_seq=0 out_seq=0 2023-08-11T14:17:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:04.953+0000 7f6f2b7fe700 10 client.26615 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.173+0000 7f6f2b7fe700 1 -- 172.21.15.3:0/3044513069 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648a549d7c0 con 0x5648a55bca90 2023-08-11T14:17:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.173+0000 7f6f2b7fe700 10 client.26615 handle_command_reply: tid=0 2023-08-11T14:17:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.173+0000 7f6f65c23b80 1 client.26615 shutdown 2023-08-11T14:17:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 msgr2=0x5648a55b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0c9ed0 tx=0x7f6f3c099890 comp rx=0 tx=0).stop 2023-08-11T14:17:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 shutdown_connections 2023-08-11T14:17:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5648a5553790 0x5648a55ba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5551f40 0x5648a55b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5648a55bca90 0x5648a55beeb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 --2- 172.21.15.3:0/3044513069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5648a55528a0 0x5648a55b7fb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 >> 172.21.15.3:0/3044513069 conn(0x5648a547f8f0 msgr2=0x5648a554e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f2b7fe700 0 client.26615 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 shutdown_connections 2023-08-11T14:17:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.174+0000 7f6f65c23b80 1 -- 172.21.15.3:0/3044513069 wait complete. 2023-08-11T14:17:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.175+0000 7f6f65c23b80 20 client.26615 trim_cache size 0 max 16384 2023-08-11T14:17:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:17:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:17:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:17:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:17:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 msgr2=0x7f6f340a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 0x7f6f340a2610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c119690 tx=0x7f6f44009230 comp rx=0 tx=0).stop 2023-08-11T14:17:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 msgr2=0x7f6f4c118b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9a0 tx=0x7f6f3c00ed60 comp rx=0 tx=0).stop 2023-08-11T14:17:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 shutdown_connections 2023-08-11T14:17:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6f340a0150 0x7f6f340a2610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f4c10cc60 0x7f6f4c116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f4c107920 0x7f6f4c118650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 --2- 172.21.15.3:0/337020173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f4c10a290 0x7f6f4c118b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.176+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 >> 172.21.15.3:0/337020173 conn(0x7f6f4c06d750 msgr2=0x7f6f4c10acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.177+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 shutdown_connections 2023-08-11T14:17:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:05.177+0000 7f6f523ff700 1 -- 172.21.15.3:0/337020173 wait complete. 2023-08-11T14:17:05.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:05 smithi103 ceph-mon[130434]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:17:05.548 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:05 smithi187 ceph-mon[130973]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:17:05.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:05 smithi003 ceph-mon[97195]: pgmap v741: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:17:06.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:06 smithi103 ceph-mon[130434]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:06.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:06 smithi003 ceph-mon[97195]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:06.554 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:06 smithi187 ceph-mon[130973]: pgmap v742: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 610.0 s, 707.2 tps, lat 142.723 ms stddev 215.610 2023-08-11T14:17:08.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:08 smithi003 ceph-mon[97195]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:08.552 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:08 smithi187 ceph-mon[130973]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:08.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:08 smithi103 ceph-mon[130434]: pgmap v743: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 920 op/s 2023-08-11T14:17:11.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:10 smithi003 ceph-mon[97195]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-08-11T14:17:11.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:10 smithi187 ceph-mon[130973]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-08-11T14:17:11.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:10 smithi103 ceph-mon[130434]: pgmap v744: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 799 op/s 2023-08-11T14:17:13.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:12 smithi003 ceph-mon[97195]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 846 op/s 2023-08-11T14:17:13.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:12 smithi187 ceph-mon[130973]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 846 op/s 2023-08-11T14:17:13.091 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 615.0 s, 741.1 tps, lat 136.452 ms stddev 238.366 2023-08-11T14:17:13.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:12 smithi103 ceph-mon[130434]: pgmap v745: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 846 op/s 2023-08-11T14:17:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:14 smithi103 ceph-mon[130434]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 979 op/s 2023-08-11T14:17:15.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:14 smithi187 ceph-mon[130973]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 979 op/s 2023-08-11T14:17:15.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:14 smithi003 ceph-mon[97195]: pgmap v746: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 979 op/s 2023-08-11T14:17:17.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:16 smithi103 ceph-mon[130434]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 940 op/s 2023-08-11T14:17:17.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:16 smithi003 ceph-mon[97195]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 940 op/s 2023-08-11T14:17:17.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:16 smithi187 ceph-mon[130973]: pgmap v747: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 940 op/s 2023-08-11T14:17:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 620.0 s, 628.9 tps, lat 155.347 ms stddev 407.665 2023-08-11T14:17:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.740+0000 7fe53c3ff700 1 -- 172.21.15.3:0/2168650945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410cc60 msgr2=0x7fe53410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.740+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/2168650945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410cc60 0x7fe53410d0e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe524009990 tx=0x7fe524009ca0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.746+0000 7fe53c3ff700 1 -- 172.21.15.3:0/2168650945 shutdown_connections 2023-08-11T14:17:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.746+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/2168650945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410cc60 0x7fe53410d0e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.746+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/2168650945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 0x7fe534109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.746+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/2168650945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410a290 0x7fe53410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.746+0000 7fe53c3ff700 1 -- 172.21.15.3:0/2168650945 >> 172.21.15.3:0/2168650945 conn(0x7fe53406d750 msgr2=0x7fe53406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.767+0000 7fe53c3ff700 1 -- 172.21.15.3:0/2168650945 shutdown_connections 2023-08-11T14:17:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.770+0000 7fe53c3ff700 1 -- 172.21.15.3:0/2168650945 wait complete. 2023-08-11T14:17:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.772+0000 7fe53c3ff700 1 Processor -- start 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.772+0000 7fe53c3ff700 1 -- start start 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 0x7fe534118630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410a290 0x7fe534118b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534117290 con 0x7fe53410a290 2023-08-11T14:17:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534117400 con 0x7fe534107920 2023-08-11T14:17:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.773+0000 7fe53c3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5341175c0 con 0x7fe53410cc60 2023-08-11T14:17:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.775+0000 7fe53a99c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.776+0000 7fe53a99c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36478/0 (socket says 172.21.15.3:36478) 2023-08-11T14:17:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.776+0000 7fe53a99c700 1 -- 172.21.15.3:0/1532508203 learned_addr learned my addr 172.21.15.3:0/1532508203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.783+0000 7fe53a19b700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 0x7fe534118630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.795+0000 7fe53a99c700 1 -- 172.21.15.3:0/1532508203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 msgr2=0x7fe534118630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.795+0000 7fe53a99c700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 0x7fe534118630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.795+0000 7fe53a99c700 1 -- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410a290 msgr2=0x7fe534118b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.795+0000 7fe53a99c700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410a290 0x7fe534118b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.795+0000 7fe53a99c700 1 -- 172.21.15.3:0/1532508203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240095f0 con 0x7fe53410cc60 2023-08-11T14:17:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.818+0000 7fe53a99c700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe524005f50 tx=0x7fe524003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.820+0000 7fe52b7fe700 1 -- 172.21.15.3:0/1532508203 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52401d070 con 0x7fe53410cc60 2023-08-11T14:17:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.820+0000 7fe52b7fe700 1 -- 172.21.15.3:0/1532508203 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe52400f460 con 0x7fe53410cc60 2023-08-11T14:17:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.820+0000 7fe52b7fe700 1 -- 172.21.15.3:0/1532508203 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524017650 con 0x7fe53410cc60 2023-08-11T14:17:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.820+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe534117840 con 0x7fe53410cc60 2023-08-11T14:17:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.820+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5341b6660 con 0x7fe53410cc60 2023-08-11T14:17:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.832+0000 7fe54fc7db80 1 Processor -- start 2023-08-11T14:17:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 -- start start 2023-08-11T14:17:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d28a0 0x560c108d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 0x560c108d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1082b5d0 con 0x560c108d1f40 2023-08-11T14:17:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c10828ef0 con 0x560c108d28a0 2023-08-11T14:17:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.833+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108247e0 con 0x560c108d3790 2023-08-11T14:17:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.836+0000 7fe53999a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 0x560c108d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.836+0000 7fe53999a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 0x560c108d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36488/0 (socket says 172.21.15.3:36488) 2023-08-11T14:17:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.836+0000 7fe53999a700 1 -- 172.21.15.3:0/909336155 learned_addr learned my addr 172.21.15.3:0/909336155 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.836+0000 7fe53a19b700 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d28a0 0x560c108d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.837+0000 7fe52b7fe700 1 -- 172.21.15.3:0/1532508203 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe5240052d0 con 0x7fe53410cc60 2023-08-11T14:17:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.837+0000 7fe53999a700 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.838+0000 7fe53999a700 1 -- 172.21.15.3:0/909336155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 msgr2=0x560c108d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.838+0000 7fe53999a700 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 0x560c108d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.838+0000 7fe53999a700 1 -- 172.21.15.3:0/909336155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d28a0 msgr2=0x560c108d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.838+0000 7fe53999a700 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d28a0 0x560c108d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.838+0000 7fe53999a700 1 -- 172.21.15.3:0/909336155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c106a45d0 con 0x560c108d1f40 2023-08-11T14:17:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.839+0000 7fe53999a700 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108d2360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0137d0 tx=0x7fe52c006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e0d8a1e3ed052 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.839+0000 7fe52b7fe700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 0x7fe5200a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.840+0000 7fe52b7fe700 1 -- 172.21.15.3:0/1532508203 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe52409a530 con 0x7fe53410cc60 2023-08-11T14:17:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.840+0000 7fe528bfe700 1 -- 172.21.15.3:0/909336155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00c070 con 0x560c108d1f40 2023-08-11T14:17:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.840+0000 7fe528bfe700 1 -- 172.21.15.3:0/909336155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe52c003c00 con 0x560c108d1f40 2023-08-11T14:17:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.840+0000 7fe528bfe700 1 -- 172.21.15.3:0/909336155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004290 con 0x560c108d1f40 2023-08-11T14:17:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.841+0000 7fe54fc7db80 1 -- 172.21.15.3:0/909336155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 msgr2=0x560c108d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.841+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108d2360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0137d0 tx=0x7fe52c006d70 comp rx=0 tx=0).stop 2023-08-11T14:17:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.847+0000 7fe53a19b700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 0x7fe5200a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.849+0000 7fe53a19b700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 0x7fe5200a2590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe530006fd0 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.851+0000 7fe54fc7db80 1 -- 172.21.15.3:0/909336155 shutdown_connections 2023-08-11T14:17:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.851+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d3790 0x560c108d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.851+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108d2360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.851+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/909336155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d28a0 0x560c108d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.851+0000 7fe54fc7db80 1 -- 172.21.15.3:0/909336155 >> 172.21.15.3:0/909336155 conn(0x560c107ff8f0 msgr2=0x560c108ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.870+0000 7fe54fc7db80 1 -- 172.21.15.3:0/909336155 shutdown_connections 2023-08-11T14:17:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.879+0000 7fe54fc7db80 1 -- 172.21.15.3:0/909336155 wait complete. 2023-08-11T14:17:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.879+0000 7fe54fc7db80 1 -- start start 2023-08-11T14:17:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 Processor -- start 2023-08-11T14:17:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c107c2a60 con 0x560c108d1f40 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c107c2760 con 0x560c108d3790 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.880+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c106a68c0 con 0x560c108d28a0 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.881+0000 7fe54fc7db80 10 client.0 _ll_register_callbacks cb 0x560c107c5a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.881+0000 7fe53999a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.887+0000 7fe53a19b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.887+0000 7fe53a19b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36026/0 (socket says 172.21.15.3:36026) 2023-08-11T14:17:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.887+0000 7fe53a19b700 1 -- 172.21.15.3:0/2347479703 learned_addr learned my addr 172.21.15.3:0/2347479703 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.892+0000 7fe53a99c700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.894+0000 7fe53a19b700 1 -- 172.21.15.3:0/2347479703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 msgr2=0x560c108dc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.894+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108dc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.894+0000 7fe53a19b700 1 -- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 msgr2=0x560c108e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.894+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.894+0000 7fe53a19b700 1 -- 172.21.15.3:0/2347479703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c0134b0 con 0x560c108d1f40 2023-08-11T14:17:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.904+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b2b0 tx=0x7fe530004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4ae03fc866f8cb server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.908+0000 7fe51dffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:17:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.919+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530020640 con 0x560c108d1f40 2023-08-11T14:17:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.919+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe530014e10 con 0x560c108d1f40 2023-08-11T14:17:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.919+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c108c57e0 con 0x560c108d1f40 2023-08-11T14:17:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.930+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5300207a0 con 0x560c108d1f40 2023-08-11T14:17:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.930+0000 7fe54fc7db80 10 client.16449 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.930+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x560c10650d80 con 0x560c108d1f40 2023-08-11T14:17:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c108c57e0 con 0x560c108d1f40 2023-08-11T14:17:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe53002aa90 con 0x560c108d1f40 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 10 client.16449 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 20 client.16449 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c108e7cd0 0x560c108ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 4 client.16449 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:17:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.939+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c108a1480 con 0x560c108e7cd0 2023-08-11T14:17:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.945+0000 7fe53999a700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c108e7cd0 0x560c108ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.982+0000 7fe53999a700 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c108e7cd0 0x560c108ea0f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0f59e9617538b4 server_cookie=246bc2e22be0a3fc in_seq=0 out_seq=0 2023-08-11T14:17:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.985+0000 7fe51dffb700 10 client.16449 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.996+0000 7fe51dffb700 1 -- 172.21.15.3:0/2347479703 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560c108a1480 con 0x560c108e7cd0 2023-08-11T14:17:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.996+0000 7fe51dffb700 10 client.16449 handle_command_reply: tid=0 2023-08-11T14:17:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.996+0000 7fe54fc7db80 1 client.16449 shutdown 2023-08-11T14:17:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.997+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 msgr2=0x560c108dbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.997+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b2b0 tx=0x7fe530004750 comp rx=0 tx=0).stop 2023-08-11T14:17:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 shutdown_connections 2023-08-11T14:17:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c108e7cd0 0x560c108ea0f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d1f40 0x560c108dbcc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2347479703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108dc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 >> 172.21.15.3:0/2347479703 conn(0x560c107ff8f0 msgr2=0x560c108cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:18.998+0000 7fe51dffb700 0 client.16449 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.009+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 shutdown_connections 2023-08-11T14:17:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.012+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2347479703 wait complete. 2023-08-11T14:17:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.013+0000 7fe54fc7db80 20 client.16449 trim_cache size 0 max 16384 2023-08-11T14:17:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.043+0000 7fe54fc7db80 1 Processor -- start 2023-08-11T14:17:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.043+0000 7fe54fc7db80 1 -- start start 2023-08-11T14:17:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d3790 0x560c108cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108d9960 con 0x560c108d3790 2023-08-11T14:17:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108d7250 con 0x560c108d1f40 2023-08-11T14:17:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.044+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108e4eb0 con 0x560c108d28a0 2023-08-11T14:17:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.052+0000 7fe53a19b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.052+0000 7fe53a19b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44216/0 (socket says 172.21.15.3:44216) 2023-08-11T14:17:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.052+0000 7fe53a19b700 1 -- 172.21.15.3:0/3345343647 learned_addr learned my addr 172.21.15.3:0/3345343647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.055+0000 7fe53a99c700 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d3790 0x560c108cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.058+0000 7fe53999a700 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.064+0000 7fe53a19b700 1 -- 172.21.15.3:0/3345343647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 msgr2=0x560c108e31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.064+0000 7fe53a19b700 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108e31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.064+0000 7fe53a19b700 1 -- 172.21.15.3:0/3345343647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d3790 msgr2=0x560c108cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.064+0000 7fe53a19b700 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d3790 0x560c108cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.064+0000 7fe53a19b700 1 -- 172.21.15.3:0/3345343647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c10639c00 con 0x560c108d1f40 2023-08-11T14:17:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.087+0000 7fe53a19b700 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe53000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3370c79514bba822 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.099+0000 7fe5293ff700 1 -- 172.21.15.3:0/3345343647 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530038070 con 0x560c108d1f40 2023-08-11T14:17:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.099+0000 7fe5293ff700 1 -- 172.21.15.3:0/3345343647 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe530019070 con 0x560c108d1f40 2023-08-11T14:17:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.099+0000 7fe5293ff700 1 -- 172.21.15.3:0/3345343647 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530015e70 con 0x560c108d1f40 2023-08-11T14:17:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.100+0000 7fe54fc7db80 1 -- 172.21.15.3:0/3345343647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 msgr2=0x560c108d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.100+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe53000b880 comp rx=0 tx=0).stop 2023-08-11T14:17:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.108+0000 7fe54fc7db80 1 -- 172.21.15.3:0/3345343647 shutdown_connections 2023-08-11T14:17:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.108+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d3790 0x560c108cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.108+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d1f40 0x560c108d2320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.108+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/3345343647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d28a0 0x560c108e31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.108+0000 7fe54fc7db80 1 -- 172.21.15.3:0/3345343647 >> 172.21.15.3:0/3345343647 conn(0x560c107ff8f0 msgr2=0x560c108d66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.126+0000 7fe54fc7db80 1 -- 172.21.15.3:0/3345343647 shutdown_connections 2023-08-11T14:17:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.130+0000 7fe54fc7db80 1 -- 172.21.15.3:0/3345343647 wait complete. 2023-08-11T14:17:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.130+0000 7fe54fc7db80 1 -- start start 2023-08-11T14:17:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.130+0000 7fe54fc7db80 1 Processor -- start 2023-08-11T14:17:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d1f40 0x560c108e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d28a0 0x560c108e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108e29b0 con 0x560c108d28a0 2023-08-11T14:17:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108e2b20 con 0x560c108d3790 2023-08-11T14:17:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c108e2ce0 con 0x560c108d1f40 2023-08-11T14:17:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.131+0000 7fe54fc7db80 10 client.0 _ll_register_callbacks cb 0x7fe53000fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.132+0000 7fe53a99c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.132+0000 7fe53a99c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44220/0 (socket says 172.21.15.3:44220) 2023-08-11T14:17:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.132+0000 7fe53a99c700 1 -- 172.21.15.3:0/2051502408 learned_addr learned my addr 172.21.15.3:0/2051502408 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.140+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d1f40 0x560c108e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.151+0000 7fe53a99c700 1 -- 172.21.15.3:0/2051502408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d1f40 msgr2=0x560c108e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.151+0000 7fe53a99c700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d1f40 0x560c108e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.151+0000 7fe53a99c700 1 -- 172.21.15.3:0/2051502408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d28a0 msgr2=0x560c108e1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.151+0000 7fe53a99c700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d28a0 0x560c108e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.151+0000 7fe53a99c700 1 -- 172.21.15.3:0/2051502408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c10650d80 con 0x560c108d3790 2023-08-11T14:17:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.170+0000 7fe53a99c700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe524099b30 tx=0x7fe524099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28dce76eda7fbdd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.170+0000 7fe51dffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:17:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.177+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5240b1070 con 0x560c108d3790 2023-08-11T14:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.177+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe52409ae40 con 0x560c108d3790 2023-08-11T14:17:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.177+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe52c003c00 con 0x560c108d3790 2023-08-11T14:17:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.180+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52409a090 con 0x560c108d3790 2023-08-11T14:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.180+0000 7fe54fc7db80 10 client.26627 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.180+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x560c106771f0 con 0x560c108d3790 2023-08-11T14:17:19.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:18 smithi103 ceph-mon[130434]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:17:19.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.186+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe52c003c00 con 0x560c108d3790 2023-08-11T14:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.186+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe524099430 con 0x560c108d3790 2023-08-11T14:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 10 client.26627 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 20 client.26627 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c1093c5e0 0x560c1093ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 4 client.26627 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:17:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.187+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c1081d7c0 con 0x560c1093c5e0 2023-08-11T14:17:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.193+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c1093c5e0 0x560c1093ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.206+0000 7fe53a19b700 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c1093c5e0 0x560c1093ea00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91041eb186e8141a server_cookie=5212cbd1d7bcbb7d in_seq=0 out_seq=0 2023-08-11T14:17:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:19.206+0000 7fe51dffb700 10 client.26627 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:19.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:18 smithi003 ceph-mon[97195]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:17:19.300 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:18 smithi187 ceph-mon[130973]: pgmap v748: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:17:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.504+0000 7fe51dffb700 1 -- 172.21.15.3:0/2051502408 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c1081d7c0 con 0x560c1093c5e0 2023-08-11T14:17:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.504+0000 7fe51dffb700 10 client.26627 handle_command_reply: tid=0 2023-08-11T14:17:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.504+0000 7fe54fc7db80 1 client.26627 shutdown 2023-08-11T14:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 msgr2=0x560c108e2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe524099b30 tx=0x7fe524099eb0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 shutdown_connections 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x560c1093c5e0 0x560c1093ea00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c108d3790 0x560c108e2470 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560c108d1f40 0x560c108e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 --2- 172.21.15.3:0/2051502408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c108d28a0 0x560c108e1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 >> 172.21.15.3:0/2051502408 conn(0x560c107ff8f0 msgr2=0x560c108cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.505+0000 7fe51dffb700 0 client.26627 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.508+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 shutdown_connections 2023-08-11T14:17:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.515+0000 7fe54fc7db80 1 -- 172.21.15.3:0/2051502408 wait complete. 2023-08-11T14:17:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.516+0000 7fe54fc7db80 20 client.26627 trim_cache size 0 max 16384 2023-08-11T14:17:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:17:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:17:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:17:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:17:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.524+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 msgr2=0x7fe5200a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.524+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 0x7fe5200a2590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe530006fd0 tx=0x7fe530008040 comp rx=0 tx=0).stop 2023-08-11T14:17:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.524+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 msgr2=0x7fe534116d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:20.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.524+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe524005f50 tx=0x7fe524003d80 comp rx=0 tx=0).stop 2023-08-11T14:17:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 shutdown_connections 2023-08-11T14:17:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53410cc60 0x7fe534116d50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe5200a00d0 0x7fe5200a2590 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534107920 0x7fe534118630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 --2- 172.21.15.3:0/1532508203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe53410a290 0x7fe534118b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.526+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 >> 172.21.15.3:0/1532508203 conn(0x7fe53406d750 msgr2=0x7fe534108350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.528+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 shutdown_connections 2023-08-11T14:17:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:20.528+0000 7fe53c3ff700 1 -- 172.21.15.3:0/1532508203 wait complete. 2023-08-11T14:17:21.303 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:20 smithi187 ceph-mon[130973]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:17:21.305 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:20 smithi003 ceph-mon[97195]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:17:21.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:20 smithi103 ceph-mon[130434]: pgmap v749: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:17:22.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:21 smithi003 ceph-mon[97195]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:17:22.301 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:21 smithi187 ceph-mon[130973]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:17:22.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:21 smithi103 ceph-mon[130434]: pgmap v750: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:17:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 625.0 s, 400.6 tps, lat 251.957 ms stddev 454.243 2023-08-11T14:17:25.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:24 smithi003 ceph-mon[97195]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:17:25.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:24 smithi187 ceph-mon[130973]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:17:25.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:24 smithi103 ceph-mon[130434]: pgmap v751: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 17 MiB/s wr, 1.29k op/s 2023-08-11T14:17:26.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:25 smithi003 ceph-mon[97195]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:17:26.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:25 smithi187 ceph-mon[130973]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:17:26.183 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:25 smithi103 ceph-mon[130434]: pgmap v752: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:17:27.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:27 smithi003 ceph-mon[97195]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-08-11T14:17:27.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:27 smithi187 ceph-mon[130973]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-08-11T14:17:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:27 smithi103 ceph-mon[130434]: pgmap v753: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.13k op/s 2023-08-11T14:17:28.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 630.0 s, 1171.7 tps, lat 85.363 ms stddev 135.598 2023-08-11T14:17:30.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:29 smithi187 ceph-mon[130973]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 935 op/s 2023-08-11T14:17:30.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:29 smithi003 ceph-mon[97195]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 935 op/s 2023-08-11T14:17:30.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:29 smithi103 ceph-mon[130434]: pgmap v754: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 935 op/s 2023-08-11T14:17:31.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:31 smithi187 ceph-mon[130973]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:17:31.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:31 smithi003 ceph-mon[97195]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:17:31.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:31 smithi103 ceph-mon[130434]: pgmap v755: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-08-11T14:17:33.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 635.0 s, 611.5 tps, lat 163.649 ms stddev 392.621 2023-08-11T14:17:33.680 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:33 smithi103 ceph-mon[130434]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:17:33.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:33 smithi187 ceph-mon[130973]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:17:33.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:33 smithi003 ceph-mon[97195]: pgmap v756: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.42k op/s 2023-08-11T14:17:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 -- 172.21.15.3:0/3232962787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 msgr2=0x7f55e010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 --2- 172.21.15.3:0/3232962787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e010d0e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f55d801c0d0 tx=0x7f55d801c3e0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 -- 172.21.15.3:0/3232962787 shutdown_connections 2023-08-11T14:17:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 --2- 172.21.15.3:0/3232962787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e010d0e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 --2- 172.21.15.3:0/3232962787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e0107920 0x7f55e0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 --2- 172.21.15.3:0/3232962787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e010a290 0x7f55e010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.016+0000 7f55e79ff700 1 -- 172.21.15.3:0/3232962787 >> 172.21.15.3:0/3232962787 conn(0x7f55e006d750 msgr2=0x7f55e006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.030+0000 7f55e79ff700 1 -- 172.21.15.3:0/3232962787 shutdown_connections 2023-08-11T14:17:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.041+0000 7f55e79ff700 1 -- 172.21.15.3:0/3232962787 wait complete. 2023-08-11T14:17:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.042+0000 7f55e79ff700 1 Processor -- start 2023-08-11T14:17:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.042+0000 7f55e79ff700 1 -- start start 2023-08-11T14:17:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e0107920 0x7f55e019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e010a290 0x7f55e019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019e820 con 0x7f55e010cc60 2023-08-11T14:17:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019e990 con 0x7f55e010a290 2023-08-11T14:17:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.043+0000 7f55e79ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019eb50 con 0x7f55e0107920 2023-08-11T14:17:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.046+0000 7f55e5f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.046+0000 7f55e5f9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39188/0 (socket says 172.21.15.3:39188) 2023-08-11T14:17:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.046+0000 7f55e5f9c700 1 -- 172.21.15.3:0/104951003 learned_addr learned my addr 172.21.15.3:0/104951003 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.049+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e010a290 0x7f55e019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.050+0000 7f55e5f9c700 1 -- 172.21.15.3:0/104951003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e0107920 msgr2=0x7f55e019ce30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:17:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.050+0000 7f55e5f9c700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e0107920 0x7f55e019ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.050+0000 7f55e5f9c700 1 -- 172.21.15.3:0/104951003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e010a290 msgr2=0x7f55e019d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.050+0000 7f55e5f9c700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e010a290 0x7f55e019d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.050+0000 7f55e5f9c700 1 -- 172.21.15.3:0/104951003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d801ad10 con 0x7f55e010cc60 2023-08-11T14:17:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.051+0000 7f55e5f9c700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55d8018f30 tx=0x7f55d8018f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.051+0000 7f55d67fc700 1 -- 172.21.15.3:0/104951003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d801f030 con 0x7f55e010cc60 2023-08-11T14:17:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.051+0000 7f55d67fc700 1 -- 172.21.15.3:0/104951003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d8005a40 con 0x7f55e010cc60 2023-08-11T14:17:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.051+0000 7f55d67fc700 1 -- 172.21.15.3:0/104951003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80233f0 con 0x7f55e010cc60 2023-08-11T14:17:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.052+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e019edd0 con 0x7f55e010cc60 2023-08-11T14:17:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.052+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e01a5540 con 0x7f55e010cc60 2023-08-11T14:17:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.058+0000 7f55fb35db80 1 Processor -- start 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 -- start start 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x56336587a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336587bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633657d35d0 con 0x56336587a8a0 2023-08-11T14:17:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633657d0ef0 con 0x563365879f40 2023-08-11T14:17:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.059+0000 7f55fb35db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633657cc7e0 con 0x56336587b790 2023-08-11T14:17:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.060+0000 7f55e579b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.060+0000 7f55e579b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39202/0 (socket says 172.21.15.3:39202) 2023-08-11T14:17:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.060+0000 7f55e579b700 1 -- 172.21.15.3:0/659299294 learned_addr learned my addr 172.21.15.3:0/659299294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.061+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x56336587a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 -- 172.21.15.3:0/659299294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 msgr2=0x56336587bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336587bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 -- 172.21.15.3:0/659299294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 msgr2=0x56336587a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x56336587a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 -- 172.21.15.3:0/659299294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56336564c5d0 con 0x56336587a8a0 2023-08-11T14:17:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.062+0000 7f55e579b700 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f55d00097d0 tx=0x7f55d0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665f7624710e854e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.066+0000 7f55d67fc700 1 -- 172.21.15.3:0/104951003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f55d8005580 con 0x7f55e010cc60 2023-08-11T14:17:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.068+0000 7f55d67fc700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 0x7f55cc0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.068+0000 7f55d67fc700 1 -- 172.21.15.3:0/104951003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f55d80dd650 con 0x7f55e010cc60 2023-08-11T14:17:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.075+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 0x7f55cc0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.083+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 0x7f55cc0a2610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56336587af70 tx=0x7f55dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.086+0000 7f55cb7fe700 1 -- 172.21.15.3:0/659299294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0011830 con 0x56336587a8a0 2023-08-11T14:17:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.086+0000 7f55cb7fe700 1 -- 172.21.15.3:0/659299294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d0011e50 con 0x56336587a8a0 2023-08-11T14:17:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.086+0000 7f55cb7fe700 1 -- 172.21.15.3:0/659299294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000f2e0 con 0x56336587a8a0 2023-08-11T14:17:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.094+0000 7f55fb35db80 1 -- 172.21.15.3:0/659299294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 msgr2=0x56336587b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.094+0000 7f55fb35db80 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f55d00097d0 tx=0x7f55d0009ae0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.103+0000 7f55fb35db80 1 -- 172.21.15.3:0/659299294 shutdown_connections 2023-08-11T14:17:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.103+0000 7f55fb35db80 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336587bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.103+0000 7f55fb35db80 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x56336587a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.103+0000 7f55fb35db80 1 --2- 172.21.15.3:0/659299294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x56336587b1b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.103+0000 7f55fb35db80 1 -- 172.21.15.3:0/659299294 >> 172.21.15.3:0/659299294 conn(0x5633657a78f0 msgr2=0x563365876920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.104+0000 7f55fb35db80 1 -- 172.21.15.3:0/659299294 shutdown_connections 2023-08-11T14:17:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.104+0000 7f55fb35db80 1 -- 172.21.15.3:0/659299294 wait complete. 2023-08-11T14:17:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.105+0000 7f55fb35db80 1 -- start start 2023-08-11T14:17:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.105+0000 7f55fb35db80 1 Processor -- start 2023-08-11T14:17:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336588b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336576aa60 con 0x56336587a8a0 2023-08-11T14:17:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336576a760 con 0x563365879f40 2023-08-11T14:17:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336564e8c0 con 0x56336587b790 2023-08-11T14:17:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.106+0000 7f55fb35db80 10 client.0 _ll_register_callbacks cb 0x56336576da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.107+0000 7f55e579b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.107+0000 7f55e579b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49988/0 (socket says 172.21.15.3:49988) 2023-08-11T14:17:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.107+0000 7f55e579b700 1 -- 172.21.15.3:0/282926905 learned_addr learned my addr 172.21.15.3:0/282926905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.107+0000 7f55e5f9c700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336588b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.111+0000 7f55e579b700 1 -- 172.21.15.3:0/282926905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 msgr2=0x56336588b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.111+0000 7f55e579b700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336588b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.111+0000 7f55e579b700 1 -- 172.21.15.3:0/282926905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 msgr2=0x5633658841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.111+0000 7f55e579b700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.111+0000 7f55e579b700 1 -- 172.21.15.3:0/282926905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d0009400 con 0x563365879f40 2023-08-11T14:17:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.120+0000 7f55e579b700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55d00105b0 tx=0x7f55d0010690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e485e1bd913af5d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.127+0000 7f55c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:17:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.132+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0011630 con 0x563365879f40 2023-08-11T14:17:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.132+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f55d0011790 con 0x563365879f40 2023-08-11T14:17:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.133+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56336586d7e0 con 0x563365879f40 2023-08-11T14:17:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.144+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d0011950 con 0x563365879f40 2023-08-11T14:17:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.144+0000 7f55fb35db80 10 client.26633 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.144+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5633655f8d80 con 0x563365879f40 2023-08-11T14:17:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.150+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56336586d7e0 con 0x563365879f40 2023-08-11T14:17:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.150+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f55d0022030 con 0x563365879f40 2023-08-11T14:17:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 10 client.26633 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 10 client.26633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 20 client.26633 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56336588fa80 0x563365891ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 4 client.26633 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.152+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563365849480 con 0x56336588fa80 2023-08-11T14:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.154+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56336588fa80 0x563365891ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.173+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56336588fa80 0x563365891ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4180ff6df36040 server_cookie=9c8deac8602ce396 in_seq=0 out_seq=0 2023-08-11T14:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.173+0000 7f55c8ff9700 10 client.26633 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.178+0000 7f55c8ff9700 1 -- 172.21.15.3:0/282926905 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563365849480 con 0x56336588fa80 2023-08-11T14:17:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.178+0000 7f55c8ff9700 10 client.26633 handle_command_reply: tid=0 2023-08-11T14:17:34.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.184+0000 7f55fb35db80 1 client.26633 shutdown 2023-08-11T14:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.192+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 msgr2=0x563365883cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.192+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f55d00105b0 tx=0x7f55d0010690 comp rx=0 tx=0).stop 2023-08-11T14:17:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 shutdown_connections 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x56336588b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x56336588fa80 0x563365891ea0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x563365883cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 --2- 172.21.15.3:0/282926905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.204+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 >> 172.21.15.3:0/282926905 conn(0x5633657a78f0 msgr2=0x563365875710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.205+0000 7f55c8ff9700 0 client.26633 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.223+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 shutdown_connections 2023-08-11T14:17:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.239+0000 7f55fb35db80 1 -- 172.21.15.3:0/282926905 wait complete. 2023-08-11T14:17:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.242+0000 7f55fb35db80 20 client.26633 trim_cache size 0 max 16384 2023-08-11T14:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 Processor -- start 2023-08-11T14:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 -- start start 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 0x56336587a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x563365889140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 0x563365875f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563365879b60 con 0x56336587a8a0 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56336587afa0 con 0x56336587b790 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.276+0000 7f55fb35db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633658819a0 con 0x563365879f40 2023-08-11T14:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.277+0000 7f55e5f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 0x563365875f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.277+0000 7f55e579b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 0x56336587a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.280+0000 7f55e5f9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 0x563365875f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49994/0 (socket says 172.21.15.3:49994) 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.280+0000 7f55e5f9c700 1 -- 172.21.15.3:0/1452851340 learned_addr learned my addr 172.21.15.3:0/1452851340 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.281+0000 7f55e579b700 1 -- 172.21.15.3:0/1452851340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 msgr2=0x563365875f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.281+0000 7f55e579b700 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 0x563365875f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.281+0000 7f55e579b700 1 -- 172.21.15.3:0/1452851340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 msgr2=0x563365889140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.281+0000 7f55e579b700 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x563365889140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.281+0000 7f55e579b700 1 -- 172.21.15.3:0/1452851340 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633655e1c00 con 0x563365879f40 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.283+0000 7f55e579b700 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 0x56336587a320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55d0007fd0 tx=0x7f55d0009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f513ebfc099dc48 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.286+0000 7f55cbfff700 1 -- 172.21.15.3:0/1452851340 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00109c0 con 0x563365879f40 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.286+0000 7f55cbfff700 1 -- 172.21.15.3:0/1452851340 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d001e070 con 0x563365879f40 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.286+0000 7f55cbfff700 1 -- 172.21.15.3:0/1452851340 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0011920 con 0x563365879f40 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.288+0000 7f55fb35db80 1 -- 172.21.15.3:0/1452851340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 msgr2=0x56336587a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.288+0000 7f55fb35db80 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 0x56336587a320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55d0007fd0 tx=0x7f55d0009f60 comp rx=0 tx=0).stop 2023-08-11T14:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.307+0000 7f55fb35db80 1 -- 172.21.15.3:0/1452851340 shutdown_connections 2023-08-11T14:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.307+0000 7f55fb35db80 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56336587b790 0x563365875f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.307+0000 7f55fb35db80 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563365879f40 0x56336587a320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.307+0000 7f55fb35db80 1 --2- 172.21.15.3:0/1452851340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x563365889140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.307+0000 7f55fb35db80 1 -- 172.21.15.3:0/1452851340 >> 172.21.15.3:0/1452851340 conn(0x5633657a78f0 msgr2=0x56336587e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.315+0000 7f55fb35db80 1 -- 172.21.15.3:0/1452851340 shutdown_connections 2023-08-11T14:17:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.319+0000 7f55fb35db80 1 -- 172.21.15.3:0/1452851340 wait complete. 2023-08-11T14:17:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.320+0000 7f55fb35db80 1 -- start start 2023-08-11T14:17:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.320+0000 7f55fb35db80 1 Processor -- start 2023-08-11T14:17:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x5633658dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x5633658e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633658e26f0 con 0x56336587a8a0 2023-08-11T14:17:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633658e2860 con 0x563365879f40 2023-08-11T14:17:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633658e2a20 con 0x56336587b790 2023-08-11T14:17:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.321+0000 7f55fb35db80 10 client.0 _ll_register_callbacks cb 0x56336576da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.327+0000 7f55e5f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x5633658e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.327+0000 7f55e4f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39238/0 (socket says 172.21.15.3:39238) 2023-08-11T14:17:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 -- 172.21.15.3:0/3426030569 learned_addr learned my addr 172.21.15.3:0/3426030569 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 -- 172.21.15.3:0/3426030569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 msgr2=0x5633658e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x5633658e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 -- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 msgr2=0x5633658dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x5633658dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.328+0000 7f55e4f9a700 1 -- 172.21.15.3:0/3426030569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633655f8d80 con 0x56336587a8a0 2023-08-11T14:17:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.329+0000 7f55e4f9a700 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00f6c0 tx=0x7f55dc024e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746ba2ea3b7cd6f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.329+0000 7f55c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:17:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.335+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc024360 con 0x56336587a8a0 2023-08-11T14:17:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.336+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f55dc01a040 con 0x56336587a8a0 2023-08-11T14:17:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.338+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55d0011e50 con 0x56336587a8a0 2023-08-11T14:17:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.348+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55dc00fec0 con 0x56336587a8a0 2023-08-11T14:17:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.350+0000 7f55fb35db80 10 client.16464 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.350+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56336561f1f0 con 0x56336587a8a0 2023-08-11T14:17:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.351+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d0011e50 con 0x56336587a8a0 2023-08-11T14:17:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.351+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f55dc021450 con 0x56336587a8a0 2023-08-11T14:17:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 10 client.16464 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 20 client.16464 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5633658e4560 0x5633658e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 4 client.16464 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.360+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633657c57c0 con 0x5633658e4560 2023-08-11T14:17:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.361+0000 7f55e579b700 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5633658e4560 0x5633658e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.375+0000 7f55e579b700 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5633658e4560 0x5633658e6980 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d335faa5ae7786 server_cookie=9c2c2d4ebff6f193 in_seq=0 out_seq=0 2023-08-11T14:17:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:34.375+0000 7f55c8ff9700 10 client.16464 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.069+0000 7f55c8ff9700 1 -- 172.21.15.3:0/3426030569 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633657c57c0 con 0x5633658e4560 2023-08-11T14:17:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.069+0000 7f55c8ff9700 10 client.16464 handle_command_reply: tid=0 2023-08-11T14:17:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.069+0000 7f55fb35db80 1 client.16464 shutdown 2023-08-11T14:17:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.070+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 msgr2=0x5633658dfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.070+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00f6c0 tx=0x7f55dc024e00 comp rx=0 tx=0).stop 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 shutdown_connections 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56336587b790 0x5633658e21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5633658e4560 0x5633658e6980 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563365879f40 0x5633658dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 --2- 172.21.15.3:0/3426030569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56336587a8a0 0x5633658dfc60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 >> 172.21.15.3:0/3426030569 conn(0x5633657a78f0 msgr2=0x563365875f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55c8ff9700 0 client.16464 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 shutdown_connections 2023-08-11T14:17:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.071+0000 7f55fb35db80 1 -- 172.21.15.3:0/3426030569 wait complete. 2023-08-11T14:17:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.082+0000 7f55fb35db80 20 client.16464 trim_cache size 0 max 16384 2023-08-11T14:17:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:17:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:17:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:17:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:17:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.099+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 msgr2=0x7f55cc0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.099+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 0x7f55cc0a2610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x56336587af70 tx=0x7f55dc008040 comp rx=0 tx=0).stop 2023-08-11T14:17:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.099+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 msgr2=0x7f55e019e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.099+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55d8018f30 tx=0x7f55d8018f60 comp rx=0 tx=0).stop 2023-08-11T14:17:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 shutdown_connections 2023-08-11T14:17:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55e010cc60 0x7f55e019e110 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e0107920 0x7f55e019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f55cc0a0150 0x7f55cc0a2610 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 --2- 172.21.15.3:0/104951003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e010a290 0x7f55e019d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.109+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 >> 172.21.15.3:0/104951003 conn(0x7f55e006d750 msgr2=0x7f55e010aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.118+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 shutdown_connections 2023-08-11T14:17:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:35.118+0000 7f55e79ff700 1 -- 172.21.15.3:0/104951003 wait complete. 2023-08-11T14:17:35.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:35 smithi103 ceph-mon[130434]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-08-11T14:17:35.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:35 smithi003 ceph-mon[97195]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-08-11T14:17:35.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:35 smithi187 ceph-mon[130973]: pgmap v757: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-08-11T14:17:37.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:37 smithi103 ceph-mon[130434]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-08-11T14:17:37.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:37 smithi003 ceph-mon[97195]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-08-11T14:17:37.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:37 smithi187 ceph-mon[130973]: pgmap v758: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.38k op/s 2023-08-11T14:17:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 640.0 s, 754.0 tps, lat 132.135 ms stddev 300.437 2023-08-11T14:17:40.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:39 smithi103 ceph-mon[130434]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:40.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:39 smithi003 ceph-mon[97195]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:40.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:39 smithi187 ceph-mon[130973]: pgmap v759: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:41.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:41 smithi103 ceph-mon[130434]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:17:41.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:41 smithi003 ceph-mon[97195]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:17:41.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:41 smithi187 ceph-mon[130973]: pgmap v760: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 19 MiB/s wr, 1.47k op/s 2023-08-11T14:17:42.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:42 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:17:42.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:42 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:17:42.800 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:42 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:17:43.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 645.0 s, 463.1 tps, lat 204.108 ms stddev 477.242 2023-08-11T14:17:43.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:43 smithi003 ceph-mon[97195]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-08-11T14:17:43.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:43 smithi187 ceph-mon[130973]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-08-11T14:17:43.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:43 smithi103 ceph-mon[130434]: pgmap v761: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 19 MiB/s wr, 1.54k op/s 2023-08-11T14:17:45.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:45 smithi103 ceph-mon[130434]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-08-11T14:17:45.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:45 smithi003 ceph-mon[97195]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-08-11T14:17:45.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:45 smithi187 ceph-mon[130973]: pgmap v762: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-08-11T14:17:46.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:17:46.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:17:46.684 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:17:46.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:17:46.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:17:46.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:17:46.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:17:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:17:46.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:17:47.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:47 smithi103 ceph-mon[130434]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:17:47.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:47 smithi187 ceph-mon[130973]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:17:47.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:47 smithi003 ceph-mon[97195]: pgmap v763: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-08-11T14:17:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 650.0 s, 798.4 tps, lat 125.474 ms stddev 302.304 2023-08-11T14:17:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.443+0000 7f3f8efff700 1 -- 172.21.15.3:0/2728865761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f8810cc60 msgr2=0x7f3f8810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.443+0000 7f3f8efff700 1 --2- 172.21.15.3:0/2728865761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f8810cc60 0x7f3f8810d0e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009f90 tx=0x7f3f8000b330 comp rx=0 tx=0).stop 2023-08-11T14:17:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.451+0000 7f3f8efff700 1 -- 172.21.15.3:0/2728865761 shutdown_connections 2023-08-11T14:17:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.451+0000 7f3f8efff700 1 --2- 172.21.15.3:0/2728865761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f8810cc60 0x7f3f8810d0e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.451+0000 7f3f8efff700 1 --2- 172.21.15.3:0/2728865761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f88107920 0x7f3f88109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.451+0000 7f3f8efff700 1 --2- 172.21.15.3:0/2728865761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810a290 0x7f3f8810c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.451+0000 7f3f8efff700 1 -- 172.21.15.3:0/2728865761 >> 172.21.15.3:0/2728865761 conn(0x7f3f8806d750 msgr2=0x7f3f8806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.473+0000 7f3f8efff700 1 -- 172.21.15.3:0/2728865761 shutdown_connections 2023-08-11T14:17:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.486+0000 7f3f8efff700 1 -- 172.21.15.3:0/2728865761 wait complete. 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 Processor -- start 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 -- start start 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 0x7f3f881a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 0x7f3f881a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810cc60 0x7f3f881a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.488+0000 7f3f8efff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a70c0 con 0x7f3f88107920 2023-08-11T14:17:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.489+0000 7f3f8efff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a7230 con 0x7f3f8810a290 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.489+0000 7f3f8efff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881a73f0 con 0x7f3f8810cc60 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.495+0000 7f3f8d59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810cc60 0x7f3f881a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.495+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 0x7f3f881a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.495+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 0x7f3f881a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42940/0 (socket says 172.21.15.3:42940) 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.495+0000 7f3f87fff700 1 -- 172.21.15.3:0/209990203 learned_addr learned my addr 172.21.15.3:0/209990203 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.495+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 0x7f3f881a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.496+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/209990203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810cc60 msgr2=0x7f3f881a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.496+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810cc60 0x7f3f881a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.496+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/209990203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 msgr2=0x7f3f881a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.496+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 0x7f3f881a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.496+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/209990203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009c40 con 0x7f3f88107920 2023-08-11T14:17:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.497+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 0x7f3f881a5630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b450 tx=0x7f3f7800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.504+0000 7f3f85ffb700 1 -- 172.21.15.3:0/209990203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78011830 con 0x7f3f88107920 2023-08-11T14:17:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.504+0000 7f3f85ffb700 1 -- 172.21.15.3:0/209990203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f78011e50 con 0x7f3f88107920 2023-08-11T14:17:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.504+0000 7f3f85ffb700 1 -- 172.21.15.3:0/209990203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800f2e0 con 0x7f3f88107920 2023-08-11T14:17:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.507+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881a76d0 con 0x7f3f88107920 2023-08-11T14:17:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.508+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88078de0 con 0x7f3f88107920 2023-08-11T14:17:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.528+0000 7f3fa289eb80 1 Processor -- start 2023-08-11T14:17:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.532+0000 7f3f85ffb700 1 -- 172.21.15.3:0/209990203 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f3f780105a0 con 0x7f3f88107920 2023-08-11T14:17:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.534+0000 7f3f85ffb700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 0x7f3f700a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.534+0000 7f3f85ffb700 1 -- 172.21.15.3:0/209990203 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3f780caf10 con 0x7f3f88107920 2023-08-11T14:17:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 -- start start 2023-08-11T14:17:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 0x558e0c742360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 0x558e0c7431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c743790 0x558e0c743c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c69b5d0 con 0x558e0c741f40 2023-08-11T14:17:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c698ef0 con 0x558e0c743790 2023-08-11T14:17:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.537+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c6947e0 con 0x558e0c7428a0 2023-08-11T14:17:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.538+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 0x558e0c7431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.538+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 0x558e0c7431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33560/0 (socket says 172.21.15.3:33560) 2023-08-11T14:17:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.538+0000 7f3f87fff700 1 -- 172.21.15.3:0/672205605 learned_addr learned my addr 172.21.15.3:0/672205605 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.539+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 0x558e0c742360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/672205605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 msgr2=0x558e0c7431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 0x558e0c7431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/672205605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c743790 msgr2=0x558e0c743c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c743790 0x558e0c743c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 -- 172.21.15.3:0/672205605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0c5145d0 con 0x558e0c741f40 2023-08-11T14:17:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.540+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 0x558e0c742360 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f780c8690 tx=0x7f3f78004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f398e021a4fcce9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.541+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 0x7f3f700a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.542+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 0x7f3f700a2610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x558e0c742f70 tx=0x7f3f80009ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.545+0000 7f3f6effd700 1 -- 172.21.15.3:0/672205605 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78098ce0 con 0x558e0c741f40 2023-08-11T14:17:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.545+0000 7f3f6effd700 1 -- 172.21.15.3:0/672205605 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f780985e0 con 0x558e0c741f40 2023-08-11T14:17:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.545+0000 7f3f6effd700 1 -- 172.21.15.3:0/672205605 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78096a30 con 0x558e0c741f40 2023-08-11T14:17:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.550+0000 7f3fa289eb80 1 -- 172.21.15.3:0/672205605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 msgr2=0x558e0c742360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.550+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 0x558e0c742360 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3f780c8690 tx=0x7f3f78004860 comp rx=0 tx=0).stop 2023-08-11T14:17:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.563+0000 7f3fa289eb80 1 -- 172.21.15.3:0/672205605 shutdown_connections 2023-08-11T14:17:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.563+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c743790 0x558e0c743c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.563+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c741f40 0x558e0c742360 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.563+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/672205605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7428a0 0x558e0c7431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.563+0000 7f3fa289eb80 1 -- 172.21.15.3:0/672205605 >> 172.21.15.3:0/672205605 conn(0x558e0c66f8f0 msgr2=0x558e0c73e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.573+0000 7f3fa289eb80 1 -- 172.21.15.3:0/672205605 shutdown_connections 2023-08-11T14:17:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.573+0000 7f3fa289eb80 1 -- 172.21.15.3:0/672205605 wait complete. 2023-08-11T14:17:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.574+0000 7f3fa289eb80 1 -- start start 2023-08-11T14:17:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.574+0000 7f3fa289eb80 1 Processor -- start 2023-08-11T14:17:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c75cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c75d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c632a60 con 0x558e0c743790 2023-08-11T14:17:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c632760 con 0x558e0c7428a0 2023-08-11T14:17:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c5168c0 con 0x558e0c741f40 2023-08-11T14:17:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.575+0000 7f3fa289eb80 10 client.0 _ll_register_callbacks cb 0x558e0c635a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.580+0000 7f3f8d59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.580+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c75d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.581+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c75d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42964/0 (socket says 172.21.15.3:42964) 2023-08-11T14:17:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.580+0000 7f3f8d59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41904/0 (socket says 172.21.15.3:41904) 2023-08-11T14:17:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.581+0000 7f3f87fff700 1 -- 172.21.15.3:0/2044882078 learned_addr learned my addr 172.21.15.3:0/2044882078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 -- 172.21.15.3:0/2044882078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 msgr2=0x558e0c75cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c75cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 -- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 msgr2=0x558e0c75d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c75d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 -- 172.21.15.3:0/2044882078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88078de0 con 0x558e0c743790 2023-08-11T14:17:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.588+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3f80007900 tx=0x7f3f80004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f08bb39b06fb8e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.589+0000 7f3f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:17:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.600+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007540 con 0x558e0c743790 2023-08-11T14:17:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.600+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3f8000bdd0 con 0x558e0c743790 2023-08-11T14:17:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.601+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e0c7357e0 con 0x558e0c743790 2023-08-11T14:17:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.613+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f80005040 con 0x558e0c743790 2023-08-11T14:17:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.614+0000 7f3fa289eb80 10 client.16476 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.614+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558e0c4c0d80 con 0x558e0c743790 2023-08-11T14:17:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.629+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e0c7357e0 con 0x558e0c743790 2023-08-11T14:17:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.629+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3f80026690 con 0x558e0c743790 2023-08-11T14:17:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 10 client.16476 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 20 client.16476 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c768af0 0x558e0c76af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 4 client.16476 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:17:48.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.636+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e0c711480 con 0x558e0c768af0 2023-08-11T14:17:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.651+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c768af0 0x558e0c76af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.694+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c768af0 0x558e0c76af10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c76accbb4bc1ee2 server_cookie=dbe094f37b271aea in_seq=0 out_seq=0 2023-08-11T14:17:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.697+0000 7f3f67fff700 10 client.16476 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.704+0000 7f3f67fff700 1 -- 172.21.15.3:0/2044882078 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558e0c711480 con 0x558e0c768af0 2023-08-11T14:17:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.704+0000 7f3f67fff700 10 client.16476 handle_command_reply: tid=0 2023-08-11T14:17:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.708+0000 7f3fa289eb80 1 client.16476 shutdown 2023-08-11T14:17:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.728+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 msgr2=0x558e0c764220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.728+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3f80007900 tx=0x7f3f80004cd0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 shutdown_connections 2023-08-11T14:17:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c764220 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c75cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c768af0 0x558e0c76af10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/2044882078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c75d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 >> 172.21.15.3:0/2044882078 conn(0x558e0c66f8f0 msgr2=0x558e0c73d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.732+0000 7f3f67fff700 0 client.16476 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.741+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 shutdown_connections 2023-08-11T14:17:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.741+0000 7f3fa289eb80 1 -- 172.21.15.3:0/2044882078 wait complete. 2023-08-11T14:17:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.742+0000 7f3fa289eb80 20 client.16476 trim_cache size 0 max 16384 2023-08-11T14:17:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.772+0000 7f3fa289eb80 1 Processor -- start 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.772+0000 7f3fa289eb80 1 -- start start 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c742320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c73e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c75bff0 con 0x558e0c743790 2023-08-11T14:17:48.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c741b70 con 0x558e0c7428a0 2023-08-11T14:17:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.773+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c75de20 con 0x558e0c741f40 2023-08-11T14:17:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.774+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.774+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:42974/0 (socket says 172.21.15.3:42974) 2023-08-11T14:17:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.774+0000 7f3f87fff700 1 -- 172.21.15.3:0/4288414687 learned_addr learned my addr 172.21.15.3:0/4288414687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.776+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c742320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.780+0000 7f3f87fff700 1 -- 172.21.15.3:0/4288414687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 msgr2=0x558e0c742320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.780+0000 7f3f87fff700 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c742320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.780+0000 7f3f87fff700 1 -- 172.21.15.3:0/4288414687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 msgr2=0x558e0c73e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.780+0000 7f3f87fff700 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c73e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.780+0000 7f3f87fff700 1 -- 172.21.15.3:0/4288414687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0c4a9c00 con 0x558e0c7428a0 2023-08-11T14:17:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.781+0000 7f3f87fff700 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009da0 tx=0x7f3f7c013740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758bfdd29d5bd72a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.783+0000 7f3f6f7fe700 1 -- 172.21.15.3:0/4288414687 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c018070 con 0x558e0c7428a0 2023-08-11T14:17:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.783+0000 7f3f6f7fe700 1 -- 172.21.15.3:0/4288414687 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c004510 con 0x558e0c7428a0 2023-08-11T14:17:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.783+0000 7f3f6f7fe700 1 -- 172.21.15.3:0/4288414687 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0116a0 con 0x558e0c7428a0 2023-08-11T14:17:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.784+0000 7f3fa289eb80 1 -- 172.21.15.3:0/4288414687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 msgr2=0x558e0c7641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.784+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009da0 tx=0x7f3f7c013740 comp rx=0 tx=0).stop 2023-08-11T14:17:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.785+0000 7f3fa289eb80 1 -- 172.21.15.3:0/4288414687 shutdown_connections 2023-08-11T14:17:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.785+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c73e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.785+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c741f40 0x558e0c742320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.785+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/4288414687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c7428a0 0x558e0c7641e0 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009da0 tx=0x7f3f7c013740 comp rx=0 tx=0).stop 2023-08-11T14:17:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.785+0000 7f3fa289eb80 1 -- 172.21.15.3:0/4288414687 >> 172.21.15.3:0/4288414687 conn(0x558e0c66f8f0 msgr2=0x558e0c749c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.794+0000 7f3fa289eb80 1 -- 172.21.15.3:0/4288414687 shutdown_connections 2023-08-11T14:17:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.801+0000 7f3fa289eb80 1 -- 172.21.15.3:0/4288414687 wait complete. 2023-08-11T14:17:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.801+0000 7f3fa289eb80 1 -- start start 2023-08-11T14:17:48.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.802+0000 7f3fa289eb80 1 Processor -- start 2023-08-11T14:17:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c741f40 0x558e0c763a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7620a0 0x558e0c762520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c762aa0 con 0x558e0c743790 2023-08-11T14:17:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c762c10 con 0x558e0c741f40 2023-08-11T14:17:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0c762dd0 con 0x558e0c7620a0 2023-08-11T14:17:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.816+0000 7f3fa289eb80 10 client.0 _ll_register_callbacks cb 0x7f3f80007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:17:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.817+0000 7f3f8cd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c741f40 0x558e0c763a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.823+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.823+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41922/0 (socket says 172.21.15.3:41922) 2023-08-11T14:17:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.823+0000 7f3f87fff700 1 -- 172.21.15.3:0/497582687 learned_addr learned my addr 172.21.15.3:0/497582687 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:17:48.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.823+0000 7f3f8d59c700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7620a0 0x558e0c762520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.826+0000 7f3f87fff700 1 -- 172.21.15.3:0/497582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7620a0 msgr2=0x558e0c762520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.826+0000 7f3f87fff700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7620a0 0x558e0c762520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.826+0000 7f3f87fff700 1 -- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c741f40 msgr2=0x558e0c763a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:17:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.826+0000 7f3f87fff700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c741f40 0x558e0c763a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:48.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.826+0000 7f3f87fff700 1 -- 172.21.15.3:0/497582687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c009a80 con 0x558e0c743790 2023-08-11T14:17:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.840+0000 7f3f87fff700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c015040 tx=0x7f3f7c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4409b618a338af3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:17:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.841+0000 7f3f67fff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:17:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.842+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c018040 con 0x558e0c743790 2023-08-11T14:17:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.842+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3f7c005040 con 0x558e0c743790 2023-08-11T14:17:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.848+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f780985e0 con 0x558e0c743790 2023-08-11T14:17:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.855+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f7c005220 con 0x558e0c743790 2023-08-11T14:17:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.859+0000 7f3fa289eb80 10 client.16479 fetch_fsmap learned FSMap version 56 2023-08-11T14:17:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.859+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558e0c4e71f0 con 0x558e0c743790 2023-08-11T14:17:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.863+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f780985e0 con 0x558e0c743790 2023-08-11T14:17:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.863+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f3f7c02c050 con 0x558e0c743790 2023-08-11T14:17:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 10 client.16479 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:17:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:17:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 20 client.16479 populate_metadata read hostname 'smithi003' 2023-08-11T14:17:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c7bde90 0x558e0c7c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:17:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 4 client.16479 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:17:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.869+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e0c68d7c0 con 0x558e0c7bde90 2023-08-11T14:17:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.870+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c7bde90 0x558e0c7c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:17:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.907+0000 7f3f8cd9b700 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c7bde90 0x558e0c7c02b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a14201ad67ecc6 server_cookie=593cb05cd9cbf404 in_seq=0 out_seq=0 2023-08-11T14:17:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:48.908+0000 7f3f67fff700 10 client.16479 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:50.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:49 smithi103 ceph-mon[130434]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-08-11T14:17:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:49 smithi003 ceph-mon[97195]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-08-11T14:17:50.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:49 smithi187 ceph-mon[130973]: pgmap v764: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2023-08-11T14:17:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.311+0000 7f3f67fff700 1 -- 172.21.15.3:0/497582687 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e0c68d7c0 con 0x558e0c7bde90 2023-08-11T14:17:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.311+0000 7f3f67fff700 10 client.16479 handle_command_reply: tid=0 2023-08-11T14:17:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.311+0000 7f3fa289eb80 1 client.16479 shutdown 2023-08-11T14:17:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 msgr2=0x558e0c763fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c015040 tx=0x7f3f7c003a80 comp rx=0 tx=0).stop 2023-08-11T14:17:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 shutdown_connections 2023-08-11T14:17:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e0c7620a0 0x558e0c762520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558e0c7bde90 0x558e0c7c02b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e0c741f40 0x558e0c763a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 --2- 172.21.15.3:0/497582687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558e0c743790 0x558e0c763fd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 >> 172.21.15.3:0/497582687 conn(0x558e0c66f8f0 msgr2=0x558e0c73e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.314+0000 7f3f67fff700 0 client.16479 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:17:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.316+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 shutdown_connections 2023-08-11T14:17:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.326+0000 7f3fa289eb80 1 -- 172.21.15.3:0/497582687 wait complete. 2023-08-11T14:17:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:17:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:17:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:17:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:17:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.349+0000 7f3fa289eb80 20 client.16479 trim_cache size 0 max 16384 2023-08-11T14:17:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.360+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 msgr2=0x7f3f700a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.360+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 0x7f3f700a2610 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x558e0c742f70 tx=0x7f3f80009ae0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.360+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 msgr2=0x7f3f881a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:17:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.360+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 0x7f3f881a5630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b450 tx=0x7f3f7800b810 comp rx=0 tx=0).stop 2023-08-11T14:17:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 shutdown_connections 2023-08-11T14:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f8810cc60 0x7f3f881a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f88107920 0x7f3f881a5630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3f700a0150 0x7f3f700a2610 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 --2- 172.21.15.3:0/209990203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f8810a290 0x7f3f881a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:17:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.372+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 >> 172.21.15.3:0/209990203 conn(0x7f3f8806d750 msgr2=0x7f3f881090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:17:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.385+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 shutdown_connections 2023-08-11T14:17:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:17:50.393+0000 7f3f8efff700 1 -- 172.21.15.3:0/209990203 wait complete. 2023-08-11T14:17:51.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:51 smithi103 ceph-mon[130434]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:17:51.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:51 smithi003 ceph-mon[97195]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:17:51.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:51 smithi187 ceph-mon[130973]: pgmap v765: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:17:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 655.0 s, 152.4 tps, lat 638.881 ms stddev 945.353 2023-08-11T14:17:53.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:53 smithi103 ceph-mon[130434]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:53.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:53 smithi003 ceph-mon[97195]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:53.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:53 smithi187 ceph-mon[130973]: pgmap v766: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:17:56.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:56 smithi103 ceph-mon[130434]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:17:56.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:56 smithi003 ceph-mon[97195]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:17:56.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:56 smithi187 ceph-mon[130973]: pgmap v767: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:17:57.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:57 smithi103 ceph-mon[130434]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:17:57.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:57 smithi003 ceph-mon[97195]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:17:57.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:57 smithi187 ceph-mon[130973]: pgmap v768: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:17:58.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 660.0 s, 924.4 tps, lat 116.986 ms stddev 138.511 2023-08-11T14:18:00.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:17:59 smithi003 ceph-mon[97195]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 990 op/s 2023-08-11T14:18:00.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:17:59 smithi187 ceph-mon[130973]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 990 op/s 2023-08-11T14:18:00.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:17:59 smithi103 ceph-mon[130434]: pgmap v769: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 990 op/s 2023-08-11T14:18:02.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:02 smithi103 ceph-mon[130434]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:18:02.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:02 smithi187 ceph-mon[130973]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:18:02.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:02 smithi003 ceph-mon[97195]: pgmap v770: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-08-11T14:18:03.091 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 665.0 s, 421.8 tps, lat 211.942 ms stddev 516.161 2023-08-11T14:18:03.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:03 smithi003 ceph-mon[97195]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:18:03.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:03 smithi187 ceph-mon[130973]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:18:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1887054400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 msgr2=0x7fe0bc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1887054400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc109d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0b001c0d0 tx=0x7fe0b001c3e0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1887054400 shutdown_connections 2023-08-11T14:18:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1887054400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10cc60 0x7fe0bc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1887054400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc109d50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1887054400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10a290 0x7fe0bc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.825+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1887054400 >> 172.21.15.3:0/1887054400 conn(0x7fe0bc06d750 msgr2=0x7fe0bc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.826+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1887054400 shutdown_connections 2023-08-11T14:18:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.826+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1887054400 wait complete. 2023-08-11T14:18:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.827+0000 7fe0c3dff700 1 Processor -- start 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 -- start start 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 0x7fe0bc19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10cc60 0x7fe0bc19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc19e860 con 0x7fe0bc10a290 2023-08-11T14:18:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc19e9d0 con 0x7fe0bc10cc60 2023-08-11T14:18:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.828+0000 7fe0c3dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc19eb90 con 0x7fe0bc107920 2023-08-11T14:18:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52360/0 (socket says 172.21.15.3:52360) 2023-08-11T14:18:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 -- 172.21.15.3:0/1240118727 learned_addr learned my addr 172.21.15.3:0/1240118727 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c239c700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10cc60 0x7fe0bc19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 0x7fe0bc19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 -- 172.21.15.3:0/1240118727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10cc60 msgr2=0x7fe0bc19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10cc60 0x7fe0bc19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 -- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 msgr2=0x7fe0bc19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 0x7fe0bc19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c1b9b700 1 -- 172.21.15.3:0/1240118727 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b001ad10 con 0x7fe0bc107920 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.829+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 0x7fe0bc19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0b001de90 tx=0x7fe0b001dec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0baffd700 1 -- 172.21.15.3:0/1240118727 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b001f050 con 0x7fe0bc107920 2023-08-11T14:18:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0baffd700 1 -- 172.21.15.3:0/1240118727 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b0005570 con 0x7fe0bc107920 2023-08-11T14:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0baffd700 1 -- 172.21.15.3:0/1240118727 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b002cb30 con 0x7fe0bc107920 2023-08-11T14:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc19ee10 con 0x7fe0bc107920 2023-08-11T14:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.830+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc1a5580 con 0x7fe0bc107920 2023-08-11T14:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.837+0000 7fe0baffd700 1 -- 172.21.15.3:0/1240118727 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe0b002cc90 con 0x7fe0bc107920 2023-08-11T14:18:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.838+0000 7fe0d7629b80 1 Processor -- start 2023-08-11T14:18:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.840+0000 7fe0d7629b80 1 -- start start 2023-08-11T14:18:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f94121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9412c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f936a5d0 con 0x5645f9412790 2023-08-11T14:18:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9367ef0 con 0x5645f9410f40 2023-08-11T14:18:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.841+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f93637e0 con 0x5645f94118a0 2023-08-11T14:18:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48278/0 (socket says 172.21.15.3:48278) 2023-08-11T14:18:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 -- 172.21.15.3:0/2576891363 learned_addr learned my addr 172.21.15.3:0/2576891363 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0baffd700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 0x7fe0ac0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0baffd700 1 -- 172.21.15.3:0/1240118727 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe0b00ddec0 con 0x7fe0bc107920 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 -- 172.21.15.3:0/2576891363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 msgr2=0x5645f94121b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f94121b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 -- 172.21.15.3:0/2576891363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 msgr2=0x5645f9412c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.842+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9412c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.843+0000 7fe0c239c700 1 -- 172.21.15.3:0/2576891363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f91e35d0 con 0x5645f9410f40 2023-08-11T14:18:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.847+0000 7fe0c239c700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 0x7fe0ac0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.848+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e00 tx=0x7fe0b4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee0f36289947222 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.849+0000 7fe0a7fff700 1 -- 172.21.15.3:0/2576891363 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4021030 con 0x5645f9410f40 2023-08-11T14:18:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.849+0000 7fe0a7fff700 1 -- 172.21.15.3:0/2576891363 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b4003680 con 0x5645f9410f40 2023-08-11T14:18:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.849+0000 7fe0a7fff700 1 -- 172.21.15.3:0/2576891363 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400f6f0 con 0x5645f9410f40 2023-08-11T14:18:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2576891363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 msgr2=0x5645f9411360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e00 tx=0x7fe0b4011720 comp rx=0 tx=0).stop 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0c239c700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 0x7fe0ac0a2490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4012010 tx=0x7fe0b400f850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2576891363 shutdown_connections 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9412c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f9411360 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2576891363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f94121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.850+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2576891363 >> 172.21.15.3:0/2576891363 conn(0x5645f933e8f0 msgr2=0x5645f940d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.851+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2576891363 shutdown_connections 2023-08-11T14:18:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.851+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2576891363 wait complete. 2023-08-11T14:18:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.852+0000 7fe0d7629b80 1 -- start start 2023-08-11T14:18:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.852+0000 7fe0d7629b80 1 Processor -- start 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.852+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9410f40 0x5645f941ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f941b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f94221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9301a60 con 0x5645f9410f40 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9301760 con 0x5645f9412790 2023-08-11T14:18:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f91e58c0 con 0x5645f94118a0 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0d7629b80 10 client.0 _ll_register_callbacks cb 0x5645f9304a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f94221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f94221e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48288/0 (socket says 172.21.15.3:48288) 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0c239c700 1 -- 172.21.15.3:0/1194797083 learned_addr learned my addr 172.21.15.3:0/1194797083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9410f40 0x5645f941ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.853+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f941b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 -- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 msgr2=0x5645f94221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f94221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 -- 172.21.15.3:0/1194797083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9410f40 msgr2=0x5645f941ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9410f40 0x5645f941ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 -- 172.21.15.3:0/1194797083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b4009980 con 0x5645f94118a0 2023-08-11T14:18:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0c139a700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f941b1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5645f9411f70 tx=0x7fe0a800e650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70b1b3a58283b7b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004a30 con 0x5645f94118a0 2023-08-11T14:18:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.854+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe0a800ee80 con 0x5645f94118a0 2023-08-11T14:18:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.855+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645f94047e0 con 0x5645f94118a0 2023-08-11T14:18:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.855+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a8004b90 con 0x5645f94118a0 2023-08-11T14:18:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.855+0000 7fe0d7629b80 10 client.26668 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.855+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5645f918fd80 con 0x5645f94118a0 2023-08-11T14:18:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.855+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645f94047e0 con 0x5645f94118a0 2023-08-11T14:18:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe0a800c070 con 0x5645f94118a0 2023-08-11T14:18:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 10 client.26668 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 20 client.26668 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f9426b40 0x5645f9428f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 4 client.26668 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:18:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.862+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645f93e0480 con 0x5645f9426b40 2023-08-11T14:18:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.863+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f9426b40 0x5645f9428f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.868+0000 7fe0a57fa700 10 client.26668 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.868+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f9426b40 0x5645f9428f60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0eb8a53cbe20d03 server_cookie=e247e60cb3849f5b in_seq=0 out_seq=0 2023-08-11T14:18:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.870+0000 7fe0a57fa700 1 -- 172.21.15.3:0/1194797083 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5645f93e0480 con 0x5645f9426b40 2023-08-11T14:18:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.870+0000 7fe0a57fa700 10 client.26668 handle_command_reply: tid=0 2023-08-11T14:18:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.870+0000 7fe0d7629b80 1 client.26668 shutdown 2023-08-11T14:18:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.872+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 msgr2=0x5645f941b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.872+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f941b1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5645f9411f70 tx=0x7fe0a800e650 comp rx=0 tx=0).stop 2023-08-11T14:18:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 shutdown_connections 2023-08-11T14:18:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f9426b40 0x5645f9428f60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f94221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9410f40 0x5645f941ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/1194797083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f941b1a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 >> 172.21.15.3:0/1194797083 conn(0x5645f933e8f0 msgr2=0x5645f940c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0a57fa700 0 client.26668 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 shutdown_connections 2023-08-11T14:18:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.873+0000 7fe0d7629b80 1 -- 172.21.15.3:0/1194797083 wait complete. 2023-08-11T14:18:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.874+0000 7fe0d7629b80 20 client.26668 trim_cache size 0 max 16384 2023-08-11T14:18:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.907+0000 7fe0d7629b80 1 Processor -- start 2023-08-11T14:18:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.907+0000 7fe0d7629b80 1 -- start start 2023-08-11T14:18:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.907+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f9410f40 0x5645f94200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.907+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f94118a0 0x5645f941ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f941a1b0 con 0x5645f94118a0 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f94047c0 con 0x5645f9412790 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9410a00 con 0x5645f9410f40 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0c239c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:48304/0 (socket says 172.21.15.3:48304) 2023-08-11T14:18:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.908+0000 7fe0c239c700 1 -- 172.21.15.3:0/2891995217 learned_addr learned my addr 172.21.15.3:0/2891995217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.909+0000 7fe0c239c700 1 -- 172.21.15.3:0/2891995217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f9410f40 msgr2=0x5645f94200f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:18:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.909+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f9410f40 0x5645f94200f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.909+0000 7fe0c239c700 1 -- 172.21.15.3:0/2891995217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f94118a0 msgr2=0x5645f941ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.909+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f94118a0 0x5645f941ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.909+0000 7fe0c239c700 1 -- 172.21.15.3:0/2891995217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f9178c00 con 0x5645f9412790 2023-08-11T14:18:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.910+0000 7fe0c239c700 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4011bd0 tx=0x7fe0b4028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6f03ee7d1aeeb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.910+0000 7fe0b8bff700 1 -- 172.21.15.3:0/2891995217 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4021030 con 0x5645f9412790 2023-08-11T14:18:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.910+0000 7fe0b8bff700 1 -- 172.21.15.3:0/2891995217 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b400fdf0 con 0x5645f9412790 2023-08-11T14:18:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.911+0000 7fe0b8bff700 1 -- 172.21.15.3:0/2891995217 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4005350 con 0x5645f9412790 2023-08-11T14:18:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2891995217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 msgr2=0x5645f941b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4011bd0 tx=0x7fe0b4028000 comp rx=0 tx=0).stop 2023-08-11T14:18:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2891995217 shutdown_connections 2023-08-11T14:18:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9412790 0x5645f941b1a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f9410f40 0x5645f94200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/2891995217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f94118a0 0x5645f941ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.912+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2891995217 >> 172.21.15.3:0/2891995217 conn(0x5645f933e8f0 msgr2=0x5645f941a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.914+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2891995217 shutdown_connections 2023-08-11T14:18:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.914+0000 7fe0d7629b80 1 -- 172.21.15.3:0/2891995217 wait complete. 2023-08-11T14:18:03.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.915+0000 7fe0d7629b80 1 -- start start 2023-08-11T14:18:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.915+0000 7fe0d7629b80 1 Processor -- start 2023-08-11T14:18:03.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f94729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9479ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f947a020 con 0x5645f9412790 2023-08-11T14:18:03.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f947a160 con 0x5645f9410f40 2023-08-11T14:18:03.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f947a2f0 con 0x5645f94118a0 2023-08-11T14:18:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0d7629b80 10 client.0 _ll_register_callbacks cb 0x7fe0a8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0c139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0c139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:52390/0 (socket says 172.21.15.3:52390) 2023-08-11T14:18:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.916+0000 7fe0c139a700 1 -- 172.21.15.3:0/3614080679 learned_addr learned my addr 172.21.15.3:0/3614080679 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.917+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f94729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.918+0000 7fe0c139a700 1 -- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 msgr2=0x5645f94729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.918+0000 7fe0c139a700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f94729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.918+0000 7fe0c139a700 1 -- 172.21.15.3:0/3614080679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 msgr2=0x5645f9479ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.918+0000 7fe0c139a700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9479ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.919+0000 7fe0c139a700 1 -- 172.21.15.3:0/3614080679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645f91e35d0 con 0x5645f94118a0 2023-08-11T14:18:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.919+0000 7fe0c139a700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009cf0 tx=0x7fe0a80232e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa744e9ee9ddea00 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:03.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.920+0000 7fe0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.920+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011bb0 con 0x5645f94118a0 2023-08-11T14:18:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.920+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe0a8000e40 con 0x5645f94118a0 2023-08-11T14:18:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.920+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0b4003680 con 0x5645f94118a0 2023-08-11T14:18:03.929 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.926+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a8004020 con 0x5645f94118a0 2023-08-11T14:18:03.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:03 smithi103 ceph-mon[130434]: pgmap v771: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:18:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.928+0000 7fe0d7629b80 10 client.26671 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.928+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5645f91b61f0 con 0x5645f94118a0 2023-08-11T14:18:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.930+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b4003680 con 0x5645f94118a0 2023-08-11T14:18:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.930+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe0a800c070 con 0x5645f94118a0 2023-08-11T14:18:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 20 client.26671 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f947bde0 0x5645f947e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 4 client.26671 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:18:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.931+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645f935c7c0 con 0x5645f947bde0 2023-08-11T14:18:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.934+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f947bde0 0x5645f947e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.941+0000 7fe0c1b9b700 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f947bde0 0x5645f947e200 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d862d8152131dee server_cookie=bf5ebe264cb37c1e in_seq=0 out_seq=0 2023-08-11T14:18:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.942+0000 7fe0a57fa700 10 client.26671 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.964+0000 7fe0a57fa700 1 -- 172.21.15.3:0/3614080679 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645f935c7c0 con 0x5645f947bde0 2023-08-11T14:18:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.964+0000 7fe0a57fa700 10 client.26671 handle_command_reply: tid=0 2023-08-11T14:18:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.964+0000 7fe0d7629b80 1 client.26671 shutdown 2023-08-11T14:18:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.970+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 msgr2=0x5645f9472f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.970+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009cf0 tx=0x7fe0a80232e0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 shutdown_connections 2023-08-11T14:18:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645f9412790 0x5645f9479ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645f947bde0 0x5645f947e200 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645f9410f40 0x5645f94729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 --2- 172.21.15.3:0/3614080679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645f94118a0 0x5645f9472f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 >> 172.21.15.3:0/3614080679 conn(0x5645f933e8f0 msgr2=0x5645f9415720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.971+0000 7fe0a57fa700 0 client.26671 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.975+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 shutdown_connections 2023-08-11T14:18:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.977+0000 7fe0d7629b80 1 -- 172.21.15.3:0/3614080679 wait complete. 2023-08-11T14:18:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.979+0000 7fe0d7629b80 20 client.26671 trim_cache size 0 max 16384 2023-08-11T14:18:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:18:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:18:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:18:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:18:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.981+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 msgr2=0x7fe0ac0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.981+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 0x7fe0ac0a2490 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4012010 tx=0x7fe0b400f850 comp rx=0 tx=0).stop 2023-08-11T14:18:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.981+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 msgr2=0x7fe0bc19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.981+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0b001de90 tx=0x7fe0b001dec0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 shutdown_connections 2023-08-11T14:18:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe0ac09ffd0 0x7fe0ac0a2490 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0bc10cc60 0x7fe0bc19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0bc107920 0x7fe0bc19ce70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 --2- 172.21.15.3:0/1240118727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc10a290 0x7fe0bc19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.982+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 >> 172.21.15.3:0/1240118727 conn(0x7fe0bc06d750 msgr2=0x7fe0bc10ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.984+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 shutdown_connections 2023-08-11T14:18:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:03.984+0000 7fe0c3dff700 1 -- 172.21.15.3:0/1240118727 wait complete. 2023-08-11T14:18:05.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:05 smithi003 ceph-mon[97195]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 865 op/s 2023-08-11T14:18:05.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:05 smithi187 ceph-mon[130973]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 865 op/s 2023-08-11T14:18:05.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:05 smithi103 ceph-mon[130434]: pgmap v772: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 865 op/s 2023-08-11T14:18:07.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:07 smithi103 ceph-mon[130434]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-08-11T14:18:07.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:07 smithi003 ceph-mon[97195]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-08-11T14:18:07.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:07 smithi187 ceph-mon[130973]: pgmap v773: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-08-11T14:18:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 670.0 s, 759.7 tps, lat 145.688 ms stddev 270.160 2023-08-11T14:18:10.294 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:09 smithi003 ceph-mon[97195]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-11T14:18:10.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:09 smithi187 ceph-mon[130973]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-11T14:18:10.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:09 smithi103 ceph-mon[130434]: pgmap v774: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 16 MiB/s wr, 1.07k op/s 2023-08-11T14:18:11.662 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:11 smithi003 ceph-mon[97195]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:18:11.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:11 smithi103 ceph-mon[130434]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:18:11.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:11 smithi187 ceph-mon[130973]: pgmap v775: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 1.14k op/s 2023-08-11T14:18:13.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 675.0 s, 303.8 tps, lat 324.702 ms stddev 684.305 2023-08-11T14:18:14.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:14 smithi103 ceph-mon[130434]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:18:14.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:14 smithi003 ceph-mon[97195]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:18:14.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:14 smithi187 ceph-mon[130973]: pgmap v776: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-08-11T14:18:15.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:15 smithi003 ceph-mon[97195]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 893 op/s 2023-08-11T14:18:15.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:15 smithi187 ceph-mon[130973]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 893 op/s 2023-08-11T14:18:15.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:15 smithi103 ceph-mon[130434]: pgmap v777: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 12 MiB/s wr, 893 op/s 2023-08-11T14:18:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/2073579154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a3810d010 msgr2=0x7f2a3810f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/2073579154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a3810d010 0x7f2a3810f4d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800ba80 tx=0x7f2a2800bd90 comp rx=0 tx=0).stop 2023-08-11T14:18:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/2073579154 shutdown_connections 2023-08-11T14:18:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/2073579154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a3810d010 0x7f2a3810f4d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/2073579154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a38107b90 0x7f2a38109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/2073579154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.339+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/2073579154 >> 172.21.15.3:0/2073579154 conn(0x7f2a3806daf0 msgr2=0x7f2a3806ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.350+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/2073579154 shutdown_connections 2023-08-11T14:18:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.361+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/2073579154 wait complete. 2023-08-11T14:18:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.362+0000 7f2a3e7ff700 1 Processor -- start 2023-08-11T14:18:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.363+0000 7f2a3e7ff700 1 -- start start 2023-08-11T14:18:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a38107b90 0x7f2a3819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a3810d010 0x7f2a3819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819e6c0 con 0x7f2a3810d010 2023-08-11T14:18:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819e830 con 0x7f2a38107b90 2023-08-11T14:18:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a3e7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3819e9f0 con 0x7f2a3810a550 2023-08-11T14:18:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a377fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a377fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:58706/0 (socket says 172.21.15.3:58706) 2023-08-11T14:18:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.364+0000 7f2a377fe700 1 -- 172.21.15.3:0/3099711997 learned_addr learned my addr 172.21.15.3:0/3099711997 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 -- 172.21.15.3:0/3099711997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a38107b90 msgr2=0x7f2a3819cc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:18:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a38107b90 0x7f2a3819cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 -- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a3810d010 msgr2=0x7f2a3819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a3810d010 0x7f2a3819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 -- 172.21.15.3:0/3099711997 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2800b730 con 0x7f2a3810a550 2023-08-11T14:18:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.365+0000 7f2a377fe700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e970 tx=0x7f2a2c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.366+0000 7f2a357fa700 1 -- 172.21.15.3:0/3099711997 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00cbd0 con 0x7f2a3810a550 2023-08-11T14:18:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.366+0000 7f2a357fa700 1 -- 172.21.15.3:0/3099711997 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2c00cd30 con 0x7f2a3810a550 2023-08-11T14:18:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.366+0000 7f2a357fa700 1 -- 172.21.15.3:0/3099711997 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c005650 con 0x7f2a3810a550 2023-08-11T14:18:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.366+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3819ecd0 con 0x7f2a3810a550 2023-08-11T14:18:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.366+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a34a0 con 0x7f2a3810a550 2023-08-11T14:18:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.378+0000 7f2a5206fb80 1 Processor -- start 2023-08-11T14:18:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.380+0000 7f2a5206fb80 1 -- start start 2023-08-11T14:18:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4d7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d42f5d0 con 0x556b0d4d7790 2023-08-11T14:18:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d42cef0 con 0x556b0d4d5f40 2023-08-11T14:18:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.381+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4287e0 con 0x556b0d4d68a0 2023-08-11T14:18:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.382+0000 7f2a3cd9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.382+0000 7f2a3cd9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41530/0 (socket says 172.21.15.3:41530) 2023-08-11T14:18:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.382+0000 7f2a3cd9c700 1 -- 172.21.15.3:0/1099432900 learned_addr learned my addr 172.21.15.3:0/1099432900 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.383+0000 7f2a357fa700 1 -- 172.21.15.3:0/3099711997 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2a2c004750 con 0x7f2a3810a550 2023-08-11T14:18:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.385+0000 7f2a357fa700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 0x7f2a200a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.385+0000 7f2a357fa700 1 -- 172.21.15.3:0/3099711997 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2a2c014070 con 0x7f2a3810a550 2023-08-11T14:18:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a377fe700 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4d7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a3cd9c700 1 -- 172.21.15.3:0/1099432900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 msgr2=0x556b0d4d71b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:18:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4d71b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a3cd9c700 1 -- 172.21.15.3:0/1099432900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 msgr2=0x556b0d4d7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4d7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.386+0000 7f2a3cd9c700 1 -- 172.21.15.3:0/1099432900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b0d2a85d0 con 0x556b0d4d5f40 2023-08-11T14:18:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.387+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005be0 tx=0x7f2a2800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c5dc89970dcad5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.387+0000 7f2a1e7fc700 1 -- 172.21.15.3:0/1099432900 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28010040 con 0x556b0d4d5f40 2023-08-11T14:18:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.387+0000 7f2a1e7fc700 1 -- 172.21.15.3:0/1099432900 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a28014630 con 0x556b0d4d5f40 2023-08-11T14:18:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.387+0000 7f2a1e7fc700 1 -- 172.21.15.3:0/1099432900 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28005580 con 0x556b0d4d5f40 2023-08-11T14:18:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.388+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1099432900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 msgr2=0x556b0d4d6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a28005be0 tx=0x7f2a2800b610 comp rx=0 tx=0).stop 2023-08-11T14:18:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1099432900 shutdown_connections 2023-08-11T14:18:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4d7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4d6360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1099432900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4d71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1099432900 >> 172.21.15.3:0/1099432900 conn(0x556b0d4038f0 msgr2=0x556b0d4d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.389+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1099432900 shutdown_connections 2023-08-11T14:18:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.390+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1099432900 wait complete. 2023-08-11T14:18:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.390+0000 7f2a5206fb80 1 -- start start 2023-08-11T14:18:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.391+0000 7f2a5206fb80 1 Processor -- start 2023-08-11T14:18:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d3c6a60 con 0x556b0d4d68a0 2023-08-11T14:18:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d3c6760 con 0x556b0d4d5f40 2023-08-11T14:18:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d2aa8c0 con 0x556b0d4d7790 2023-08-11T14:18:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a5206fb80 10 client.0 _ll_register_callbacks cb 0x556b0d3c9a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41546/0 (socket says 172.21.15.3:41546) 2023-08-11T14:18:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.392+0000 7f2a37fff700 1 -- 172.21.15.3:0/2053559135 learned_addr learned my addr 172.21.15.3:0/2053559135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a37fff700 1 -- 172.21.15.3:0/2053559135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 msgr2=0x556b0d4e7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a37fff700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4e7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a37fff700 1 -- 172.21.15.3:0/2053559135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 msgr2=0x556b0d4e00e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:18:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a37fff700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e00e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.393+0000 7f2a37fff700 1 -- 172.21.15.3:0/2053559135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381100c0 con 0x556b0d4d5f40 2023-08-11T14:18:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.394+0000 7f2a37fff700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000a6d0 tx=0x7f2a3000aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a200a0a7268b02c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.394+0000 7f2a177fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:18:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.394+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000f040 con 0x556b0d4d5f40 2023-08-11T14:18:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.394+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2a30003e10 con 0x556b0d4d5f40 2023-08-11T14:18:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.394+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b0d4c97e0 con 0x556b0d4d5f40 2023-08-11T14:18:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.395+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 0x7f2a200a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.396+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 0x7f2a200a2610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800ba80 tx=0x7f2a2802a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.402+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3000aea0 con 0x556b0d4d5f40 2023-08-11T14:18:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.402+0000 7f2a5206fb80 10 client.26675 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.402+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556b0d254d80 con 0x556b0d4d5f40 2023-08-11T14:18:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b0d4c97e0 con 0x556b0d4d5f40 2023-08-11T14:18:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2a3001d870 con 0x556b0d4d5f40 2023-08-11T14:18:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 10 client.26675 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 20 client.26675 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d4edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 4 client.26675 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:18:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.403+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b0d4a5480 con 0x556b0d4ebb50 2023-08-11T14:18:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.404+0000 7f2a377fe700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d4edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.405+0000 7f2a377fe700 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d4edf70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cbebef90b1809b server_cookie=500eea844652f085 in_seq=0 out_seq=0 2023-08-11T14:18:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a177fe700 10 client.26675 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a177fe700 1 -- 172.21.15.3:0/2053559135 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556b0d4a5480 con 0x556b0d4ebb50 2023-08-11T14:18:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a177fe700 10 client.26675 handle_command_reply: tid=0 2023-08-11T14:18:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a5206fb80 1 client.26675 shutdown 2023-08-11T14:18:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 msgr2=0x556b0d4dfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000a6d0 tx=0x7f2a3000aa90 comp rx=0 tx=0).stop 2023-08-11T14:18:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.406+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 shutdown_connections 2023-08-11T14:18:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d4edf70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4dfba0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/2053559135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a177fe700 0 client.26675 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 >> 172.21.15.3:0/2053559135 conn(0x556b0d4038f0 msgr2=0x556b0d4d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 shutdown_connections 2023-08-11T14:18:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 1 -- 172.21.15.3:0/2053559135 wait complete. 2023-08-11T14:18:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.407+0000 7f2a5206fb80 20 client.26675 trim_cache size 0 max 16384 2023-08-11T14:18:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 Processor -- start 2023-08-11T14:18:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 -- start start 2023-08-11T14:18:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4dda10 con 0x556b0d4d68a0 2023-08-11T14:18:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4e8dc0 con 0x556b0d4d5f40 2023-08-11T14:18:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.438+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4de2c0 con 0x556b0d4d7790 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41550/0 (socket says 172.21.15.3:41550) 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a377fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 -- 172.21.15.3:0/1152275725 learned_addr learned my addr 172.21.15.3:0/1152275725 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a3cd9c700 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 -- 172.21.15.3:0/1152275725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 msgr2=0x556b0d4da6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4da6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 -- 172.21.15.3:0/1152275725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 msgr2=0x556b0d4e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a37fff700 1 -- 172.21.15.3:0/1152275725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b0d23dc00 con 0x556b0d4d5f40 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.439+0000 7f2a377fe700 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.440+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001dc80 tx=0x7f2a3000d170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74bc7fb233ff859 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.440+0000 7f2a1effd700 1 -- 172.21.15.3:0/1152275725 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000aea0 con 0x556b0d4d5f40 2023-08-11T14:18:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.440+0000 7f2a1effd700 1 -- 172.21.15.3:0/1152275725 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3001a840 con 0x556b0d4d5f40 2023-08-11T14:18:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.440+0000 7f2a1effd700 1 -- 172.21.15.3:0/1152275725 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000f040 con 0x556b0d4d5f40 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.441+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1152275725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 msgr2=0x556b0d4f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.441+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001dc80 tx=0x7f2a3000d170 comp rx=0 tx=0).stop 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1152275725 shutdown_connections 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d7790 0x556b0d4da6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4f28d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1152275725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d68a0 0x556b0d4e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1152275725 >> 172.21.15.3:0/1152275725 conn(0x556b0d4038f0 msgr2=0x556b0d4ddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1152275725 shutdown_connections 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1152275725 wait complete. 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.442+0000 7f2a5206fb80 1 -- start start 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.443+0000 7f2a5206fb80 1 Processor -- start 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.443+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.443+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a5206fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4e6950 con 0x556b0d4d7790 2023-08-11T14:18:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4e6ac0 con 0x556b0d4d5f40 2023-08-11T14:18:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a5206fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d4e6c80 con 0x556b0d4d68a0 2023-08-11T14:18:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a5206fb80 10 client.0 _ll_register_callbacks cb 0x7f2a30007ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a37fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41562/0 (socket says 172.21.15.3:41562) 2023-08-11T14:18:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a37fff700 1 -- 172.21.15.3:0/1863298540 learned_addr learned my addr 172.21.15.3:0/1863298540 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.444+0000 7f2a377fe700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 -- 172.21.15.3:0/1863298540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 msgr2=0x556b0d4e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 -- 172.21.15.3:0/1863298540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 msgr2=0x556b0d4e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 -- 172.21.15.3:0/1863298540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381a34a0 con 0x556b0d4d5f40 2023-08-11T14:18:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a37fff700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001dc00 tx=0x7f2a3001aed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e5ab0022e72976 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a177fe700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000aea0 con 0x556b0d4d5f40 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a28014630 con 0x556b0d4d5f40 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.445+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2a3001d860 con 0x556b0d4d5f40 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.446+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a300135c0 con 0x556b0d4d5f40 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.446+0000 7f2a5206fb80 10 client.26681 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.446+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556b0d254d80 con 0x556b0d4d5f40 2023-08-11T14:18:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.449+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a28014630 con 0x556b0d4d5f40 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.449+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f2a300053d0 con 0x556b0d4d5f40 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 10 client.26681 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 20 client.26681 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d542610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 4 client.26681 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:18:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b0d4217c0 con 0x556b0d4ebb50 2023-08-11T14:18:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.452+0000 7f2a377fe700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d542610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.453+0000 7f2a377fe700 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d542610 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc85f10989135e68 server_cookie=199ec2e0259d9254 in_seq=0 out_seq=0 2023-08-11T14:18:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.453+0000 7f2a177fe700 10 client.26681 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.491+0000 7f2a177fe700 1 -- 172.21.15.3:0/1863298540 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b0d4217c0 con 0x556b0d4ebb50 2023-08-11T14:18:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.491+0000 7f2a177fe700 10 client.26681 handle_command_reply: tid=0 2023-08-11T14:18:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.491+0000 7f2a5206fb80 1 client.26681 shutdown 2023-08-11T14:18:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.494+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 msgr2=0x556b0d4e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.494+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a3001dc00 tx=0x7f2a3001aed0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 shutdown_connections 2023-08-11T14:18:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556b0d4d7790 0x556b0d4e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x556b0d4ebb50 0x556b0d542610 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556b0d4d5f40 0x556b0d4e4ec0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 --2- 172.21.15.3:0/1863298540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b0d4d68a0 0x556b0d4e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.495+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 >> 172.21.15.3:0/1863298540 conn(0x556b0d4038f0 msgr2=0x556b0d4d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.496+0000 7f2a177fe700 0 client.26681 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.497+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 shutdown_connections 2023-08-11T14:18:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.497+0000 7f2a5206fb80 1 -- 172.21.15.3:0/1863298540 wait complete. 2023-08-11T14:18:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.498+0000 7f2a5206fb80 20 client.26681 trim_cache size 0 max 16384 2023-08-11T14:18:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:18:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:18:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:18:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:18:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.515+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 msgr2=0x7f2a200a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.515+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 0x7f2a200a2610 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800ba80 tx=0x7f2a2802a040 comp rx=0 tx=0).stop 2023-08-11T14:18:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.515+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 msgr2=0x7f2a3819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.515+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00e970 tx=0x7f2a2c00ed30 comp rx=0 tx=0).stop 2023-08-11T14:18:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 shutdown_connections 2023-08-11T14:18:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a3810d010 0x7f2a3819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2a200a0150 0x7f2a200a2610 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a38107b90 0x7f2a3819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 --2- 172.21.15.3:0/3099711997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a3810a550 0x7f2a3819d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.517+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 >> 172.21.15.3:0/3099711997 conn(0x7f2a3806daf0 msgr2=0x7f2a3810b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.518+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 shutdown_connections 2023-08-11T14:18:17.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:17.518+0000 7f2a3e7ff700 1 -- 172.21.15.3:0/3099711997 wait complete. 2023-08-11T14:18:17.681 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:17 smithi103 ceph-mon[130434]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:18:17.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:17 smithi003 ceph-mon[97195]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:18:17.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:17 smithi187 ceph-mon[130973]: pgmap v778: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:18:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 680.0 s, 650.7 tps, lat 156.215 ms stddev 331.529 2023-08-11T14:18:20.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:19 smithi003 ceph-mon[97195]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 987 op/s 2023-08-11T14:18:20.297 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:19 smithi187 ceph-mon[130973]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 987 op/s 2023-08-11T14:18:20.434 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:19 smithi103 ceph-mon[130434]: pgmap v779: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 987 op/s 2023-08-11T14:18:21.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:21 smithi003 ceph-mon[97195]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 881 op/s 2023-08-11T14:18:21.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:21 smithi187 ceph-mon[130973]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 881 op/s 2023-08-11T14:18:21.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:21 smithi103 ceph-mon[130434]: pgmap v780: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 881 op/s 2023-08-11T14:18:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 685.0 s, 676.1 tps, lat 135.819 ms stddev 340.435 2023-08-11T14:18:23.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:23 smithi103 ceph-mon[130434]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:18:23.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:23 smithi003 ceph-mon[97195]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:18:23.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:23 smithi187 ceph-mon[130973]: pgmap v781: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:18:25.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:25 smithi003 ceph-mon[97195]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-08-11T14:18:25.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:25 smithi187 ceph-mon[130973]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-08-11T14:18:25.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:25 smithi103 ceph-mon[130434]: pgmap v782: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-08-11T14:18:27.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:27 smithi103 ceph-mon[130434]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:18:27.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:27 smithi003 ceph-mon[97195]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:18:27.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:27 smithi187 ceph-mon[130973]: pgmap v783: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.21k op/s 2023-08-11T14:18:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 690.0 s, 843.4 tps, lat 129.509 ms stddev 315.572 2023-08-11T14:18:30.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:29 smithi003 ceph-mon[97195]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-08-11T14:18:30.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:29 smithi187 ceph-mon[130973]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-08-11T14:18:30.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:29 smithi103 ceph-mon[130434]: pgmap v784: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 1.20k op/s 2023-08-11T14:18:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.821+0000 7f6c419ff700 1 -- 172.21.15.3:0/3753663251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 msgr2=0x7f6c3c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.821+0000 7f6c419ff700 1 --2- 172.21.15.3:0/3753663251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 0x7f6c3c10d0e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c01a9a0 tx=0x7f6c2c01acb0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.822+0000 7f6c419ff700 1 -- 172.21.15.3:0/3753663251 shutdown_connections 2023-08-11T14:18:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.822+0000 7f6c419ff700 1 --2- 172.21.15.3:0/3753663251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 0x7f6c3c10d0e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.822+0000 7f6c419ff700 1 --2- 172.21.15.3:0/3753663251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 0x7f6c3c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.822+0000 7f6c419ff700 1 --2- 172.21.15.3:0/3753663251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.822+0000 7f6c419ff700 1 -- 172.21.15.3:0/3753663251 >> 172.21.15.3:0/3753663251 conn(0x7f6c3c06d750 msgr2=0x7f6c3c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.832+0000 7f6c419ff700 1 -- 172.21.15.3:0/3753663251 shutdown_connections 2023-08-11T14:18:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.839+0000 7f6c419ff700 1 -- 172.21.15.3:0/3753663251 wait complete. 2023-08-11T14:18:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.840+0000 7f6c419ff700 1 Processor -- start 2023-08-11T14:18:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.851+0000 7f6c419ff700 1 -- start start 2023-08-11T14:18:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.851+0000 7f6c419ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 0x7f6c3c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.851+0000 7f6c419ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.851+0000 7f6c419ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 0x7f6c3c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.851+0000 7f6c419ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c19e790 con 0x7f6c3c10cc60 2023-08-11T14:18:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c419ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c19e900 con 0x7f6c3c10a290 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c419ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c19eac0 con 0x7f6c3c107920 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c3a7fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c3a7fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60914/0 (socket says 172.21.15.3:60914) 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1872288639 learned_addr learned my addr 172.21.15.3:0/1872288639 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.852+0000 7f6c3affd700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 0x7f6c3c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.853+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1872288639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 msgr2=0x7f6c3c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.853+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 0x7f6c3c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.853+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 msgr2=0x7f6c3c19e080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-11T14:18:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.853+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 0x7f6c3c19e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.854+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1872288639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c01a600 con 0x7f6c3c10a290 2023-08-11T14:18:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.854+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400ada0 tx=0x7f6c3400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.855+0000 7f6c409fd700 1 -- 172.21.15.3:0/1872288639 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400f070 con 0x7f6c3c10a290 2023-08-11T14:18:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.855+0000 7f6c409fd700 1 -- 172.21.15.3:0/1872288639 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c3400ae90 con 0x7f6c3c10a290 2023-08-11T14:18:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.855+0000 7f6c409fd700 1 -- 172.21.15.3:0/1872288639 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34005440 con 0x7f6c3c10a290 2023-08-11T14:18:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.862+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c19eda0 con 0x7f6c3c10a290 2023-08-11T14:18:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.862+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c1a54b0 con 0x7f6c3c10a290 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.872+0000 7f6c553bbb80 1 Processor -- start 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.874+0000 7f6c553bbb80 1 -- start start 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556322848a0 0x5556322851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632285c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556321dd5d0 con 0x555632285790 2023-08-11T14:18:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556321daef0 con 0x5556322848a0 2023-08-11T14:18:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.875+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556321d67e0 con 0x555632283f40 2023-08-11T14:18:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.880+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.880+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:40950/0 (socket says 172.21.15.3:40950) 2023-08-11T14:18:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.880+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/4116330125 learned_addr learned my addr 172.21.15.3:0/4116330125 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.885+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632285c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.886+0000 7f6c409fd700 1 -- 172.21.15.3:0/1872288639 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6c34004ad0 con 0x7f6c3c10a290 2023-08-11T14:18:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.888+0000 7f6c409fd700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 0x7f6c240a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.888+0000 7f6c409fd700 1 -- 172.21.15.3:0/1872288639 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c340ccc50 con 0x7f6c3c10a290 2023-08-11T14:18:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.889+0000 7f6c3affd700 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556322848a0 0x5556322851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.891+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/4116330125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556322848a0 msgr2=0x5556322851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.891+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556322848a0 0x5556322851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.891+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/4116330125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 msgr2=0x555632285c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.891+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632285c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.891+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/4116330125 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556320565d0 con 0x555632283f40 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.893+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 0x7f6c240a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.893+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c003820 tx=0x7f6c2c005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7937264e42c7fdf server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.904+0000 7f6c21ffb700 1 -- 172.21.15.3:0/4116330125 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c01cab0 con 0x555632283f40 2023-08-11T14:18:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.904+0000 7f6c21ffb700 1 -- 172.21.15.3:0/4116330125 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c2c02e070 con 0x555632283f40 2023-08-11T14:18:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.904+0000 7f6c21ffb700 1 -- 172.21.15.3:0/4116330125 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0326a0 con 0x555632283f40 2023-08-11T14:18:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 -- 172.21.15.3:0/4116330125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 msgr2=0x555632284360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c003820 tx=0x7f6c2c005c40 comp rx=0 tx=0).stop 2023-08-11T14:18:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 -- 172.21.15.3:0/4116330125 shutdown_connections 2023-08-11T14:18:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632285c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x555632284360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/4116330125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556322848a0 0x5556322851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.905+0000 7f6c553bbb80 1 -- 172.21.15.3:0/4116330125 >> 172.21.15.3:0/4116330125 conn(0x5556321b18f0 msgr2=0x555632280920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.908+0000 7f6c553bbb80 1 -- 172.21.15.3:0/4116330125 shutdown_connections 2023-08-11T14:18:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.912+0000 7f6c553bbb80 1 -- 172.21.15.3:0/4116330125 wait complete. 2023-08-11T14:18:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.913+0000 7f6c553bbb80 1 -- start start 2023-08-11T14:18:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.913+0000 7f6c553bbb80 1 Processor -- start 2023-08-11T14:18:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.921+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 0x7f6c240a2610 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c01a360 tx=0x7f6c2c005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x55563228db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 0x55563228e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632285790 0x555632295090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555632174a60 con 0x5556322848a0 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555632174760 con 0x555632285790 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556320588c0 con 0x555632283f40 2023-08-11T14:18:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.924+0000 7f6c553bbb80 10 client.0 _ll_register_callbacks cb 0x555632177a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.928+0000 7f6c3a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 0x55563228e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.929+0000 7f6c3a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 0x55563228e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42390/0 (socket says 172.21.15.3:42390) 2023-08-11T14:18:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.929+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/2828029422 learned_addr learned my addr 172.21.15.3:0/2828029422 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.929+0000 7f6c3affd700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x55563228db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.935+0000 7f6c3affd700 1 -- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632285790 msgr2=0x555632295090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.935+0000 7f6c3affd700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632285790 0x555632295090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.935+0000 7f6c3affd700 1 -- 172.21.15.3:0/2828029422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 msgr2=0x55563228e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.935+0000 7f6c3affd700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 0x55563228e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.935+0000 7f6c3affd700 1 -- 172.21.15.3:0/2828029422 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c10dc60 con 0x555632283f40 2023-08-11T14:18:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.939+0000 7f6c3affd700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x55563228db10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c19caf0 tx=0x7f6c30011540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45e61616f20d77b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.940+0000 7f6c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.940+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30015460 con 0x555632283f40 2023-08-11T14:18:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.940+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c300155c0 con 0x555632283f40 2023-08-11T14:18:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.940+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556322777e0 con 0x555632283f40 2023-08-11T14:18:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.941+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c30015780 con 0x555632283f40 2023-08-11T14:18:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.945+0000 7f6c553bbb80 10 client.26692 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.945+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x555632002d80 con 0x555632283f40 2023-08-11T14:18:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556322777e0 con 0x555632283f40 2023-08-11T14:18:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6c3001eb40 con 0x555632283f40 2023-08-11T14:18:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 10 client.26692 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 10 client.26692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 20 client.26692 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555632299ab0 0x55563229bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 4 client.26692 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:18:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.951+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555632253480 con 0x555632299ab0 2023-08-11T14:18:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.952+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555632299ab0 0x55563229bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.952+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555632299ab0 0x55563229bed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1787978aafbe48f8 server_cookie=64b5327255329a7f in_seq=0 out_seq=0 2023-08-11T14:18:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.952+0000 7f6c1affd700 10 client.26692 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.954+0000 7f6c1affd700 1 -- 172.21.15.3:0/2828029422 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555632253480 con 0x555632299ab0 2023-08-11T14:18:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.954+0000 7f6c1affd700 10 client.26692 handle_command_reply: tid=0 2023-08-11T14:18:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.954+0000 7f6c553bbb80 1 client.26692 shutdown 2023-08-11T14:18:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 msgr2=0x55563228db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x55563228db10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c19caf0 tx=0x7f6c30011540 comp rx=0 tx=0).stop 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 shutdown_connections 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555632299ab0 0x55563229bed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632285790 0x555632295090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555632283f40 0x55563228db10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2828029422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5556322848a0 0x55563228e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.965+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 >> 172.21.15.3:0/2828029422 conn(0x5556321b18f0 msgr2=0x55563227f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.966+0000 7f6c1affd700 0 client.26692 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.966+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 shutdown_connections 2023-08-11T14:18:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.966+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2828029422 wait complete. 2023-08-11T14:18:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:30.979+0000 7f6c553bbb80 20 client.26692 trim_cache size 0 max 16384 2023-08-11T14:18:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.023+0000 7f6c553bbb80 1 Processor -- start 2023-08-11T14:18:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.027+0000 7f6c553bbb80 1 -- start start 2023-08-11T14:18:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x5556322802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632295050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556322777c0 con 0x555632285790 2023-08-11T14:18:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556322a0cb0 con 0x555632283f40 2023-08-11T14:18:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.028+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555632279ea0 con 0x5556322848a0 2023-08-11T14:18:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.031+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.031+0000 7f6c3b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42396/0 (socket says 172.21.15.3:42396) 2023-08-11T14:18:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.031+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/2310977915 learned_addr learned my addr 172.21.15.3:0/2310977915 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.033+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/2310977915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 msgr2=0x555632295050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:18:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.033+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632295050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.033+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/2310977915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 msgr2=0x5556322802c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:18:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.033+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x5556322802c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.033+0000 7f6c3b7fe700 1 -- 172.21.15.3:0/2310977915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555631febc00 con 0x555632285790 2023-08-11T14:18:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.037+0000 7f6c3b7fe700 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c003820 tx=0x7f6c2c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d3bbefe0633909 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.047+0000 7f6c227fc700 1 -- 172.21.15.3:0/2310977915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c003dd0 con 0x555632285790 2023-08-11T14:18:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.047+0000 7f6c227fc700 1 -- 172.21.15.3:0/2310977915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c2c003f30 con 0x555632285790 2023-08-11T14:18:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.047+0000 7f6c227fc700 1 -- 172.21.15.3:0/2310977915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c032e60 con 0x555632285790 2023-08-11T14:18:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.048+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2310977915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 msgr2=0x555632280800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.048+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c003820 tx=0x7f6c2c032000 comp rx=0 tx=0).stop 2023-08-11T14:18:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.052+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2310977915 shutdown_connections 2023-08-11T14:18:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.052+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632280800 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.052+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x5556322802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.052+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/2310977915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632295050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.054 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.052+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2310977915 >> 172.21.15.3:0/2310977915 conn(0x5556321b18f0 msgr2=0x5556322886b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.078+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2310977915 shutdown_connections 2023-08-11T14:18:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.081+0000 7f6c553bbb80 1 -- 172.21.15.3:0/2310977915 wait complete. 2023-08-11T14:18:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.081+0000 7f6c553bbb80 1 -- start start 2023-08-11T14:18:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 Processor -- start 2023-08-11T14:18:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x555632290c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632291140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632294160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556322946a0 con 0x555632285790 2023-08-11T14:18:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555632294810 con 0x555632283f40 2023-08-11T14:18:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556322949d0 con 0x5556322848a0 2023-08-11T14:18:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.082+0000 7f6c553bbb80 10 client.0 _ll_register_callbacks cb 0x7f6c30009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.083+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x555632290c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.083+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x555632290c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60942/0 (socket says 172.21.15.3:60942) 2023-08-11T14:18:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.083+0000 7f6c3affd700 1 -- 172.21.15.3:0/1535668310 learned_addr learned my addr 172.21.15.3:0/1535668310 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.083+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632291140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.089+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 msgr2=0x555632290c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.089+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x555632290c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.089+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1535668310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 msgr2=0x555632294160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.089+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632294160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.089+0000 7f6c3a7fc700 1 -- 172.21.15.3:0/1535668310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556320565d0 con 0x5556322848a0 2023-08-11T14:18:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.091+0000 7f6c3a7fc700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632291140 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c340968b0 tx=0x7f6c340cdf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199ef82540ef4746 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.094+0000 7f6c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.094+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340192b0 con 0x5556322848a0 2023-08-11T14:18:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.094+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c3409aab0 con 0x5556322848a0 2023-08-11T14:18:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.094+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c2c02e070 con 0x5556322848a0 2023-08-11T14:18:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.095+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c3409a3b0 con 0x5556322848a0 2023-08-11T14:18:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.096+0000 7f6c553bbb80 10 client.26695 fetch_fsmap learned FSMap version 56 2023-08-11T14:18:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.096+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5556320291f0 con 0x5556322848a0 2023-08-11T14:18:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.096+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c340b6070 con 0x5556322848a0 2023-08-11T14:18:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.096+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 56) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6c340998a0 con 0x5556322848a0 2023-08-11T14:18:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 56 2023-08-11T14:18:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 20 client.26695 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5556322ee5a0 0x5556322f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 4 client.26695 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:18:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.097+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556321cf7c0 con 0x5556322ee5a0 2023-08-11T14:18:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.098+0000 7f6c3affd700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5556322ee5a0 0x5556322f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.106+0000 7f6c1affd700 10 client.26695 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.116+0000 7f6c3affd700 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5556322ee5a0 0x5556322f09c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9989dfa1f08c39 server_cookie=a8ddf39f63d5c503 in_seq=0 out_seq=0 2023-08-11T14:18:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.486+0000 7f6c1affd700 1 -- 172.21.15.3:0/1535668310 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556321cf7c0 con 0x5556322ee5a0 2023-08-11T14:18:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.486+0000 7f6c1affd700 10 client.26695 handle_command_reply: tid=0 2023-08-11T14:18:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.486+0000 7f6c553bbb80 1 client.26695 shutdown 2023-08-11T14:18:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.487+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 msgr2=0x555632291140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.487+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632291140 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c340968b0 tx=0x7f6c340cdf90 comp rx=0 tx=0).stop 2023-08-11T14:18:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 shutdown_connections 2023-08-11T14:18:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555632285790 0x555632294160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5556322ee5a0 0x5556322f09c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555632283f40 0x555632290c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 --2- 172.21.15.3:0/1535668310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556322848a0 0x555632291140 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 >> 172.21.15.3:0/1535668310 conn(0x5556321b18f0 msgr2=0x5556322801a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.492+0000 7f6c1affd700 0 client.26695 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.493+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 shutdown_connections 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.493+0000 7f6c553bbb80 1 -- 172.21.15.3:0/1535668310 wait complete. 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.495+0000 7f6c553bbb80 20 client.26695 trim_cache size 0 max 16384 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:18:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:18:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:18:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:18:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.508+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 msgr2=0x7f6c240a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.508+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 0x7f6c240a2610 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c01a360 tx=0x7f6c2c005c40 comp rx=0 tx=0).stop 2023-08-11T14:18:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.508+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 msgr2=0x7f6c3c19d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.508+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400ada0 tx=0x7f6c3400c330 comp rx=0 tx=0).stop 2023-08-11T14:18:31.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 shutdown_connections 2023-08-11T14:18:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c3c10cc60 0x7f6c3c19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c3c107920 0x7f6c3c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c240a0150 0x7f6c240a2610 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 --2- 172.21.15.3:0/1872288639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10a290 0x7f6c3c19d2e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:31.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.514+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 >> 172.21.15.3:0/1872288639 conn(0x7f6c3c06d750 msgr2=0x7f6c3c10ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.520+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 shutdown_connections 2023-08-11T14:18:31.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:31.520+0000 7f6c419ff700 1 -- 172.21.15.3:0/1872288639 wait complete. 2023-08-11T14:18:31.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:31 smithi003 ceph-mon[97195]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-08-11T14:18:31.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:31 smithi187 ceph-mon[130973]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-08-11T14:18:31.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:31 smithi103 ceph-mon[130434]: pgmap v785: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-08-11T14:18:33.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 695.0 s, 432.0 tps, lat 227.452 ms stddev 555.926 2023-08-11T14:18:34.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:34 smithi103 ceph-mon[130434]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-08-11T14:18:34.798 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:34 smithi187 ceph-mon[130973]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-08-11T14:18:34.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:34 smithi003 ceph-mon[97195]: pgmap v786: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.49k op/s 2023-08-11T14:18:35.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:35 smithi003 ceph-mon[97195]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 973 op/s 2023-08-11T14:18:35.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:35 smithi187 ceph-mon[130973]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 973 op/s 2023-08-11T14:18:35.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:35 smithi103 ceph-mon[130434]: pgmap v787: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 973 op/s 2023-08-11T14:18:37.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:37 smithi003 ceph-mon[97195]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:18:37.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:37 smithi187 ceph-mon[130973]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:18:37.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:37 smithi103 ceph-mon[130434]: pgmap v788: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:18:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 700.0 s, 685.1 tps, lat 146.672 ms stddev 312.911 2023-08-11T14:18:40.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:39 smithi003 ceph-mon[97195]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-08-11T14:18:40.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:39 smithi187 ceph-mon[130973]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-08-11T14:18:40.436 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:39 smithi103 ceph-mon[130434]: pgmap v789: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-08-11T14:18:41.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:41 smithi003 ceph-mon[97195]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 960 op/s 2023-08-11T14:18:41.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:41 smithi187 ceph-mon[130973]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 960 op/s 2023-08-11T14:18:41.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:41 smithi103 ceph-mon[130434]: pgmap v790: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 960 op/s 2023-08-11T14:18:43.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 705.0 s, 478.4 tps, lat 210.198 ms stddev 538.811 2023-08-11T14:18:43.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:43 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:43.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:43 smithi003 ceph-mon[97195]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.36k op/s 2023-08-11T14:18:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:43 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:43.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:43 smithi187 ceph-mon[130973]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.36k op/s 2023-08-11T14:18:43.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:43 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:43.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:43 smithi103 ceph-mon[130434]: pgmap v791: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 17 MiB/s wr, 1.36k op/s 2023-08-11T14:18:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 -- 172.21.15.3:0/99534292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814108b10 msgr2=0x7f381410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 --2- 172.21.15.3:0/99534292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814108b10 0x7f381410f3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3804009990 tx=0x7f3804009ca0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 -- 172.21.15.3:0/99534292 shutdown_connections 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 --2- 172.21.15.3:0/99534292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814108b10 0x7f381410f3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 --2- 172.21.15.3:0/99534292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814106f80 0x7f38141073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 --2- 172.21.15.3:0/99534292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f38141085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 -- 172.21.15.3:0/99534292 >> 172.21.15.3:0/99534292 conn(0x7f3814075f00 msgr2=0x7f3814078360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.815+0000 7f3818fff700 1 -- 172.21.15.3:0/99534292 shutdown_connections 2023-08-11T14:18:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.816+0000 7f3818fff700 1 -- 172.21.15.3:0/99534292 wait complete. 2023-08-11T14:18:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.817+0000 7f3818fff700 1 Processor -- start 2023-08-11T14:18:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.817+0000 7f3818fff700 1 -- start start 2023-08-11T14:18:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 0x7f381419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f381419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814108b10 0x7f381419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381419e740 con 0x7f3814108b10 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381419e8b0 con 0x7f3814108170 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3818fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381419ea70 con 0x7f3814106f80 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f381419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f381419d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55640/0 (socket says 172.21.15.3:55640) 2023-08-11T14:18:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3811d9b700 1 -- 172.21.15.3:0/2571912342 learned_addr learned my addr 172.21.15.3:0/2571912342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 0x7f381419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f3812d9d700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814108b10 0x7f381419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 -- 172.21.15.3:0/2571912342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 msgr2=0x7f381419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f381419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 -- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814108b10 msgr2=0x7f381419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814108b10 0x7f381419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.818+0000 7f381259c700 1 -- 172.21.15.3:0/2571912342 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040095f0 con 0x7f3814106f80 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f381259c700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 0x7f381419cd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f380c00eee0 tx=0x7f380c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f38037fe700 1 -- 172.21.15.3:0/2571912342 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c018070 con 0x7f3814106f80 2023-08-11T14:18:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f38037fe700 1 -- 172.21.15.3:0/2571912342 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f380c00f040 con 0x7f3814106f80 2023-08-11T14:18:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f38037fe700 1 -- 172.21.15.3:0/2571912342 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c004280 con 0x7f3814106f80 2023-08-11T14:18:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381419ecf0 con 0x7f3814106f80 2023-08-11T14:18:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.819+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141a5460 con 0x7f3814106f80 2023-08-11T14:18:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 Processor -- start 2023-08-11T14:18:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 -- start start 2023-08-11T14:18:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f38037fe700 1 -- 172.21.15.3:0/2571912342 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f380c00ca70 con 0x7f3814106f80 2023-08-11T14:18:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab198a0 0x55a08ab1a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.827+0000 7f382c909b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08aa725d0 con 0x55a08ab18f40 2023-08-11T14:18:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.828+0000 7f382c909b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08aa6fef0 con 0x55a08ab1a790 2023-08-11T14:18:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.828+0000 7f382c909b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08aa6b7e0 con 0x55a08ab198a0 2023-08-11T14:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55646/0 (socket says 172.21.15.3:55646) 2023-08-11T14:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 -- 172.21.15.3:0/3546356722 learned_addr learned my addr 172.21.15.3:0/3546356722 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 -- 172.21.15.3:0/3546356722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab198a0 msgr2=0x55a08ab1a1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:18:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab198a0 0x55a08ab1a1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 -- 172.21.15.3:0/3546356722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 msgr2=0x55a08ab19360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.829+0000 7f3811d9b700 1 -- 172.21.15.3:0/3546356722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a08a8eb5d0 con 0x55a08ab1a790 2023-08-11T14:18:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.830+0000 7f3811d9b700 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3808009e50 tx=0x7f3808011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6627acb1db846 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.831+0000 7f3800ff9700 1 -- 172.21.15.3:0/3546356722 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808004a30 con 0x55a08ab1a790 2023-08-11T14:18:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.831+0000 7f3800ff9700 1 -- 172.21.15.3:0/3546356722 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38080054e0 con 0x55a08ab1a790 2023-08-11T14:18:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.831+0000 7f3800ff9700 1 -- 172.21.15.3:0/3546356722 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808019930 con 0x55a08ab1a790 2023-08-11T14:18:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.831+0000 7f38037fe700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 0x7f37fc0a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.831+0000 7f38037fe700 1 -- 172.21.15.3:0/2571912342 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f380c0cb540 con 0x7f3814106f80 2023-08-11T14:18:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 -- 172.21.15.3:0/3546356722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 msgr2=0x55a08ab1ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3808009e50 tx=0x7f3808011720 comp rx=0 tx=0).stop 2023-08-11T14:18:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f3812d9d700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 0x7f37fc0a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 -- 172.21.15.3:0/3546356722 shutdown_connections 2023-08-11T14:18:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab1a790 0x55a08ab1ac10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab19360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 --2- 172.21.15.3:0/3546356722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab198a0 0x55a08ab1a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.833+0000 7f382c909b80 1 -- 172.21.15.3:0/3546356722 >> 172.21.15.3:0/3546356722 conn(0x55a08aa468f0 msgr2=0x55a08ab15920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.834+0000 7f382c909b80 1 -- 172.21.15.3:0/3546356722 shutdown_connections 2023-08-11T14:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.834+0000 7f382c909b80 1 -- 172.21.15.3:0/3546356722 wait complete. 2023-08-11T14:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.834+0000 7f3812d9d700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 0x7f37fc0a2510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38040098e0 tx=0x7f380400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.835+0000 7f382c909b80 1 -- start start 2023-08-11T14:18:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.835+0000 7f382c909b80 1 Processor -- start 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab33cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab3b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08aa09a60 con 0x55a08ab1a790 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08aa09760 con 0x55a08ab198a0 2023-08-11T14:18:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08a8ed8c0 con 0x55a08ab18f40 2023-08-11T14:18:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.836+0000 7f382c909b80 10 client.0 _ll_register_callbacks cb 0x55a08aa0ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f3811d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab34200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:55648/0 (socket says 172.21.15.3:55648) 2023-08-11T14:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f3811d9b700 1 -- 172.21.15.3:0/427939544 learned_addr learned my addr 172.21.15.3:0/427939544 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab33cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f3812d9d700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab3b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 -- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 msgr2=0x55a08ab34200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab34200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 -- 172.21.15.3:0/427939544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 msgr2=0x55a08ab3b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab3b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 -- 172.21.15.3:0/427939544 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808009a80 con 0x55a08ab18f40 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.837+0000 7f381259c700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab33cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f380c00c7a0 tx=0x7f380c0991e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be7060460dc2b86a server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f37f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0b80d0 con 0x55a08ab18f40 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f380c0993a0 con 0x55a08ab18f40 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a08ab0c7e0 con 0x55a08ab18f40 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f380c098600 con 0x55a08ab18f40 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f382c909b80 10 client.26704 fetch_fsmap learned FSMap version 57 2023-08-11T14:18:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.838+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55a08a897d80 con 0x55a08ab18f40 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a08ab0c7e0 con 0x55a08ab18f40 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f380c0a0590 con 0x55a08ab18f40 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 10 client.26704 fetch_fsmap finished waiting for FSMap version 57 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 10 client.26704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 20 client.26704 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab3fa30 0x55a08ab41e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 4 client.26704 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.839+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a08aae8480 con 0x55a08ab3fa30 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.840+0000 7f3811d9b700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab3fa30 0x55a08ab41e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.840+0000 7f37f9ffb700 10 client.26704 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.840+0000 7f3811d9b700 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab3fa30 0x55a08ab41e50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3fbd02b4258872 server_cookie=f78d7f76f7248d76 in_seq=0 out_seq=0 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.841+0000 7f37f9ffb700 1 -- 172.21.15.3:0/427939544 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a08aae8480 con 0x55a08ab3fa30 2023-08-11T14:18:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.841+0000 7f37f9ffb700 10 client.26704 handle_command_reply: tid=0 2023-08-11T14:18:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.841+0000 7f382c909b80 1 client.26704 shutdown 2023-08-11T14:18:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.846+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 msgr2=0x55a08ab33cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.846+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab33cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f380c00c7a0 tx=0x7f380c0991e0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 shutdown_connections 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab3b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab33cc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab3fa30 0x55a08ab41e50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 --2- 172.21.15.3:0/427939544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab34200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.847+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 >> 172.21.15.3:0/427939544 conn(0x55a08aa468f0 msgr2=0x55a08ab14710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.848+0000 7f37f9ffb700 0 client.26704 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.848+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 shutdown_connections 2023-08-11T14:18:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.848+0000 7f382c909b80 1 -- 172.21.15.3:0/427939544 wait complete. 2023-08-11T14:18:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.849+0000 7f382c909b80 20 client.26704 trim_cache size 0 max 16384 2023-08-11T14:18:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 Processor -- start 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 -- start start 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab3cf70 con 0x55a08ab18f40 2023-08-11T14:18:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab46c80 con 0x55a08ab198a0 2023-08-11T14:18:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.875+0000 7f382c909b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab20a50 con 0x55a08ab1a790 2023-08-11T14:18:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.876+0000 7f3812d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.876+0000 7f3812d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36752/0 (socket says 172.21.15.3:36752) 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.876+0000 7f3812d9d700 1 -- 172.21.15.3:0/3200789308 learned_addr learned my addr 172.21.15.3:0/3200789308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f381259c700 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 -- 172.21.15.3:0/3200789308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 msgr2=0x55a08ab3b200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 -- 172.21.15.3:0/3200789308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 msgr2=0x55a08ab338d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab338d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 -- 172.21.15.3:0/3200789308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a08a880c00 con 0x55a08ab1a790 2023-08-11T14:18:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.877+0000 7f3812d9d700 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f380400bda0 tx=0x7f380401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5daa24fa241c4696 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.878+0000 7f38017fa700 1 -- 172.21.15.3:0/3200789308 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804005650 con 0x55a08ab1a790 2023-08-11T14:18:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.878+0000 7f38017fa700 1 -- 172.21.15.3:0/3200789308 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f380402c040 con 0x55a08ab1a790 2023-08-11T14:18:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.878+0000 7f38017fa700 1 -- 172.21.15.3:0/3200789308 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380400f6c0 con 0x55a08ab1a790 2023-08-11T14:18:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.879+0000 7f382c909b80 1 -- 172.21.15.3:0/3200789308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 msgr2=0x55a08ab33e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.879+0000 7f382c909b80 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f380400bda0 tx=0x7f380401f000 comp rx=0 tx=0).stop 2023-08-11T14:18:44.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.880+0000 7f382c909b80 1 -- 172.21.15.3:0/3200789308 shutdown_connections 2023-08-11T14:18:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.880+0000 7f382c909b80 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab1a790 0x55a08ab33e10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.880+0000 7f382c909b80 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab18f40 0x55a08ab338d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.880+0000 7f382c909b80 1 --2- 172.21.15.3:0/3200789308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.880+0000 7f382c909b80 1 -- 172.21.15.3:0/3200789308 >> 172.21.15.3:0/3200789308 conn(0x55a08aa468f0 msgr2=0x55a08ab1d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.881+0000 7f382c909b80 1 -- 172.21.15.3:0/3200789308 shutdown_connections 2023-08-11T14:18:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.881+0000 7f382c909b80 1 -- 172.21.15.3:0/3200789308 wait complete. 2023-08-11T14:18:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.882+0000 7f382c909b80 1 -- start start 2023-08-11T14:18:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.882+0000 7f382c909b80 1 Processor -- start 2023-08-11T14:18:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab39780 con 0x55a08ab1a790 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab398f0 con 0x55a08ab198a0 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a08ab39ab0 con 0x55a08ab18f40 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f382c909b80 10 client.0 _ll_register_callbacks cb 0x55a08aa0ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f381259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f381259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36760/0 (socket says 172.21.15.3:36760) 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f381259c700 1 -- 172.21.15.3:0/2229300415 learned_addr learned my addr 172.21.15.3:0/2229300415 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.883+0000 7f3811d9b700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f3812d9d700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 -- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 msgr2=0x55a08ab3b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 -- 172.21.15.3:0/2229300415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 msgr2=0x55a08ab391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 -- 172.21.15.3:0/2229300415 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381410fd10 con 0x55a08ab18f40 2023-08-11T14:18:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f3812d9d700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:18:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f381259c700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f380c094a20 tx=0x7f380c009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ceed01891baea2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f37f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0b80d0 con 0x55a08ab18f40 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f380c09c8a0 con 0x55a08ab18f40 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38080054e0 con 0x55a08ab18f40 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f380c098600 con 0x55a08ab18f40 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f382c909b80 10 client.26710 fetch_fsmap learned FSMap version 57 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.884+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55a08a8be1f0 con 0x55a08ab18f40 2023-08-11T14:18:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38080054e0 con 0x55a08ab18f40 2023-08-11T14:18:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f380c077bc0 con 0x55a08ab18f40 2023-08-11T14:18:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 10 client.26710 fetch_fsmap finished waiting for FSMap version 57 2023-08-11T14:18:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 20 client.26710 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab94650 0x55a08ab96a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 4 client.26710 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.885+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a08aa647c0 con 0x55a08ab94650 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.886+0000 7f3811d9b700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab94650 0x55a08ab96a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.886+0000 7f3811d9b700 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab94650 0x55a08ab96a70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e94feec15861d7b server_cookie=75a590b9baecc4b0 in_seq=0 out_seq=0 2023-08-11T14:18:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:44.886+0000 7f37f9ffb700 10 client.26710 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.087+0000 7f37f9ffb700 1 -- 172.21.15.3:0/2229300415 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a08aa647c0 con 0x55a08ab94650 2023-08-11T14:18:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.087+0000 7f37f9ffb700 10 client.26710 handle_command_reply: tid=0 2023-08-11T14:18:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.088+0000 7f382c909b80 1 client.26710 shutdown 2023-08-11T14:18:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.088+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 msgr2=0x55a08ab3ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.088+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f380c094a20 tx=0x7f380c009350 comp rx=0 tx=0).stop 2023-08-11T14:18:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 shutdown_connections 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a08ab1a790 0x55a08ab391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08ab18f40 0x55a08ab3ab90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a08ab94650 0x55a08ab96a70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 --2- 172.21.15.3:0/2229300415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a08ab198a0 0x55a08ab3b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 >> 172.21.15.3:0/2229300415 conn(0x55a08aa468f0 msgr2=0x55a08ab338b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.089+0000 7f37f9ffb700 0 client.26710 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:45.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.096+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 shutdown_connections 2023-08-11T14:18:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.100+0000 7f382c909b80 1 -- 172.21.15.3:0/2229300415 wait complete. 2023-08-11T14:18:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.102+0000 7f382c909b80 20 client.26710 trim_cache size 0 max 16384 2023-08-11T14:18:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:18:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:18:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:18:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:18:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.109+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 msgr2=0x7f37fc0a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.109+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 0x7f37fc0a2510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38040098e0 tx=0x7f380400b260 comp rx=0 tx=0).stop 2023-08-11T14:18:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.109+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 msgr2=0x7f381419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.109+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 0x7f381419cd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f380c00eee0 tx=0x7f380c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 shutdown_connections 2023-08-11T14:18:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3814108b10 0x7f381419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3814106f80 0x7f381419cd50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f37fc0a0050 0x7f37fc0a2510 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 --2- 172.21.15.3:0/2571912342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3814108170 0x7f381419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 >> 172.21.15.3:0/2571912342 conn(0x7f3814075f00 msgr2=0x7f381410b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.112+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 shutdown_connections 2023-08-11T14:18:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:45.113+0000 7f3818fff700 1 -- 172.21.15.3:0/2571912342 wait complete. 2023-08-11T14:18:46.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:46 smithi103 ceph-mon[130434]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 896 op/s 2023-08-11T14:18:46.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:46 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:18:46.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:46 smithi003 ceph-mon[97195]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 896 op/s 2023-08-11T14:18:46.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:46 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:18:46.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:46 smithi187 ceph-mon[130973]: pgmap v792: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 896 op/s 2023-08-11T14:18:46.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:46 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:18:47.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:47 smithi187 ceph-mon[130973]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:18:47.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:47 smithi003 ceph-mon[97195]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:18:47.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:47 smithi103 ceph-mon[130434]: pgmap v793: 129 pgs: 129 active+clean; 8.4 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 1.03k op/s 2023-08-11T14:18:48.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 710.0 s, 788.6 tps, lat 112.553 ms stddev 146.245 2023-08-11T14:18:48.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:48 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:18:48.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:48 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:18:48.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:48 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:18:48.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:48 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:18:48.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:48 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:18:48.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:48 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:18:48.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:48 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:18:48.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:48 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:18:48.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:48 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:18:50.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:49 smithi003 ceph-mon[97195]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 957 op/s 2023-08-11T14:18:50.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:49 smithi187 ceph-mon[130973]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 957 op/s 2023-08-11T14:18:50.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:49 smithi103 ceph-mon[130434]: pgmap v794: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 957 op/s 2023-08-11T14:18:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:52 smithi003 ceph-mon[97195]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 822 op/s 2023-08-11T14:18:52.804 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:52 smithi187 ceph-mon[130973]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 822 op/s 2023-08-11T14:18:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:52 smithi103 ceph-mon[130434]: pgmap v795: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 822 op/s 2023-08-11T14:18:53.087 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 715.0 s, 259.4 tps, lat 426.056 ms stddev 626.942 2023-08-11T14:18:53.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:53 smithi003 ceph-mon[97195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:53.823 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:53 smithi003 ceph-mon[97195]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:18:53.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:53 smithi187 ceph-mon[130973]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:53.824 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:53 smithi187 ceph-mon[130973]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:18:53.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:53 smithi103 ceph-mon[130434]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-08-11T14:18:53.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:53 smithi103 ceph-mon[130434]: pgmap v796: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:18:56.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:56 smithi103 ceph-mon[130434]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 910 op/s 2023-08-11T14:18:56.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:56 smithi003 ceph-mon[97195]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 910 op/s 2023-08-11T14:18:56.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:56 smithi187 ceph-mon[130973]: pgmap v797: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 910 op/s 2023-08-11T14:18:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 720.0 s, 1062.7 tps, lat 95.180 ms stddev 73.341 2023-08-11T14:18:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 -- 172.21.15.3:0/2662113409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa17410a290 msgr2=0x7fa17410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 --2- 172.21.15.3:0/2662113409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa17410a290 0x7fa17410c720 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa16801a9a0 tx=0x7fa16801acb0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 -- 172.21.15.3:0/2662113409 shutdown_connections 2023-08-11T14:18:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 --2- 172.21.15.3:0/2662113409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 0x7fa17410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 --2- 172.21.15.3:0/2662113409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa174107920 0x7fa174109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 --2- 172.21.15.3:0/2662113409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa17410a290 0x7fa17410c720 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa16801a9a0 tx=0x7fa16801acb0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.420+0000 7fa179bff700 1 -- 172.21.15.3:0/2662113409 >> 172.21.15.3:0/2662113409 conn(0x7fa17406d750 msgr2=0x7fa17406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.427+0000 7fa179bff700 1 -- 172.21.15.3:0/2662113409 shutdown_connections 2023-08-11T14:18:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.427+0000 7fa179bff700 1 -- 172.21.15.3:0/2662113409 wait complete. 2023-08-11T14:18:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.429+0000 7fa179bff700 1 Processor -- start 2023-08-11T14:18:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.429+0000 7fa179bff700 1 -- start start 2023-08-11T14:18:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 0x7fa17419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17419e380 0x7fa17419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17419edd0 con 0x7fa17410cc60 2023-08-11T14:18:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17419ef40 con 0x7fa17419e380 2023-08-11T14:18:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa179bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17419f100 con 0x7fa174107920 2023-08-11T14:18:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa1737fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa1737fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33406/0 (socket says 172.21.15.3:33406) 2023-08-11T14:18:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa1737fe700 1 -- 172.21.15.3:0/3470231843 learned_addr learned my addr 172.21.15.3:0/3470231843 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.430+0000 7fa172ffd700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 0x7fa17419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 -- 172.21.15.3:0/3470231843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17419e380 msgr2=0x7fa17419e800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:18:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17419e380 0x7fa17419e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 -- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 msgr2=0x7fa17419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 0x7fa17419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 -- 172.21.15.3:0/3470231843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16801a600 con 0x7fa174107920 2023-08-11T14:18:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.431+0000 7fa1737fe700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d950 tx=0x7fa16400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.432+0000 7fa170ff9700 1 -- 172.21.15.3:0/3470231843 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164009940 con 0x7fa174107920 2023-08-11T14:18:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.433+0000 7fa170ff9700 1 -- 172.21.15.3:0/3470231843 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa164010460 con 0x7fa174107920 2023-08-11T14:18:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.433+0000 7fa170ff9700 1 -- 172.21.15.3:0/3470231843 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400f620 con 0x7fa174107920 2023-08-11T14:18:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.433+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1741a5630 con 0x7fa174107920 2023-08-11T14:18:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.433+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741a5b70 con 0x7fa174107920 2023-08-11T14:18:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.441+0000 7fa18d56db80 1 Processor -- start 2023-08-11T14:18:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 -- start start 2023-08-11T14:18:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa170ff9700 1 -- 172.21.15.3:0/3470231843 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fa1640105d0 con 0x7fa174107920 2023-08-11T14:18:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 0x55e16ae35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae36c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ad8e5d0 con 0x55e16ae358a0 2023-08-11T14:18:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ad8bef0 con 0x55e16ae36790 2023-08-11T14:18:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa18d56db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ad877e0 con 0x55e16ae34f40 2023-08-11T14:18:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa173fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 0x55e16ae35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa173fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 0x55e16ae35360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:33410/0 (socket says 172.21.15.3:33410) 2023-08-11T14:18:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa173fff700 1 -- 172.21.15.3:0/3795354033 learned_addr learned my addr 172.21.15.3:0/3795354033 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.442+0000 7fa172ffd700 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae36c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa1737fe700 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa172ffd700 1 -- 172.21.15.3:0/3795354033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 msgr2=0x55e16ae35360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa172ffd700 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 0x55e16ae35360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa172ffd700 1 -- 172.21.15.3:0/3795354033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 msgr2=0x55e16ae361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa172ffd700 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.443+0000 7fa172ffd700 1 -- 172.21.15.3:0/3795354033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16ac075d0 con 0x55e16ae36790 2023-08-11T14:18:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.444+0000 7fa172ffd700 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae36c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa168019000 tx=0x7fa168005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8396199a587f8708 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.444+0000 7fa159ffb700 1 -- 172.21.15.3:0/3795354033 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16802d050 con 0x55e16ae36790 2023-08-11T14:18:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.444+0000 7fa159ffb700 1 -- 172.21.15.3:0/3795354033 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16802e040 con 0x55e16ae36790 2023-08-11T14:18:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.444+0000 7fa159ffb700 1 -- 172.21.15.3:0/3795354033 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680048c0 con 0x55e16ae36790 2023-08-11T14:18:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.445+0000 7fa170ff9700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 0x7fa15c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.445+0000 7fa170ff9700 1 -- 172.21.15.3:0/3470231843 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa1640cb060 con 0x7fa174107920 2023-08-11T14:18:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.445+0000 7fa18d56db80 1 -- 172.21.15.3:0/3795354033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 msgr2=0x55e16ae36c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.445+0000 7fa18d56db80 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae36c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa168019000 tx=0x7fa168005fd0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa173fff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 0x7fa15c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa18d56db80 1 -- 172.21.15.3:0/3795354033 shutdown_connections 2023-08-11T14:18:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa18d56db80 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae36c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa18d56db80 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae34f40 0x55e16ae35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa18d56db80 1 --2- 172.21.15.3:0/3795354033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.446+0000 7fa18d56db80 1 -- 172.21.15.3:0/3795354033 >> 172.21.15.3:0/3795354033 conn(0x55e16ad628f0 msgr2=0x55e16ae31920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.447+0000 7fa18d56db80 1 -- 172.21.15.3:0/3795354033 shutdown_connections 2023-08-11T14:18:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.448+0000 7fa18d56db80 1 -- 172.21.15.3:0/3795354033 wait complete. 2023-08-11T14:18:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.449+0000 7fa18d56db80 1 -- start start 2023-08-11T14:18:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.449+0000 7fa18d56db80 1 Processor -- start 2023-08-11T14:18:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae34f40 0x55e16ae3eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae358a0 0x55e16ae3f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ad25a60 con 0x55e16ae34f40 2023-08-11T14:18:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ad25760 con 0x55e16ae36790 2023-08-11T14:18:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ac098c0 con 0x55e16ae358a0 2023-08-11T14:18:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.450+0000 7fa18d56db80 10 client.0 _ll_register_callbacks cb 0x55e16ad28a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.451+0000 7fa173fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.451+0000 7fa172ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae358a0 0x55e16ae3f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.451+0000 7fa173fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53588/0 (socket says 172.21.15.3:53588) 2023-08-11T14:18:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.451+0000 7fa173fff700 1 -- 172.21.15.3:0/4089541583 learned_addr learned my addr 172.21.15.3:0/4089541583 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.451+0000 7fa1737fe700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae34f40 0x55e16ae3eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 0x7fa15c0a24e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0071c0 tx=0x7fa16c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 -- 172.21.15.3:0/4089541583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae358a0 msgr2=0x55e16ae3f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae358a0 0x55e16ae3f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 -- 172.21.15.3:0/4089541583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae34f40 msgr2=0x55e16ae3eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae34f40 0x55e16ae3eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.452+0000 7fa173fff700 1 -- 172.21.15.3:0/4089541583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17410dc60 con 0x55e16ae36790 2023-08-11T14:18:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.453+0000 7fa173fff700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa16c017a60 tx=0x7fa16c004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d675de61cee1dacd server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.456+0000 7fa152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:18:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.456+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01c070 con 0x55e16ae36790 2023-08-11T14:18:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.456+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa16c006e40 con 0x55e16ae36790 2023-08-11T14:18:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.457+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e16ae287e0 con 0x55e16ae36790 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.457+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16c017d70 con 0x55e16ae36790 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.459+0000 7fa18d56db80 10 client.26714 fetch_fsmap learned FSMap version 58 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.459+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e16abb3d80 con 0x55e16ae36790 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.460+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e16ae287e0 con 0x55e16ae36790 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.460+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa16c0229c0 con 0x55e16ae36790 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.460+0000 7fa18d56db80 10 client.26714 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:18:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.460+0000 7fa18d56db80 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.461+0000 7fa18d56db80 20 client.26714 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.461+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae4a9f0 0x55e16ae4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.461+0000 7fa18d56db80 4 client.26714 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:18:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.461+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e16ae04480 con 0x55e16ae4a9f0 2023-08-11T14:18:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.461+0000 7fa1737fe700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae4a9f0 0x55e16ae4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.467+0000 7fa1737fe700 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae4a9f0 0x55e16ae4ce10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab01fe7cf3d043bc server_cookie=763ad3e5c771b401 in_seq=0 out_seq=0 2023-08-11T14:18:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.467+0000 7fa152ffd700 10 client.26714 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.468+0000 7fa152ffd700 1 -- 172.21.15.3:0/4089541583 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e16ae04480 con 0x55e16ae4a9f0 2023-08-11T14:18:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.468+0000 7fa152ffd700 10 client.26714 handle_command_reply: tid=0 2023-08-11T14:18:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.468+0000 7fa18d56db80 1 client.26714 shutdown 2023-08-11T14:18:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 msgr2=0x55e16ae46110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa16c017a60 tx=0x7fa16c004be0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 shutdown_connections 2023-08-11T14:18:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae4a9f0 0x55e16ae4ce10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae36790 0x55e16ae46110 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae34f40 0x55e16ae3eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 --2- 172.21.15.3:0/4089541583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae358a0 0x55e16ae3f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa152ffd700 0 client.26714 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 >> 172.21.15.3:0/4089541583 conn(0x55e16ad628f0 msgr2=0x55e16ae30710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.469+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 shutdown_connections 2023-08-11T14:18:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.470+0000 7fa18d56db80 1 -- 172.21.15.3:0/4089541583 wait complete. 2023-08-11T14:18:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.471+0000 7fa18d56db80 20 client.26714 trim_cache size 0 max 16384 2023-08-11T14:18:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.500+0000 7fa18d56db80 1 Processor -- start 2023-08-11T14:18:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.502+0000 7fa18d56db80 1 -- start start 2023-08-11T14:18:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae39650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae35fa0 con 0x55e16ae358a0 2023-08-11T14:18:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae3c8f0 con 0x55e16ae34f40 2023-08-11T14:18:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa18d56db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae3a1e0 con 0x55e16ae36790 2023-08-11T14:18:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa1737fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa1737fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:53604/0 (socket says 172.21.15.3:53604) 2023-08-11T14:18:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa1737fe700 1 -- 172.21.15.3:0/302886838 learned_addr learned my addr 172.21.15.3:0/302886838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa172ffd700 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.503+0000 7fa173fff700 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae39650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 -- 172.21.15.3:0/302886838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 msgr2=0x55e16ae39650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae39650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 -- 172.21.15.3:0/302886838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 msgr2=0x55e16ae460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 -- 172.21.15.3:0/302886838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16ab9cc00 con 0x55e16ae34f40 2023-08-11T14:18:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa1737fe700 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1640951a0 tx=0x7fa164083040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c046d2c3d4840f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.504+0000 7fa15a7fc700 1 -- 172.21.15.3:0/302886838 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164014030 con 0x55e16ae34f40 2023-08-11T14:18:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa15a7fc700 1 -- 172.21.15.3:0/302886838 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa164098090 con 0x55e16ae34f40 2023-08-11T14:18:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa15a7fc700 1 -- 172.21.15.3:0/302886838 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640a05b0 con 0x55e16ae34f40 2023-08-11T14:18:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 -- 172.21.15.3:0/302886838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 msgr2=0x55e16ae51770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa1640951a0 tx=0x7fa164083040 comp rx=0 tx=0).stop 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 -- 172.21.15.3:0/302886838 shutdown_connections 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae39650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae51770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 --2- 172.21.15.3:0/302886838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.505+0000 7fa18d56db80 1 -- 172.21.15.3:0/302886838 >> 172.21.15.3:0/302886838 conn(0x55e16ad628f0 msgr2=0x55e16ae3cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.506+0000 7fa18d56db80 1 -- 172.21.15.3:0/302886838 shutdown_connections 2023-08-11T14:18:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.506+0000 7fa18d56db80 1 -- 172.21.15.3:0/302886838 wait complete. 2023-08-11T14:18:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.507+0000 7fa18d56db80 1 -- start start 2023-08-11T14:18:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.507+0000 7fa18d56db80 1 Processor -- start 2023-08-11T14:18:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae45d50 con 0x55e16ae358a0 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae45ec0 con 0x55e16ae34f40 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ae46080 con 0x55e16ae36790 2023-08-11T14:18:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.508+0000 7fa18d56db80 10 client.0 _ll_register_callbacks cb 0x7fa16c017430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.509+0000 7fa172ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.509+0000 7fa172ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46808/0 (socket says 172.21.15.3:46808) 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.509+0000 7fa172ffd700 1 -- 172.21.15.3:0/852551022 learned_addr learned my addr 172.21.15.3:0/852551022 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa1737fe700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 -- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 msgr2=0x55e16ae422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 -- 172.21.15.3:0/852551022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 msgr2=0x55e16ae427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:18:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.510+0000 7fa173fff700 1 -- 172.21.15.3:0/852551022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741a5b70 con 0x55e16ae36790 2023-08-11T14:18:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.511+0000 7fa173fff700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae45810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f80 tx=0x7fa16c022b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c21c256c834be99 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:18:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.511+0000 7fa152ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:18:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.511+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01c070 con 0x55e16ae36790 2023-08-11T14:18:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.511+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fa16c02b8e0 con 0x55e16ae36790 2023-08-11T14:18:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.511+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa16802e040 con 0x55e16ae36790 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.513+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16c02b450 con 0x55e16ae36790 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.513+0000 7fa18d56db80 10 client.26728 fetch_fsmap learned FSMap version 58 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.513+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e16abda1f0 con 0x55e16ae36790 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16802e040 con 0x55e16ae36790 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa16c02c420 con 0x55e16ae36790 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 10 client.26728 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:18:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:18:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 20 client.26728 populate_metadata read hostname 'smithi003' 2023-08-11T14:18:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae9f9c0 0x55e16aea1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:18:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 4 client.26728 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:18:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e16ad807c0 con 0x55e16ae9f9c0 2023-08-11T14:18:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.514+0000 7fa1737fe700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae9f9c0 0x55e16aea1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:18:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.515+0000 7fa1737fe700 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae9f9c0 0x55e16aea1de0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddc86a83aaafcbe server_cookie=82c71cc1b147a61f in_seq=0 out_seq=0 2023-08-11T14:18:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.516+0000 7fa152ffd700 10 client.26728 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.534+0000 7fa152ffd700 1 -- 172.21.15.3:0/852551022 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e16ad807c0 con 0x55e16ae9f9c0 2023-08-11T14:18:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.534+0000 7fa152ffd700 10 client.26728 handle_command_reply: tid=0 2023-08-11T14:18:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.534+0000 7fa18d56db80 1 client.26728 shutdown 2023-08-11T14:18:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.535+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 msgr2=0x55e16ae45810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.535+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae45810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f80 tx=0x7fa16c022b90 comp rx=0 tx=0).stop 2023-08-11T14:18:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 shutdown_connections 2023-08-11T14:18:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e16ae36790 0x55e16ae45810 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e16ae9f9c0 0x55e16aea1de0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e16ae34f40 0x55e16ae422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 --2- 172.21.15.3:0/852551022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e16ae358a0 0x55e16ae427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.539+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 >> 172.21.15.3:0/852551022 conn(0x55e16ad628f0 msgr2=0x55e16ae39650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.540+0000 7fa152ffd700 0 client.26728 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:18:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.540+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 shutdown_connections 2023-08-11T14:18:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.540+0000 7fa18d56db80 1 -- 172.21.15.3:0/852551022 wait complete. 2023-08-11T14:18:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.541+0000 7fa18d56db80 20 client.26728 trim_cache size 0 max 16384 2023-08-11T14:18:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:18:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:18:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:18:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:18:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 msgr2=0x7fa15c0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 0x7fa15c0a24e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0071c0 tx=0x7fa16c0094d0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 msgr2=0x7fa17419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d950 tx=0x7fa16400dc60 comp rx=0 tx=0).stop 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 shutdown_connections 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fa15c0a0020 0x7fa15c0a24e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17419e380 0x7fa17419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa174107920 0x7fa17419cfc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 --2- 172.21.15.3:0/3470231843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa17410cc60 0x7fa17419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:18:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.543+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 >> 172.21.15.3:0/3470231843 conn(0x7fa17406d750 msgr2=0x7fa17410aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:18:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.546+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 shutdown_connections 2023-08-11T14:18:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:18:58.547+0000 7fa179bff700 1 -- 172.21.15.3:0/3470231843 wait complete. 2023-08-11T14:18:58.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:18:58 smithi187 ceph-mon[130973]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:18:58.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:18:58 smithi003 ceph-mon[97195]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:18:58.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:18:58 smithi103 ceph-mon[130434]: pgmap v798: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-08-11T14:19:00.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:00 smithi003 ceph-mon[97195]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 998 op/s 2023-08-11T14:19:00.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:00 smithi187 ceph-mon[130973]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 998 op/s 2023-08-11T14:19:00.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:00 smithi103 ceph-mon[130434]: pgmap v799: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 998 op/s 2023-08-11T14:19:02.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:02 smithi003 ceph-mon[97195]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:19:02.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:02 smithi187 ceph-mon[130973]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:19:02.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:02 smithi103 ceph-mon[130434]: pgmap v800: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 898 op/s 2023-08-11T14:19:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 725.0 s, 573.2 tps, lat 174.903 ms stddev 510.122 2023-08-11T14:19:04.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:04 smithi103 ceph-mon[130434]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-08-11T14:19:05.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:04 smithi003 ceph-mon[97195]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-08-11T14:19:05.052 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:04 smithi187 ceph-mon[130973]: pgmap v801: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 1.30k op/s 2023-08-11T14:19:06.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:05 smithi003 ceph-mon[97195]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-08-11T14:19:06.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:05 smithi187 ceph-mon[130973]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-08-11T14:19:06.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:05 smithi103 ceph-mon[130434]: pgmap v802: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-08-11T14:19:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 730.0 s, 654.8 tps, lat 148.055 ms stddev 338.358 2023-08-11T14:19:08.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:08 smithi003 ceph-mon[97195]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:19:08.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:08 smithi187 ceph-mon[130973]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:19:08.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:08 smithi103 ceph-mon[130434]: pgmap v803: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:19:10.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:10 smithi103 ceph-mon[130434]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:19:10.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:10 smithi003 ceph-mon[97195]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:19:10.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:10 smithi187 ceph-mon[130973]: pgmap v804: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 1.32k op/s 2023-08-11T14:19:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.833+0000 7f2e60bff700 1 -- 172.21.15.3:0/3724036212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c078070 msgr2=0x7f2e5c07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.833+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3724036212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c078070 0x7f2e5c07a500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400b0a0 tx=0x7f2e5400b3b0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.840+0000 7f2e60bff700 1 -- 172.21.15.3:0/3724036212 shutdown_connections 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.840+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3724036212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c078070 0x7f2e5c07a500 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.840+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3724036212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e5c0727c0 0x7f2e5c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.840+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3724036212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c077b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.840+0000 7f2e60bff700 1 -- 172.21.15.3:0/3724036212 >> 172.21.15.3:0/3724036212 conn(0x7f2e5c06d750 msgr2=0x7f2e5c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.841+0000 7f2e60bff700 1 -- 172.21.15.3:0/3724036212 shutdown_connections 2023-08-11T14:19:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.849+0000 7f2e60bff700 1 -- 172.21.15.3:0/3724036212 wait complete. 2023-08-11T14:19:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.850+0000 7f2e60bff700 1 Processor -- start 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 -- start start 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c0727c0 0x7f2e5c1b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e5c078070 0x7f2e5c1b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1b59e0 con 0x7f2e5c078070 2023-08-11T14:19:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1b5b50 con 0x7f2e5c0756a0 2023-08-11T14:19:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.851+0000 7f2e60bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1b5d10 con 0x7f2e5c0727c0 2023-08-11T14:19:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41868/0 (socket says 172.21.15.3:41868) 2023-08-11T14:19:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 -- 172.21.15.3:0/3317848948 learned_addr learned my addr 172.21.15.3:0/3317848948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 -- 172.21.15.3:0/3317848948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c0727c0 msgr2=0x7f2e5c1b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c0727c0 0x7f2e5c1b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 -- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e5c078070 msgr2=0x7f2e5c1b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e5c078070 0x7f2e5c1b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.852+0000 7f2e59d9b700 1 -- 172.21.15.3:0/3317848948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54009d00 con 0x7f2e5c0756a0 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000e9a0 tx=0x7f2e5000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e4b7fe700 1 -- 172.21.15.3:0/3317848948 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000cbd0 con 0x7f2e5c0756a0 2023-08-11T14:19:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e4b7fe700 1 -- 172.21.15.3:0/3317848948 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e5000cd30 con 0x7f2e5c0756a0 2023-08-11T14:19:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e4b7fe700 1 -- 172.21.15.3:0/3317848948 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50018680 con 0x7f2e5c0756a0 2023-08-11T14:19:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c1b5f90 con 0x7f2e5c0756a0 2023-08-11T14:19:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.853+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c1b64d0 con 0x7f2e5c0756a0 2023-08-11T14:19:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.861+0000 7f2e744bfb80 1 Processor -- start 2023-08-11T14:19:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.861+0000 7f2e744bfb80 1 -- start start 2023-08-11T14:19:11.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.861+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.861+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4035d0 con 0x55853c4aa8a0 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c400ef0 con 0x55853c4a9f40 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c3fc7e0 con 0x55853c4ab790 2023-08-11T14:19:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e5ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e5ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41882/0 (socket says 172.21.15.3:41882) 2023-08-11T14:19:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.862+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1144027880 learned_addr learned my addr 172.21.15.3:0/1144027880 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.863+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.864+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1144027880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 msgr2=0x55853c4abc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.864+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4abc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.864+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1144027880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 msgr2=0x55853c4ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.864+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.864+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1144027880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853c27c5d0 con 0x55853c4a9f40 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.865+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e540062a0 tx=0x7f2e540094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f3471d09b91ed9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.867+0000 7f2e48ff9700 1 -- 172.21.15.3:0/1144027880 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400e040 con 0x55853c4a9f40 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.867+0000 7f2e48ff9700 1 -- 172.21.15.3:0/1144027880 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e54004a50 con 0x55853c4a9f40 2023-08-11T14:19:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.867+0000 7f2e48ff9700 1 -- 172.21.15.3:0/1144027880 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5401d470 con 0x55853c4a9f40 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.868+0000 7f2e4b7fe700 1 -- 172.21.15.3:0/3317848948 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f2e50005190 con 0x7f2e5c0756a0 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.868+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1144027880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 msgr2=0x55853c4aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.868+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e540062a0 tx=0x7f2e540094b0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1144027880 shutdown_connections 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4abc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4aa360 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1144027880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1144027880 >> 172.21.15.3:0/1144027880 conn(0x55853c3d78f0 msgr2=0x55853c4a6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1144027880 shutdown_connections 2023-08-11T14:19:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.869+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1144027880 wait complete. 2023-08-11T14:19:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.870+0000 7f2e744bfb80 1 -- start start 2023-08-11T14:19:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.871+0000 7f2e744bfb80 1 Processor -- start 2023-08-11T14:19:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.874+0000 7f2e4b7fe700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 0x7f2e440a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.874+0000 7f2e4b7fe700 1 -- 172.21.15.3:0/3317848948 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2e50014070 con 0x7f2e5c0756a0 2023-08-11T14:19:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4bb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c39aa60 con 0x55853c4aa8a0 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c39a760 con 0x55853c4a9f40 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c27e8c0 con 0x55853c4ab790 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e744bfb80 10 client.0 _ll_register_callbacks cb 0x55853c39da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e5a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e5a59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41890/0 (socket says 172.21.15.3:41890) 2023-08-11T14:19:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.876+0000 7f2e5a59c700 1 -- 172.21.15.3:0/2768162907 learned_addr learned my addr 172.21.15.3:0/2768162907 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.877+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4bb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.878+0000 7f2e5a59c700 1 -- 172.21.15.3:0/2768162907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 msgr2=0x55853c4bb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.878+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4bb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.878+0000 7f2e5a59c700 1 -- 172.21.15.3:0/2768162907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 msgr2=0x55853c4b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.878+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.878+0000 7f2e5a59c700 1 -- 172.21.15.3:0/2768162907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c07af30 con 0x55853c4a9f40 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.881+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009eb0 tx=0x7f2e4c00b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e9ae5ed7ca192f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.881+0000 7f2e41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:19:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.885+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00be60 con 0x55853c4a9f40 2023-08-11T14:19:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.885+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2e4c011420 con 0x55853c4a9f40 2023-08-11T14:19:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.888+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55853c49d7e0 con 0x55853c4a9f40 2023-08-11T14:19:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.889+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e4c0115e0 con 0x55853c4a9f40 2023-08-11T14:19:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.891+0000 7f2e744bfb80 10 client.26732 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.891+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55853c228d80 con 0x55853c4a9f40 2023-08-11T14:19:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55853c49d7e0 con 0x55853c4a9f40 2023-08-11T14:19:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2e4c01a9a0 con 0x55853c4a9f40 2023-08-11T14:19:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 0x7f2e440a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 10 client.26732 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 20 client.26732 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c4bfb10 0x55853c4c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 4 client.26732 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:19:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.894+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55853c479480 con 0x55853c4bfb10 2023-08-11T14:19:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.896+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c4bfb10 0x55853c4c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.897+0000 7f2e41ffb700 10 client.26732 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.898+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c4bfb10 0x55853c4c1f30 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e06ac641827b031 server_cookie=f5bfbdcea7915767 in_seq=0 out_seq=0 2023-08-11T14:19:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.901+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 0x7f2e440a2530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007b60 tx=0x7f2e54006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.907+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2768162907 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55853c479480 con 0x55853c4bfb10 2023-08-11T14:19:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.907+0000 7f2e41ffb700 10 client.26732 handle_command_reply: tid=0 2023-08-11T14:19:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.907+0000 7f2e744bfb80 1 client.26732 shutdown 2023-08-11T14:19:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 msgr2=0x55853c4b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009eb0 tx=0x7f2e4c00b3c0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 shutdown_connections 2023-08-11T14:19:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4ab790 0x55853c4bb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c4bfb10 0x55853c4c1f30 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4a9f40 0x55853c4b3be0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2768162907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4aa8a0 0x55853c4b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 >> 172.21.15.3:0/2768162907 conn(0x55853c3d78f0 msgr2=0x55853c4a5710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e41ffb700 0 client.26732 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.913+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 shutdown_connections 2023-08-11T14:19:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.914+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2768162907 wait complete. 2023-08-11T14:19:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.925+0000 7f2e744bfb80 20 client.26732 trim_cache size 0 max 16384 2023-08-11T14:19:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.955+0000 7f2e744bfb80 1 Processor -- start 2023-08-11T14:19:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.955+0000 7f2e744bfb80 1 -- start start 2023-08-11T14:19:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.955+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.955+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4b2380 con 0x55853c4a9f40 2023-08-11T14:19:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4aac80 con 0x55853c4ab790 2023-08-11T14:19:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4aafa0 con 0x55853c4aa8a0 2023-08-11T14:19:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e59d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e59d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4bb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51132/0 (socket says 172.21.15.3:51132) 2023-08-11T14:19:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e59d9b700 1 -- 172.21.15.3:0/1451745292 learned_addr learned my addr 172.21.15.3:0/1451745292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.956+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1451745292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 msgr2=0x55853c4bb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4bb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1451745292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 msgr2=0x55853c4a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.957+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/1451745292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853c211c00 con 0x55853c4ab790 2023-08-11T14:19:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.958+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4a6720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ca0 tx=0x7f2e54026000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b243647003fbbab server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.958+0000 7f2e497fa700 1 -- 172.21.15.3:0/1451745292 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400e040 con 0x55853c4ab790 2023-08-11T14:19:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.958+0000 7f2e497fa700 1 -- 172.21.15.3:0/1451745292 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e54004020 con 0x55853c4ab790 2023-08-11T14:19:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.958+0000 7f2e497fa700 1 -- 172.21.15.3:0/1451745292 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5401d6d0 con 0x55853c4ab790 2023-08-11T14:19:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1451745292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 msgr2=0x55853c4a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4a6720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009ca0 tx=0x7f2e54026000 comp rx=0 tx=0).stop 2023-08-11T14:19:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1451745292 shutdown_connections 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4a6720 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/1451745292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1451745292 >> 172.21.15.3:0/1451745292 conn(0x55853c3d78f0 msgr2=0x55853c4ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1451745292 shutdown_connections 2023-08-11T14:19:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.960+0000 7f2e744bfb80 1 -- 172.21.15.3:0/1451745292 wait complete. 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.961+0000 7f2e744bfb80 1 -- start start 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.961+0000 7f2e744bfb80 1 Processor -- start 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4ba740 con 0x55853c4a9f40 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4ba8b0 con 0x55853c4ab790 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853c4baa70 con 0x55853c4aa8a0 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.962+0000 7f2e744bfb80 10 client.0 _ll_register_callbacks cb 0x7f2e4c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.963+0000 7f2e5ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.963+0000 7f2e5ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:41914/0 (socket says 172.21.15.3:41914) 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.963+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/2828859936 learned_addr learned my addr 172.21.15.3:0/2828859936 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.963+0000 7f2e59d9b700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.963+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/2828859936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 msgr2=0x55853c4b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/2828859936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 msgr2=0x55853c4b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 -- 172.21.15.3:0/2828859936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55853c27c5d0 con 0x55853c4ab790 2023-08-11T14:19:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e5ad9d700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400b070 tx=0x7f2e540077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b3a9184d7a34a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.964+0000 7f2e41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400e040 con 0x55853c4ab790 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f2e5402d040 con 0x55853c4ab790 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e54004a50 con 0x55853c4ab790 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e54009490 con 0x55853c4ab790 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e744bfb80 10 client.26738 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.965+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55853c24f1f0 con 0x55853c4ab790 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e54004a50 con 0x55853c4ab790 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2e54041070 con 0x55853c4ab790 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 10 client.26738 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 20 client.26738 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c5140f0 0x55853c516510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 4 client.26738 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:19:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.966+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55853c3f57c0 con 0x55853c5140f0 2023-08-11T14:19:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.976+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c5140f0 0x55853c516510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.978+0000 7f2e5a59c700 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c5140f0 0x55853c516510 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922441b6cc1e0559 server_cookie=8f4ecb0bbeaff25d in_seq=0 out_seq=0 2023-08-11T14:19:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:11.978+0000 7f2e41ffb700 10 client.26738 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.001+0000 7f2e41ffb700 1 -- 172.21.15.3:0/2828859936 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55853c3f57c0 con 0x55853c5140f0 2023-08-11T14:19:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.001+0000 7f2e41ffb700 10 client.26738 handle_command_reply: tid=0 2023-08-11T14:19:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.001+0000 7f2e744bfb80 1 client.26738 shutdown 2023-08-11T14:19:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.001+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 msgr2=0x55853c4ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.001+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400b070 tx=0x7f2e540077f0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 shutdown_connections 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55853c5140f0 0x55853c516510 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55853c4ab790 0x55853c4ba200 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853c4a9f40 0x55853c4b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 --2- 172.21.15.3:0/2828859936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55853c4aa8a0 0x55853c4b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e41ffb700 0 client.26738 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 >> 172.21.15.3:0/2828859936 conn(0x55853c3d78f0 msgr2=0x55853c4a61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 shutdown_connections 2023-08-11T14:19:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.002+0000 7f2e744bfb80 1 -- 172.21.15.3:0/2828859936 wait complete. 2023-08-11T14:19:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.005+0000 7f2e744bfb80 20 client.26738 trim_cache size 0 max 16384 2023-08-11T14:19:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:19:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:19:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:19:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:19:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 msgr2=0x7f2e440a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 0x7f2e440a2530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007b60 tx=0x7f2e54006210 comp rx=0 tx=0).stop 2023-08-11T14:19:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 msgr2=0x7f2e5c1b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000e9a0 tx=0x7f2e5000ed60 comp rx=0 tx=0).stop 2023-08-11T14:19:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 shutdown_connections 2023-08-11T14:19:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e5c078070 0x7f2e5c1b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c0727c0 0x7f2e5c1b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f2e440a0070 0x7f2e440a2530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 --2- 172.21.15.3:0/3317848948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c0756a0 0x7f2e5c1b2ba0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.006+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 >> 172.21.15.3:0/3317848948 conn(0x7f2e5c06d750 msgr2=0x7f2e5c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.007+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 shutdown_connections 2023-08-11T14:19:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:12.007+0000 7f2e60bff700 1 -- 172.21.15.3:0/3317848948 wait complete. 2023-08-11T14:19:12.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:12 smithi187 ceph-mon[130973]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.21k op/s 2023-08-11T14:19:12.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:12 smithi003 ceph-mon[97195]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.21k op/s 2023-08-11T14:19:12.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:12 smithi103 ceph-mon[130434]: pgmap v805: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 1.21k op/s 2023-08-11T14:19:13.088 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 735.0 s, 491.9 tps, lat 206.484 ms stddev 366.662 2023-08-11T14:19:14.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:14 smithi003 ceph-mon[97195]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.51k op/s 2023-08-11T14:19:14.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:14 smithi187 ceph-mon[130973]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.51k op/s 2023-08-11T14:19:14.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:14 smithi103 ceph-mon[130434]: pgmap v806: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.51k op/s 2023-08-11T14:19:16.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:16 smithi003 ceph-mon[97195]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:19:16.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:16 smithi187 ceph-mon[130973]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:19:16.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:16 smithi103 ceph-mon[130434]: pgmap v807: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-08-11T14:19:17.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:17 smithi103 ceph-mon[130434]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:19:18.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:17 smithi003 ceph-mon[97195]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:19:18.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:17 smithi187 ceph-mon[130973]: pgmap v808: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 1.11k op/s 2023-08-11T14:19:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 740.0 s, 869.8 tps, lat 86.382 ms stddev 84.036 2023-08-11T14:19:20.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:19 smithi003 ceph-mon[97195]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:19:20.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:19 smithi187 ceph-mon[130973]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:19:20.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:19 smithi103 ceph-mon[130434]: pgmap v809: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.22k op/s 2023-08-11T14:19:22.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:22 smithi003 ceph-mon[97195]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:19:22.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:22 smithi187 ceph-mon[130973]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:19:22.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:22 smithi103 ceph-mon[130434]: pgmap v810: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:19:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 745.0 s, 876.2 tps, lat 142.892 ms stddev 325.983 2023-08-11T14:19:23.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:23 smithi103 ceph-mon[130434]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-08-11T14:19:24.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:23 smithi187 ceph-mon[130973]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-08-11T14:19:24.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:23 smithi003 ceph-mon[97195]: pgmap v811: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 1.30k op/s 2023-08-11T14:19:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.356+0000 7f61acdff700 1 -- 172.21.15.3:0/996526625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8107920 msgr2=0x7f61a8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.356+0000 7f61acdff700 1 --2- 172.21.15.3:0/996526625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8107920 0x7f61a8109d50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f619800b0a0 tx=0x7f619800b3b0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.365+0000 7f61acdff700 1 -- 172.21.15.3:0/996526625 shutdown_connections 2023-08-11T14:19:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.365+0000 7f61acdff700 1 --2- 172.21.15.3:0/996526625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a810cc60 0x7f61a810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.365+0000 7f61acdff700 1 --2- 172.21.15.3:0/996526625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a8107920 0x7f61a8109d50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.365+0000 7f61acdff700 1 --2- 172.21.15.3:0/996526625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61a810a290 0x7f61a810c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.365+0000 7f61acdff700 1 -- 172.21.15.3:0/996526625 >> 172.21.15.3:0/996526625 conn(0x7f61a806d750 msgr2=0x7f61a806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.375+0000 7f61acdff700 1 -- 172.21.15.3:0/996526625 shutdown_connections 2023-08-11T14:19:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.376+0000 7f61acdff700 1 -- 172.21.15.3:0/996526625 wait complete. 2023-08-11T14:19:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.377+0000 7f61acdff700 1 Processor -- start 2023-08-11T14:19:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.378+0000 7f61acdff700 1 -- start start 2023-08-11T14:19:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61a8107920 0x7f61a8118120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a810a290 0x7f61a8118660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a811fea0 con 0x7f61a810cc60 2023-08-11T14:19:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8120010 con 0x7f61a810a290 2023-08-11T14:19:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.379+0000 7f61acdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81201d0 con 0x7f61a8107920 2023-08-11T14:19:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60564/0 (socket says 172.21.15.3:60564) 2023-08-11T14:19:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 -- 172.21.15.3:0/2105778547 learned_addr learned my addr 172.21.15.3:0/2105778547 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 -- 172.21.15.3:0/2105778547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61a8107920 msgr2=0x7f61a8118120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61a8107920 0x7f61a8118120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 -- 172.21.15.3:0/2105778547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a810a290 msgr2=0x7f61a8118660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a810a290 0x7f61a8118660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.380+0000 7f61a6d9d700 1 -- 172.21.15.3:0/2105778547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198009d00 con 0x7f61a810cc60 2023-08-11T14:19:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.381+0000 7f61a6d9d700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b4a0 tx=0x7f61a000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.390+0000 7f619ffff700 1 -- 172.21.15.3:0/2105778547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011830 con 0x7f61a810cc60 2023-08-11T14:19:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.390+0000 7f619ffff700 1 -- 172.21.15.3:0/2105778547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61a0011e50 con 0x7f61a810cc60 2023-08-11T14:19:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.390+0000 7f619ffff700 1 -- 172.21.15.3:0/2105778547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0010e70 con 0x7f61a810cc60 2023-08-11T14:19:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.391+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a81204b0 con 0x7f61a810cc60 2023-08-11T14:19:25.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.391+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a81209f0 con 0x7f61a810cc60 2023-08-11T14:19:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.397+0000 7f61c07c0b80 1 Processor -- start 2023-08-11T14:19:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.409+0000 7f61c07c0b80 1 -- start start 2023-08-11T14:19:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de7f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de74a5d0 con 0x5609de7f18a0 2023-08-11T14:19:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de747ef0 con 0x5609de7f0f40 2023-08-11T14:19:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de7437e0 con 0x5609de7f2790 2023-08-11T14:19:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f619dd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f619dd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:36828/0 (socket says 172.21.15.3:36828) 2023-08-11T14:19:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.410+0000 7f619dd9b700 1 -- 172.21.15.3:0/331479785 learned_addr learned my addr 172.21.15.3:0/331479785 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.415+0000 7f619ffff700 1 -- 172.21.15.3:0/2105778547 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f61a0010510 con 0x7f61a810cc60 2023-08-11T14:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.417+0000 7f619ffff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 0x7f61940a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619ffff700 1 -- 172.21.15.3:0/2105778547 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f61a00cad00 con 0x7f61a810cc60 2023-08-11T14:19:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f61a659c700 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de7f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619dd9b700 1 -- 172.21.15.3:0/331479785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 msgr2=0x5609de7f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619dd9b700 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619dd9b700 1 -- 172.21.15.3:0/331479785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 msgr2=0x5609de7f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619dd9b700 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de7f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.418+0000 7f619dd9b700 1 -- 172.21.15.3:0/331479785 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609de5c35d0 con 0x5609de7f2790 2023-08-11T14:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.420+0000 7f619dd9b700 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6190001850 tx=0x7f6190001b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=919ee6dee8153a60 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.423+0000 7f61a6d9d700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 0x7f61940a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.426+0000 7f61a6d9d700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 0x7f61940a2610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f61a0098360 tx=0x7f61a0094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.429+0000 7f619cd99700 1 -- 172.21.15.3:0/331479785 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190004c70 con 0x5609de7f2790 2023-08-11T14:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.429+0000 7f619cd99700 1 -- 172.21.15.3:0/331479785 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6190004dd0 con 0x5609de7f2790 2023-08-11T14:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.429+0000 7f619cd99700 1 -- 172.21.15.3:0/331479785 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000e8b0 con 0x5609de7f2790 2023-08-11T14:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.430+0000 7f61c07c0b80 1 -- 172.21.15.3:0/331479785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 msgr2=0x5609de7f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.430+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6190001850 tx=0x7f6190001b60 comp rx=0 tx=0).stop 2023-08-11T14:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.439+0000 7f61c07c0b80 1 -- 172.21.15.3:0/331479785 shutdown_connections 2023-08-11T14:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.439+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f2c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.439+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.439+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/331479785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de7f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.439+0000 7f61c07c0b80 1 -- 172.21.15.3:0/331479785 >> 172.21.15.3:0/331479785 conn(0x5609de71e8f0 msgr2=0x5609de7ed920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.455+0000 7f61c07c0b80 1 -- 172.21.15.3:0/331479785 shutdown_connections 2023-08-11T14:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.465+0000 7f61c07c0b80 1 -- 172.21.15.3:0/331479785 wait complete. 2023-08-11T14:19:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.466+0000 7f61c07c0b80 1 -- start start 2023-08-11T14:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.467+0000 7f61c07c0b80 1 Processor -- start 2023-08-11T14:19:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.467+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.467+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 0x5609de7fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f2790 0x5609de802240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de6e1a60 con 0x5609de7f2790 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de6e1760 con 0x5609de7f0f40 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de5c58c0 con 0x5609de7f18a0 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f61c07c0b80 10 client.0 _ll_register_callbacks cb 0x5609de6e4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f619dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f619dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7facc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60070/0 (socket says 172.21.15.3:60070) 2023-08-11T14:19:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.468+0000 7f619dd9b700 1 -- 172.21.15.3:0/2677687343 learned_addr learned my addr 172.21.15.3:0/2677687343 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.473+0000 7f61a659c700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 0x5609de7fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f2790 0x5609de802240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f61a659c700 1 -- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 msgr2=0x5609de7facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f61a659c700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f61a659c700 1 -- 172.21.15.3:0/2677687343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f2790 msgr2=0x5609de802240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f61a659c700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f2790 0x5609de802240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.476+0000 7f61a659c700 1 -- 172.21.15.3:0/2677687343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6190001500 con 0x5609de7f18a0 2023-08-11T14:19:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.496+0000 7f61a659c700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 0x5609de7fb200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6198009fd0 tx=0x7f6198003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e026eb5235aeb3c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.498+0000 7f618dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:19:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.500+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800e070 con 0x5609de7f18a0 2023-08-11T14:19:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.500+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f61980077f0 con 0x5609de7f18a0 2023-08-11T14:19:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.505+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609de7e47e0 con 0x5609de7f18a0 2023-08-11T14:19:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.517+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61980079b0 con 0x5609de7f18a0 2023-08-11T14:19:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.525+0000 7f61c07c0b80 10 client.26749 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.526+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5609de56fd80 con 0x5609de7f18a0 2023-08-11T14:19:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.537+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609de7e47e0 con 0x5609de7f18a0 2023-08-11T14:19:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.537+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f619800b0d0 con 0x5609de7f18a0 2023-08-11T14:19:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 10 client.26749 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 10 client.26749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 20 client.26749 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de806b30 0x5609de808f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 4 client.26749 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:19:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.538+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609de7c0480 con 0x5609de806b30 2023-08-11T14:19:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.549+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de806b30 0x5609de808f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.575+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de806b30 0x5609de808f50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f2d0c7a50a2c1f server_cookie=1e39f02d62a7164f in_seq=0 out_seq=0 2023-08-11T14:19:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.578+0000 7f618dffb700 10 client.26749 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.591+0000 7f618dffb700 1 -- 172.21.15.3:0/2677687343 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5609de7c0480 con 0x5609de806b30 2023-08-11T14:19:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.591+0000 7f618dffb700 10 client.26749 handle_command_reply: tid=0 2023-08-11T14:19:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.591+0000 7f61c07c0b80 1 client.26749 shutdown 2023-08-11T14:19:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.600+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 msgr2=0x5609de7fb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.600+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 0x5609de7fb200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6198009fd0 tx=0x7f6198003ee0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.606+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 shutdown_connections 2023-08-11T14:19:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.606+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f2790 0x5609de802240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.606+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de806b30 0x5609de808f50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.607+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.607+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2677687343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f18a0 0x5609de7fb200 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.607+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 >> 172.21.15.3:0/2677687343 conn(0x5609de71e8f0 msgr2=0x5609de7ec710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.607+0000 7f618dffb700 0 client.26749 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.627+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 shutdown_connections 2023-08-11T14:19:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.632+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2677687343 wait complete. 2023-08-11T14:19:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.632+0000 7f61c07c0b80 20 client.26749 trim_cache size 0 max 16384 2023-08-11T14:19:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.662+0000 7f61c07c0b80 1 Processor -- start 2023-08-11T14:19:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 -- start start 2023-08-11T14:19:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de802200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de7ecf50 con 0x5609de7f18a0 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de8095b0 con 0x5609de7f0f40 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.670+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de7f8a10 con 0x5609de7f2790 2023-08-11T14:19:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.673+0000 7f61a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de802200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.673+0000 7f61a659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de802200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60600/0 (socket says 172.21.15.3:60600) 2023-08-11T14:19:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.673+0000 7f61a659c700 1 -- 172.21.15.3:0/2220212068 learned_addr learned my addr 172.21.15.3:0/2220212068 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.673+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 -- 172.21.15.3:0/2220212068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 msgr2=0x5609de7f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 -- 172.21.15.3:0/2220212068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 msgr2=0x5609de802200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de802200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.684+0000 7f619dd9b700 1 -- 172.21.15.3:0/2220212068 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609de558c00 con 0x5609de7f2790 2023-08-11T14:19:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.685+0000 7f619dd9b700 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f8c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5609de809820 tx=0x7f619000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46266de7f1964a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.704+0000 7f619d59a700 1 -- 172.21.15.3:0/2220212068 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190007680 con 0x5609de7f2790 2023-08-11T14:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.704+0000 7f619d59a700 1 -- 172.21.15.3:0/2220212068 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619001a040 con 0x5609de7f2790 2023-08-11T14:19:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.704+0000 7f619d59a700 1 -- 172.21.15.3:0/2220212068 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190025bd0 con 0x5609de7f2790 2023-08-11T14:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.705+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2220212068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 msgr2=0x5609de7f8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.705+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f8c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x5609de809820 tx=0x7f619000b040 comp rx=0 tx=0).stop 2023-08-11T14:19:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.709+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2220212068 shutdown_connections 2023-08-11T14:19:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.709+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de7f8c60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.709+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f0f40 0x5609de7f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.709+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/2220212068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f18a0 0x5609de802200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.709+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2220212068 >> 172.21.15.3:0/2220212068 conn(0x5609de71e8f0 msgr2=0x5609de7fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.728+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2220212068 shutdown_connections 2023-08-11T14:19:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.728+0000 7f61c07c0b80 1 -- 172.21.15.3:0/2220212068 wait complete. 2023-08-11T14:19:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.728+0000 7f61c07c0b80 1 -- start start 2023-08-11T14:19:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 Processor -- start 2023-08-11T14:19:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f0f40 0x5609de7fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 0x5609de800440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de801450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de801990 con 0x5609de7f0f40 2023-08-11T14:19:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de801b00 con 0x5609de7f18a0 2023-08-11T14:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609de801cc0 con 0x5609de7f2790 2023-08-11T14:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.729+0000 7f61c07c0b80 10 client.0 _ll_register_callbacks cb 0x7f6198009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.731+0000 7f61a659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 0x5609de800440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.731+0000 7f61a659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 0x5609de800440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:60086/0 (socket says 172.21.15.3:60086) 2023-08-11T14:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.731+0000 7f61a659c700 1 -- 172.21.15.3:0/915660832 learned_addr learned my addr 172.21.15.3:0/915660832 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.733+0000 7f619dd9b700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de801450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f0f40 0x5609de7fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 -- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 msgr2=0x5609de800440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 0x5609de800440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 -- 172.21.15.3:0/915660832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f0f40 msgr2=0x5609de7fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f0f40 0x5609de7fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.739+0000 7f619dd9b700 1 -- 172.21.15.3:0/915660832 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609de5c35d0 con 0x5609de7f2790 2023-08-11T14:19:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.756+0000 7f619dd9b700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de801450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f619000e400 tx=0x7f6190000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71b827301b85dd9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.757+0000 7f618dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:19:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.768+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190004c60 con 0x5609de7f2790 2023-08-11T14:19:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.768+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6190008a10 con 0x5609de7f2790 2023-08-11T14:19:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.768+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6190004dd0 con 0x5609de7f2790 2023-08-11T14:19:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.779+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6190008bd0 con 0x5609de7f2790 2023-08-11T14:19:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.790+0000 7f61c07c0b80 10 client.26755 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.790+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5609de5961f0 con 0x5609de7f2790 2023-08-11T14:19:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.797+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6190004dd0 con 0x5609de7f2790 2023-08-11T14:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.797+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6190001880 con 0x5609de7f2790 2023-08-11T14:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 20 client.26755 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de85b1e0 0x5609de85d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 4 client.26755 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:19:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.798+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609de73c7c0 con 0x5609de85b1e0 2023-08-11T14:19:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.806+0000 7f61a659c700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de85b1e0 0x5609de85d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.845+0000 7f61a659c700 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de85b1e0 0x5609de85d600 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b72b2b500a7258 server_cookie=aa6e320ba77915d5 in_seq=0 out_seq=0 2023-08-11T14:19:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:25.846+0000 7f618dffb700 10 client.26755 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:26.793 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:26 smithi003 ceph-mon[97195]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:19:26.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:26 smithi187 ceph-mon[130973]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:19:26.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:26 smithi103 ceph-mon[130434]: pgmap v812: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-08-11T14:19:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.034+0000 7f618dffb700 1 -- 172.21.15.3:0/915660832 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609de73c7c0 con 0x5609de85b1e0 2023-08-11T14:19:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.034+0000 7f618dffb700 10 client.26755 handle_command_reply: tid=0 2023-08-11T14:19:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.034+0000 7f61c07c0b80 1 client.26755 shutdown 2023-08-11T14:19:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:19:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:19:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:19:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:19:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.039+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 msgr2=0x5609de801450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.039+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de801450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f619000e400 tx=0x7f6190000bc0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 shutdown_connections 2023-08-11T14:19:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5609de7f2790 0x5609de801450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5609de7f0f40 0x5609de7fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5609de85b1e0 0x5609de85d600 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 --2- 172.21.15.3:0/915660832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609de7f18a0 0x5609de800440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 >> 172.21.15.3:0/915660832 conn(0x5609de71e8f0 msgr2=0x5609de7ed2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f618dffb700 0 client.26755 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 shutdown_connections 2023-08-11T14:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 1 -- 172.21.15.3:0/915660832 wait complete. 2023-08-11T14:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.040+0000 7f61c07c0b80 20 client.26755 trim_cache size 0 max 16384 2023-08-11T14:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 msgr2=0x7f61940a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 0x7f61940a2610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f61a0098360 tx=0x7f61a0094040 comp rx=0 tx=0).stop 2023-08-11T14:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 msgr2=0x7f61a811f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b4a0 tx=0x7f61a000b860 comp rx=0 tx=0).stop 2023-08-11T14:19:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 shutdown_connections 2023-08-11T14:19:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61a810cc60 0x7f61a811f8a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61a8107920 0x7f61a8118120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f61940a0150 0x7f61940a2610 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 --2- 172.21.15.3:0/2105778547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a810a290 0x7f61a8118660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.042+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 >> 172.21.15.3:0/2105778547 conn(0x7f61a806d750 msgr2=0x7f61a81083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.045+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 shutdown_connections 2023-08-11T14:19:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:27.046+0000 7f61acdff700 1 -- 172.21.15.3:0/2105778547 wait complete. 2023-08-11T14:19:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 750.0 s, 469.6 tps, lat 209.708 ms stddev 521.171 2023-08-11T14:19:28.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:28 smithi003 ceph-mon[97195]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:19:28.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:28 smithi187 ceph-mon[130973]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:19:28.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:28 smithi103 ceph-mon[130434]: pgmap v813: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2023-08-11T14:19:30.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:30 smithi187 ceph-mon[130973]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:19:30.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:30 smithi003 ceph-mon[97195]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:19:30.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:30 smithi103 ceph-mon[130434]: pgmap v814: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-08-11T14:19:32.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:32 smithi187 ceph-mon[130973]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:19:32.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:32 smithi003 ceph-mon[97195]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:19:32.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:32 smithi103 ceph-mon[130434]: pgmap v815: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:19:33.088 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 755.0 s, 713.0 tps, lat 139.420 ms stddev 251.533 2023-08-11T14:19:33.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:33 smithi103 ceph-mon[130434]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:19:34.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:33 smithi003 ceph-mon[97195]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:19:34.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:33 smithi187 ceph-mon[130973]: pgmap v816: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 1.31k op/s 2023-08-11T14:19:36.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:36 smithi003 ceph-mon[97195]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-08-11T14:19:36.795 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:36 smithi187 ceph-mon[130973]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-08-11T14:19:36.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:36 smithi103 ceph-mon[130434]: pgmap v817: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-08-11T14:19:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 760.0 s, 808.9 tps, lat 126.483 ms stddev 244.876 2023-08-11T14:19:38.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:38 smithi003 ceph-mon[97195]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:19:38.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:38 smithi187 ceph-mon[130973]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:19:38.938 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:38 smithi103 ceph-mon[130434]: pgmap v818: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.385+0000 7f23c81ff700 1 -- 172.21.15.3:0/849474418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c00756a0 msgr2=0x7f23c0077b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.385+0000 7f23c81ff700 1 --2- 172.21.15.3:0/849474418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c00756a0 0x7f23c0077b30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f23bc008580 tx=0x7f23bc00ac60 comp rx=0 tx=0).stop 2023-08-11T14:19:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.386+0000 7f23c81ff700 1 -- 172.21.15.3:0/849474418 shutdown_connections 2023-08-11T14:19:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.386+0000 7f23c81ff700 1 --2- 172.21.15.3:0/849474418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c0078070 0x7f23c007a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.386+0000 7f23c81ff700 1 --2- 172.21.15.3:0/849474418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c0072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.386+0000 7f23c81ff700 1 --2- 172.21.15.3:0/849474418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c00756a0 0x7f23c0077b30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.386+0000 7f23c81ff700 1 -- 172.21.15.3:0/849474418 >> 172.21.15.3:0/849474418 conn(0x7f23c006d750 msgr2=0x7f23c006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.397+0000 7f23c81ff700 1 -- 172.21.15.3:0/849474418 shutdown_connections 2023-08-11T14:19:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.397+0000 7f23c81ff700 1 -- 172.21.15.3:0/849474418 wait complete. 2023-08-11T14:19:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.398+0000 7f23c81ff700 1 Processor -- start 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 -- start start 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c01b9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c00756a0 0x7f23c01bb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 0x7f23c01bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01be310 con 0x7f23c0078070 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0129960 con 0x7f23c00727c0 2023-08-11T14:19:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.399+0000 7f23c81ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0129b20 con 0x7f23c00756a0 2023-08-11T14:19:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.400+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c01b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.400+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c01b9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57728/0 (socket says 172.21.15.3:57728) 2023-08-11T14:19:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.400+0000 7f23c5f9b700 1 -- 172.21.15.3:0/4241813856 learned_addr learned my addr 172.21.15.3:0/4241813856 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.401+0000 7f23c679c700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 0x7f23c01bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.402+0000 7f23c679c700 1 -- 172.21.15.3:0/4241813856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c00756a0 msgr2=0x7f23c01bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.402+0000 7f23c679c700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c00756a0 0x7f23c01bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.402+0000 7f23c679c700 1 -- 172.21.15.3:0/4241813856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 msgr2=0x7f23c01b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.402+0000 7f23c679c700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c01b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.402+0000 7f23c679c700 1 -- 172.21.15.3:0/4241813856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc008230 con 0x7f23c0078070 2023-08-11T14:19:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.403+0000 7f23c679c700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 0x7f23c01bdb40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f23b000a820 tx=0x7f23b000ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.406+0000 7f23b6ffd700 1 -- 172.21.15.3:0/4241813856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0010070 con 0x7f23c0078070 2023-08-11T14:19:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.406+0000 7f23b6ffd700 1 -- 172.21.15.3:0/4241813856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b0004cf0 con 0x7f23c0078070 2023-08-11T14:19:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.406+0000 7f23b6ffd700 1 -- 172.21.15.3:0/4241813856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0007890 con 0x7f23c0078070 2023-08-11T14:19:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.411+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c0129e00 con 0x7f23c0078070 2023-08-11T14:19:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.411+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c012a2c0 con 0x7f23c0078070 2023-08-11T14:19:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.418+0000 7f23dbbffb80 1 Processor -- start 2023-08-11T14:19:40.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:40 smithi103 ceph-mon[130434]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.431+0000 7f23b6ffd700 1 -- 172.21.15.3:0/4241813856 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f23b0005120 con 0x7f23c0078070 2023-08-11T14:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.433+0000 7f23b6ffd700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 0x7f23ac0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.433+0000 7f23b6ffd700 1 -- 172.21.15.3:0/4241813856 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23b00ca700 con 0x7f23c0078070 2023-08-11T14:19:40.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.441+0000 7f23c579a700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 0x7f23ac0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 -- start start 2023-08-11T14:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3622a5d0 con 0x559c362d0f40 2023-08-11T14:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c36227ef0 con 0x559c362d2790 2023-08-11T14:19:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.442+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362237e0 con 0x559c362d18a0 2023-08-11T14:19:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.454+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.454+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34280/0 (socket says 172.21.15.3:34280) 2023-08-11T14:19:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.454+0000 7f23c5f9b700 1 -- 172.21.15.3:0/2983299833 learned_addr learned my addr 172.21.15.3:0/2983299833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.455+0000 7f23c679c700 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.456+0000 7f23c579a700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 0x7f23ac0a2610 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00a270 tx=0x7f23bc00ae40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.459+0000 7f23c579a700 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.467+0000 7f23c5f9b700 1 -- 172.21.15.3:0/2983299833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 msgr2=0x559c362d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.467+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.467+0000 7f23c5f9b700 1 -- 172.21.15.3:0/2983299833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 msgr2=0x559c362d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.467+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.467+0000 7f23c5f9b700 1 -- 172.21.15.3:0/2983299833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c360a35d0 con 0x559c362d0f40 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.487+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23b800e240 tx=0x7f23b8009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb557472e14dd2f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.488+0000 7f23abfff700 1 -- 172.21.15.3:0/2983299833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8009c40 con 0x559c362d0f40 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.489+0000 7f23abfff700 1 -- 172.21.15.3:0/2983299833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b800f040 con 0x559c362d0f40 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.489+0000 7f23abfff700 1 -- 172.21.15.3:0/2983299833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8013940 con 0x559c362d0f40 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.490+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2983299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 msgr2=0x559c362d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.490+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23b800e240 tx=0x7f23b8009790 comp rx=0 tx=0).stop 2023-08-11T14:19:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.491+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2983299833 shutdown_connections 2023-08-11T14:19:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.491+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.491+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362d1360 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.491+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2983299833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.491+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2983299833 >> 172.21.15.3:0/2983299833 conn(0x559c361fe8f0 msgr2=0x559c362cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:40.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.493+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2983299833 shutdown_connections 2023-08-11T14:19:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.493+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2983299833 wait complete. 2023-08-11T14:19:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.494+0000 7f23dbbffb80 1 -- start start 2023-08-11T14:19:40.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.494+0000 7f23dbbffb80 1 Processor -- start 2023-08-11T14:19:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 0x559c362e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c361c1a60 con 0x559c362d0f40 2023-08-11T14:19:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c361c1760 con 0x559c362d18a0 2023-08-11T14:19:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c360a58c0 con 0x559c362d2790 2023-08-11T14:19:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.495+0000 7f23dbbffb80 10 client.0 _ll_register_callbacks cb 0x559c361c4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.496+0000 7f23c579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.496+0000 7f23c579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:57736/0 (socket says 172.21.15.3:57736) 2023-08-11T14:19:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.496+0000 7f23c579a700 1 -- 172.21.15.3:0/2543209031 learned_addr learned my addr 172.21.15.3:0/2543209031 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c679c700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 0x559c362e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c579a700 1 -- 172.21.15.3:0/2543209031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 msgr2=0x559c362e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c579a700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 0x559c362e21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c579a700 1 -- 172.21.15.3:0/2543209031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 msgr2=0x559c362dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c579a700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362dac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c579a700 1 -- 172.21.15.3:0/2543209031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8009da0 con 0x559c362d18a0 2023-08-11T14:19:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.500+0000 7f23c679c700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 0x559c362e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:19:40.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:40 smithi187 ceph-mon[130973]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:19:40.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:40 smithi003 ceph-mon[97195]: pgmap v819: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-08-11T14:19:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.542+0000 7f23c579a700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23bc008580 tx=0x7f23bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf3f0829c990cd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.542+0000 7f23a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:19:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.544+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003680 con 0x559c362d18a0 2023-08-11T14:19:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.544+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23bc0037e0 con 0x559c362d18a0 2023-08-11T14:19:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.544+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c362c47e0 con 0x559c362d18a0 2023-08-11T14:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.555+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23bc0039a0 con 0x559c362d18a0 2023-08-11T14:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.557+0000 7f23dbbffb80 10 client.26756 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.557+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x559c3604fd80 con 0x559c362d18a0 2023-08-11T14:19:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.583+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c362c47e0 con 0x559c362d18a0 2023-08-11T14:19:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.583+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f23bc030070 con 0x559c362d18a0 2023-08-11T14:19:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 10 client.26756 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 10 client.26756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 20 client.26756 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c362e6a70 0x559c362e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 4 client.26756 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:19:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.584+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c362a0480 con 0x559c362e6a70 2023-08-11T14:19:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.586+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c362e6a70 0x559c362e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.605+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c362e6a70 0x559c362e8e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50542332467d17 server_cookie=5ae68e5dc180a503 in_seq=0 out_seq=0 2023-08-11T14:19:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.606+0000 7f23a97fa700 10 client.26756 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.612+0000 7f23a97fa700 1 -- 172.21.15.3:0/2543209031 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559c362a0480 con 0x559c362e6a70 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.612+0000 7f23a97fa700 10 client.26756 handle_command_reply: tid=0 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.612+0000 7f23dbbffb80 1 client.26756 shutdown 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.612+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 msgr2=0x559c362db190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.612+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23bc008580 tx=0x7f23bc000bc0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 shutdown_connections 2023-08-11T14:19:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d2790 0x559c362e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c362e6a70 0x559c362e8e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/2543209031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d18a0 0x559c362db190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 >> 172.21.15.3:0/2543209031 conn(0x559c361fe8f0 msgr2=0x559c362cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23a97fa700 0 client.26756 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 shutdown_connections 2023-08-11T14:19:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.613+0000 7f23dbbffb80 1 -- 172.21.15.3:0/2543209031 wait complete. 2023-08-11T14:19:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.619+0000 7f23dbbffb80 20 client.26756 trim_cache size 0 max 16384 2023-08-11T14:19:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.657+0000 7f23dbbffb80 1 Processor -- start 2023-08-11T14:19:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 -- start start 2023-08-11T14:19:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362c6ea0 con 0x559c362d0f40 2023-08-11T14:19:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362d89d0 con 0x559c362d2790 2023-08-11T14:19:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.659+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362d62c0 con 0x559c362d18a0 2023-08-11T14:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.673+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.673+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362ed7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34294/0 (socket says 172.21.15.3:34294) 2023-08-11T14:19:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.673+0000 7f23c5f9b700 1 -- 172.21.15.3:0/3698009850 learned_addr learned my addr 172.21.15.3:0/3698009850 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.673+0000 7f23c579a700 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.673+0000 7f23c679c700 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.679+0000 7f23c579a700 1 -- 172.21.15.3:0/3698009850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 msgr2=0x559c362e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.679+0000 7f23c579a700 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.679+0000 7f23c579a700 1 -- 172.21.15.3:0/3698009850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 msgr2=0x559c362ed7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.679+0000 7f23c579a700 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362ed7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.679+0000 7f23c579a700 1 -- 172.21.15.3:0/3698009850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c36038c00 con 0x559c362d18a0 2023-08-11T14:19:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.687+0000 7f23c579a700 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362e2190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d010 tx=0x7f23bc00b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a41f716b899509 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.692+0000 7f23b4ff9700 1 -- 172.21.15.3:0/3698009850 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc013680 con 0x559c362d18a0 2023-08-11T14:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.692+0000 7f23b4ff9700 1 -- 172.21.15.3:0/3698009850 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23bc013d70 con 0x559c362d18a0 2023-08-11T14:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.692+0000 7f23b4ff9700 1 -- 172.21.15.3:0/3698009850 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003900 con 0x559c362d18a0 2023-08-11T14:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.693+0000 7f23dbbffb80 1 -- 172.21.15.3:0/3698009850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 msgr2=0x559c362e2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.693+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362e2190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d010 tx=0x7f23bc00b450 comp rx=0 tx=0).stop 2023-08-11T14:19:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.712+0000 7f23dbbffb80 1 -- 172.21.15.3:0/3698009850 shutdown_connections 2023-08-11T14:19:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.712+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.712+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d0f40 0x559c362ed7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.712+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/3698009850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d18a0 0x559c362e2190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.712+0000 7f23dbbffb80 1 -- 172.21.15.3:0/3698009850 >> 172.21.15.3:0/3698009850 conn(0x559c361fe8f0 msgr2=0x559c362d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.724+0000 7f23dbbffb80 1 -- 172.21.15.3:0/3698009850 shutdown_connections 2023-08-11T14:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.732+0000 7f23dbbffb80 1 -- 172.21.15.3:0/3698009850 wait complete. 2023-08-11T14:19:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.732+0000 7f23dbbffb80 1 -- start start 2023-08-11T14:19:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.733+0000 7f23dbbffb80 1 Processor -- start 2023-08-11T14:19:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 0x559c362df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 0x559c362dfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362e0f80 con 0x559c362d18a0 2023-08-11T14:19:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362e10f0 con 0x559c362d2790 2023-08-11T14:19:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c362e12b0 con 0x559c362d0f40 2023-08-11T14:19:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.738+0000 7f23dbbffb80 10 client.0 _ll_register_callbacks cb 0x559c361c4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.739+0000 7f23c5f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 0x559c362df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.743+0000 7f23c579a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 0x559c362dfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.743+0000 7f23c579a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 0x559c362dfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34308/0 (socket says 172.21.15.3:34308) 2023-08-11T14:19:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.743+0000 7f23c579a700 1 -- 172.21.15.3:0/1933635595 learned_addr learned my addr 172.21.15.3:0/1933635595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.750+0000 7f23c679c700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.757+0000 7f23c5f9b700 1 -- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 msgr2=0x559c362e0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.757+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.757+0000 7f23c5f9b700 1 -- 172.21.15.3:0/1933635595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 msgr2=0x559c362dfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.757+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 0x559c362dfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.757+0000 7f23c5f9b700 1 -- 172.21.15.3:0/1933635595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c3604fd80 con 0x559c362d0f40 2023-08-11T14:19:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.767+0000 7f23c5f9b700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 0x559c362df4f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23b8004c20 tx=0x7f23b8007000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814e8d6c9315b70b server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.768+0000 7f23a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:19:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.776+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80298b0 con 0x559c362d0f40 2023-08-11T14:19:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.776+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f23b8022040 con 0x559c362d0f40 2023-08-11T14:19:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.777+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b800f040 con 0x559c362d0f40 2023-08-11T14:19:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.788+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b8029a10 con 0x559c362d0f40 2023-08-11T14:19:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.794+0000 7f23dbbffb80 10 client.26767 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.794+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x559c360761f0 con 0x559c362d0f40 2023-08-11T14:19:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.794+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b800f040 con 0x559c362d0f40 2023-08-11T14:19:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.794+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f23b801d420 con 0x559c362d0f40 2023-08-11T14:19:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 10 client.26767 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 10 client.26767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 20 client.26767 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c3633ac90 0x559c3633d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 4 client.26767 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.795+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c3621c7c0 con 0x559c3633ac90 2023-08-11T14:19:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.798+0000 7f23c579a700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c3633ac90 0x559c3633d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.822+0000 7f23c579a700 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c3633ac90 0x559c3633d0b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6bc3393c0ef50e server_cookie=80668a839687a3c3 in_seq=0 out_seq=0 2023-08-11T14:19:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:40.823+0000 7f23a97fa700 10 client.26767 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:42.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:42 smithi187 ceph-mon[130973]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 970 op/s 2023-08-11T14:19:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.890+0000 7f23a97fa700 1 -- 172.21.15.3:0/1933635595 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c3621c7c0 con 0x559c3633ac90 2023-08-11T14:19:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.890+0000 7f23a97fa700 10 client.26767 handle_command_reply: tid=0 2023-08-11T14:19:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.890+0000 7f23dbbffb80 1 client.26767 shutdown 2023-08-11T14:19:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.894+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 msgr2=0x559c362df4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.894+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 0x559c362df4f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23b8004c20 tx=0x7f23b8007000 comp rx=0 tx=0).stop 2023-08-11T14:19:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 shutdown_connections 2023-08-11T14:19:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x559c3633ac90 0x559c3633d0b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c362d2790 0x559c362e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c362d0f40 0x559c362df4f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 --2- 172.21.15.3:0/1933635595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559c362d18a0 0x559c362dfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 >> 172.21.15.3:0/1933635595 conn(0x559c361fe8f0 msgr2=0x559c362e2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.902+0000 7f23a97fa700 0 client.26767 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.908+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 shutdown_connections 2023-08-11T14:19:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.908+0000 7f23dbbffb80 1 -- 172.21.15.3:0/1933635595 wait complete. 2023-08-11T14:19:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.909+0000 7f23dbbffb80 20 client.26767 trim_cache size 0 max 16384 2023-08-11T14:19:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:19:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:19:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:19:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:19:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.914+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 msgr2=0x7f23ac0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.914+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 0x7f23ac0a2610 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00a270 tx=0x7f23bc00ae40 comp rx=0 tx=0).stop 2023-08-11T14:19:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.914+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 msgr2=0x7f23c01bdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.914+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 0x7f23c01bdb40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f23b000a820 tx=0x7f23b000ab30 comp rx=0 tx=0).stop 2023-08-11T14:19:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 shutdown_connections 2023-08-11T14:19:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0078070 0x7f23c01bdb40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f23ac0a0150 0x7f23ac0a2610 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23c00727c0 0x7f23c01b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 --2- 172.21.15.3:0/4241813856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23c00756a0 0x7f23c01bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.916+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 >> 172.21.15.3:0/4241813856 conn(0x7f23c006d750 msgr2=0x7f23c0078d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.917+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 shutdown_connections 2023-08-11T14:19:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:42.917+0000 7f23c81ff700 1 -- 172.21.15.3:0/4241813856 wait complete. 2023-08-11T14:19:42.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:42 smithi103 ceph-mon[130434]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 970 op/s 2023-08-11T14:19:43.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:42 smithi003 ceph-mon[97195]: pgmap v820: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 14 MiB/s wr, 970 op/s 2023-08-11T14:19:43.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 765.0 s, 212.0 tps, lat 426.394 ms stddev 948.516 2023-08-11T14:19:44.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:44 smithi103 ceph-mon[130434]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-08-11T14:19:45.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:44 smithi187 ceph-mon[130973]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-08-11T14:19:45.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:44 smithi003 ceph-mon[97195]: pgmap v821: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.28k op/s 2023-08-11T14:19:46.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:45 smithi187 ceph-mon[130973]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-08-11T14:19:46.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:45 smithi003 ceph-mon[97195]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-08-11T14:19:46.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:45 smithi103 ceph-mon[130434]: pgmap v822: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-08-11T14:19:48.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 770.0 s, 520.8 tps, lat 207.530 ms stddev 327.181 2023-08-11T14:19:48.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:48 smithi003 ceph-mon[97195]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:19:48.805 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:48 smithi187 ceph-mon[130973]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:19:48.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:48 smithi103 ceph-mon[130434]: pgmap v823: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:19:49.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:49 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:19:50.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:49 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:19:50.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:49 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:19:50.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:50 smithi103 ceph-mon[130434]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2023-08-11T14:19:51.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:50 smithi187 ceph-mon[130973]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2023-08-11T14:19:51.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:50 smithi003 ceph-mon[97195]: pgmap v824: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 14 MiB/s wr, 1.22k op/s 2023-08-11T14:19:52.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:51 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:19:52.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:51 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:19:52.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:51 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:19:52.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:51 smithi003 ceph-mon[97195]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 957 op/s 2023-08-11T14:19:52.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:51 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:19:52.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:51 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:19:52.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:51 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:19:52.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:51 smithi187 ceph-mon[130973]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 957 op/s 2023-08-11T14:19:52.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:51 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:19:52.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:51 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:19:52.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:51 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:19:52.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:51 smithi103 ceph-mon[130434]: pgmap v825: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 957 op/s 2023-08-11T14:19:53.083 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 775.0 s, 757.5 tps, lat 135.073 ms stddev 169.446 2023-08-11T14:19:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:54 smithi103 ceph-mon[130434]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:19:55.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:54 smithi187 ceph-mon[130973]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:19:55.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:54 smithi003 ceph-mon[97195]: pgmap v826: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-08-11T14:19:56.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:55 smithi187 ceph-mon[130973]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 815 op/s 2023-08-11T14:19:56.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:55 smithi003 ceph-mon[97195]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 815 op/s 2023-08-11T14:19:56.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:55 smithi103 ceph-mon[130434]: pgmap v827: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 815 op/s 2023-08-11T14:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.231+0000 7f832b5ff700 1 -- 172.21.15.3:0/2990968521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410a290 msgr2=0x7f832410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.231+0000 7f832b5ff700 1 --2- 172.21.15.3:0/2990968521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410a290 0x7f832410c720 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f831c01c0d0 tx=0x7f831c01c3e0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.234+0000 7f832b5ff700 1 -- 172.21.15.3:0/2990968521 shutdown_connections 2023-08-11T14:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.234+0000 7f832b5ff700 1 --2- 172.21.15.3:0/2990968521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410cc60 0x7f832410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.235+0000 7f832b5ff700 1 --2- 172.21.15.3:0/2990968521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8324107920 0x7f8324109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.235+0000 7f832b5ff700 1 --2- 172.21.15.3:0/2990968521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410a290 0x7f832410c720 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.235+0000 7f832b5ff700 1 -- 172.21.15.3:0/2990968521 >> 172.21.15.3:0/2990968521 conn(0x7f832406d750 msgr2=0x7f832406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.255+0000 7f832b5ff700 1 -- 172.21.15.3:0/2990968521 shutdown_connections 2023-08-11T14:19:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.259+0000 7f832b5ff700 1 -- 172.21.15.3:0/2990968521 wait complete. 2023-08-11T14:19:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.260+0000 7f832b5ff700 1 Processor -- start 2023-08-11T14:19:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.262+0000 7f832b5ff700 1 -- start start 2023-08-11T14:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.262+0000 7f832b5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8324107920 0x7f832419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.262+0000 7f832b5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f832b5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410cc60 0x7f832419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f832b5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419e7d0 con 0x7f8324107920 2023-08-11T14:19:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f832b5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419e940 con 0x7f832410cc60 2023-08-11T14:19:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f832b5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419eb00 con 0x7f832410a290 2023-08-11T14:19:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f8328b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f8328b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47544/0 (socket says 172.21.15.3:47544) 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.263+0000 7f8328b9a700 1 -- 172.21.15.3:0/3570327735 learned_addr learned my addr 172.21.15.3:0/3570327735 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.264+0000 7f8328b9a700 1 -- 172.21.15.3:0/3570327735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410cc60 msgr2=0x7f832419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.264+0000 7f8328b9a700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410cc60 0x7f832419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.264+0000 7f8328b9a700 1 -- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8324107920 msgr2=0x7f832419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.264+0000 7f8328b9a700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8324107920 0x7f832419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.264+0000 7f8328b9a700 1 -- 172.21.15.3:0/3570327735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c01ad10 con 0x7f832410a290 2023-08-11T14:19:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.276+0000 7f8328b9a700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f831c01c7b0 tx=0x7f831c01ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.283+0000 7f831a7fc700 1 -- 172.21.15.3:0/3570327735 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c01f040 con 0x7f832410a290 2023-08-11T14:19:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.284+0000 7f831a7fc700 1 -- 172.21.15.3:0/3570327735 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c003c10 con 0x7f832410a290 2023-08-11T14:19:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.284+0000 7f831a7fc700 1 -- 172.21.15.3:0/3570327735 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0052a0 con 0x7f832410a290 2023-08-11T14:19:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.284+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832419ed80 con 0x7f832410a290 2023-08-11T14:19:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.284+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83241a5520 con 0x7f832410a290 2023-08-11T14:19:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.290+0000 7f833eef8b80 1 Processor -- start 2023-08-11T14:19:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 -- start start 2023-08-11T14:19:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446d8a0 0x55df4446e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4446ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df443c65d0 con 0x55df4446e790 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df443c3ef0 con 0x55df4446d8a0 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.291+0000 7f833eef8b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df443bf7e0 con 0x55df4446cf40 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.292+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.292+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47560/0 (socket says 172.21.15.3:47560) 2023-08-11T14:19:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.292+0000 7f8329b9c700 1 -- 172.21.15.3:0/3241628484 learned_addr learned my addr 172.21.15.3:0/3241628484 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.294+0000 7f8329b9c700 1 -- 172.21.15.3:0/3241628484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446d8a0 msgr2=0x55df4446e1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:19:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.294+0000 7f8329b9c700 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446d8a0 0x55df4446e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.294+0000 7f8329b9c700 1 -- 172.21.15.3:0/3241628484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 msgr2=0x55df4446ec10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.294+0000 7f8329b9c700 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4446ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.294+0000 7f8329b9c700 1 -- 172.21.15.3:0/3241628484 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df4423f5d0 con 0x55df4446cf40 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.297+0000 7f8329b9c700 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8320009cd0 tx=0x7f8320030360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b39a73cbf40a757 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.299+0000 7f830f7fe700 1 -- 172.21.15.3:0/3241628484 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320030c10 con 0x55df4446cf40 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.299+0000 7f830f7fe700 1 -- 172.21.15.3:0/3241628484 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8320030d70 con 0x55df4446cf40 2023-08-11T14:19:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.299+0000 7f830f7fe700 1 -- 172.21.15.3:0/3241628484 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320032440 con 0x55df4446cf40 2023-08-11T14:19:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.300+0000 7f833eef8b80 1 -- 172.21.15.3:0/3241628484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 msgr2=0x55df4446d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.300+0000 7f833eef8b80 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8320009cd0 tx=0x7f8320030360 comp rx=0 tx=0).stop 2023-08-11T14:19:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.311+0000 7f833eef8b80 1 -- 172.21.15.3:0/3241628484 shutdown_connections 2023-08-11T14:19:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.311+0000 7f833eef8b80 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4446ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.311+0000 7f833eef8b80 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df4446d360 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.311+0000 7f833eef8b80 1 --2- 172.21.15.3:0/3241628484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446d8a0 0x55df4446e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.311+0000 7f833eef8b80 1 -- 172.21.15.3:0/3241628484 >> 172.21.15.3:0/3241628484 conn(0x55df4439a8f0 msgr2=0x55df44469920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.314+0000 7f831a7fc700 1 -- 172.21.15.3:0/3570327735 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f831c02c460 con 0x7f832410a290 2023-08-11T14:19:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.316+0000 7f831a7fc700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 0x7f83100a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.316+0000 7f831a7fc700 1 -- 172.21.15.3:0/3570327735 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f831c0dd140 con 0x7f832410a290 2023-08-11T14:19:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.325+0000 7f833eef8b80 1 -- 172.21.15.3:0/3241628484 shutdown_connections 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.332+0000 7f833eef8b80 1 -- 172.21.15.3:0/3241628484 wait complete. 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.333+0000 7f833eef8b80 1 -- start start 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.333+0000 7f833eef8b80 1 Processor -- start 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446cf40 0x55df44476cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df444771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4435da60 con 0x55df4446cf40 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4435d760 con 0x55df4446e790 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df442418c0 con 0x55df4446d8a0 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.334+0000 7f833eef8b80 10 client.0 _ll_register_callbacks cb 0x55df44360a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.335+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.335+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45512/0 (socket says 172.21.15.3:45512) 2023-08-11T14:19:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.335+0000 7f8329b9c700 1 -- 172.21.15.3:0/143872441 learned_addr learned my addr 172.21.15.3:0/143872441 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8328b9a700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df444771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8329b9c700 1 -- 172.21.15.3:0/143872441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 msgr2=0x55df444771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8329b9c700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df444771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8329b9c700 1 -- 172.21.15.3:0/143872441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446cf40 msgr2=0x55df44476cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8329b9c700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446cf40 0x55df44476cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.337+0000 7f8329b9c700 1 -- 172.21.15.3:0/143872441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8320009980 con 0x55df4446e790 2023-08-11T14:19:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.338+0000 7f8329b9c700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ca0 tx=0x7f8320030cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae164e453c82aeb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.338+0000 7f830cff9700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:19:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.339+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320004030 con 0x55df4446e790 2023-08-11T14:19:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.340+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8320036070 con 0x55df4446e790 2023-08-11T14:19:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.340+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df444607e0 con 0x55df4446e790 2023-08-11T14:19:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.341+0000 7f832939b700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 0x7f83100a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.343+0000 7f832939b700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 0x7f83100a2610 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55df444692f0 tx=0x7f8314006cd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.344+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8320004190 con 0x55df4446e790 2023-08-11T14:19:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.344+0000 7f833eef8b80 10 client.26768 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.344+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55df441ebd80 con 0x55df4446e790 2023-08-11T14:19:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df444607e0 con 0x55df4446e790 2023-08-11T14:19:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f832004d070 con 0x55df4446e790 2023-08-11T14:19:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 10 client.26768 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 20 client.26768 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df44482900 0x55df44484d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 4 client.26768 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:19:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.346+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df4443c480 con 0x55df44482900 2023-08-11T14:19:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.348+0000 7f832939b700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df44482900 0x55df44484d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.349+0000 7f832939b700 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df44482900 0x55df44484d20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0765e7f9497a07 server_cookie=c384484c353f2302 in_seq=0 out_seq=0 2023-08-11T14:19:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.352+0000 7f830cff9700 10 client.26768 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.352+0000 7f830cff9700 1 -- 172.21.15.3:0/143872441 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55df4443c480 con 0x55df44482900 2023-08-11T14:19:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.352+0000 7f830cff9700 10 client.26768 handle_command_reply: tid=0 2023-08-11T14:19:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.352+0000 7f833eef8b80 1 client.26768 shutdown 2023-08-11T14:19:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 msgr2=0x55df4447e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ca0 tx=0x7f8320030cf0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 shutdown_connections 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df44482900 0x55df44484d20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e230 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446cf40 0x55df44476cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 --2- 172.21.15.3:0/143872441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df444771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 >> 172.21.15.3:0/143872441 conn(0x55df4439a8f0 msgr2=0x55df44468710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.353+0000 7f830cff9700 0 client.26768 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.364+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 shutdown_connections 2023-08-11T14:19:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.364+0000 7f833eef8b80 1 -- 172.21.15.3:0/143872441 wait complete. 2023-08-11T14:19:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.376+0000 7f833eef8b80 20 client.26768 trim_cache size 0 max 16384 2023-08-11T14:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.416+0000 7f833eef8b80 1 Processor -- start 2023-08-11T14:19:56.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 -- start start 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df44471720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446d8a0 0x55df44471c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df44474a30 con 0x55df4446d8a0 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df44472290 con 0x55df4446e790 2023-08-11T14:19:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.417+0000 7f833eef8b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4446cb30 con 0x55df4446cf40 2023-08-11T14:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45528/0 (socket says 172.21.15.3:45528) 2023-08-11T14:19:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 -- 172.21.15.3:0/770440468 learned_addr learned my addr 172.21.15.3:0/770440468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f832939b700 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df44471720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 -- 172.21.15.3:0/770440468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 msgr2=0x55df44471720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df44471720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 -- 172.21.15.3:0/770440468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446d8a0 msgr2=0x55df44471c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446d8a0 0x55df44471c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.418+0000 7f8329b9c700 1 -- 172.21.15.3:0/770440468 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df441d4c00 con 0x55df4446e790 2023-08-11T14:19:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.419+0000 7f8329b9c700 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8320030760 tx=0x7f8320030360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77eb86be300a31d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.433+0000 7f830ffff700 1 -- 172.21.15.3:0/770440468 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320003680 con 0x55df4446e790 2023-08-11T14:19:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.433+0000 7f830ffff700 1 -- 172.21.15.3:0/770440468 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8320005c20 con 0x55df4446e790 2023-08-11T14:19:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.433+0000 7f830ffff700 1 -- 172.21.15.3:0/770440468 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320004a90 con 0x55df4446e790 2023-08-11T14:19:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.434+0000 7f833eef8b80 1 -- 172.21.15.3:0/770440468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 msgr2=0x55df4447e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.434+0000 7f833eef8b80 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8320030760 tx=0x7f8320030360 comp rx=0 tx=0).stop 2023-08-11T14:19:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.441+0000 7f833eef8b80 1 -- 172.21.15.3:0/770440468 shutdown_connections 2023-08-11T14:19:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.441+0000 7f833eef8b80 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446e790 0x55df4447e150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.441+0000 7f833eef8b80 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446cf40 0x55df44471720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.441+0000 7f833eef8b80 1 --2- 172.21.15.3:0/770440468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446d8a0 0x55df44471c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.441+0000 7f833eef8b80 1 -- 172.21.15.3:0/770440468 >> 172.21.15.3:0/770440468 conn(0x55df4439a8f0 msgr2=0x55df44484910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.447+0000 7f833eef8b80 1 -- 172.21.15.3:0/770440468 shutdown_connections 2023-08-11T14:19:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.447+0000 7f833eef8b80 1 -- 172.21.15.3:0/770440468 wait complete. 2023-08-11T14:19:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.448+0000 7f833eef8b80 1 -- start start 2023-08-11T14:19:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.449+0000 7f833eef8b80 1 Processor -- start 2023-08-11T14:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.460+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446cf40 0x55df4447c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4447d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4447db00 con 0x55df4446e790 2023-08-11T14:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4447dc70 con 0x55df4446cf40 2023-08-11T14:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df4447de30 con 0x55df4446d8a0 2023-08-11T14:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f833eef8b80 10 client.0 _ll_register_callbacks cb 0x55df44360a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f8328b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f8328b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:47588/0 (socket says 172.21.15.3:47588) 2023-08-11T14:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.461+0000 7f8328b9a700 1 -- 172.21.15.3:0/1120128017 learned_addr learned my addr 172.21.15.3:0/1120128017 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8329b9c700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4447d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8328b9a700 1 -- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446cf40 msgr2=0x55df4447c070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:19:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8328b9a700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446cf40 0x55df4447c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8328b9a700 1 -- 172.21.15.3:0/1120128017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 msgr2=0x55df4447d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8328b9a700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4447d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.462+0000 7f8328b9a700 1 -- 172.21.15.3:0/1120128017 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df441ebd80 con 0x55df4446d8a0 2023-08-11T14:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.463+0000 7f8328b9a700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f831c0da8c0 tx=0x7f831c0aa330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89956109b5d841ea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:19:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.463+0000 7f830cff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:19:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.469+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c02a320 con 0x55df4446d8a0 2023-08-11T14:19:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.469+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f831c0ab020 con 0x55df4446d8a0 2023-08-11T14:19:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.469+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8320030d70 con 0x55df4446d8a0 2023-08-11T14:19:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.479+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f831c0aa920 con 0x55df4446d8a0 2023-08-11T14:19:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.480+0000 7f833eef8b80 10 client.26782 fetch_fsmap learned FSMap version 58 2023-08-11T14:19:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.480+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55df442121f0 con 0x55df4446d8a0 2023-08-11T14:19:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.498+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8320030d70 con 0x55df4446d8a0 2023-08-11T14:19:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.498+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f831c0b2210 con 0x55df4446d8a0 2023-08-11T14:19:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 10 client.26782 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:19:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 10 client.26782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:19:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 20 client.26782 populate_metadata read hostname 'smithi003' 2023-08-11T14:19:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df444d7360 0x55df444d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:19:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 4 client.26782 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:19:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.499+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df443b87c0 con 0x55df444d7360 2023-08-11T14:19:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.511+0000 7f832939b700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df444d7360 0x55df444d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:19:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.516+0000 7f832939b700 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df444d7360 0x55df444d9780 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234677fa41c443bf server_cookie=815a7e0732a85340 in_seq=0 out_seq=0 2023-08-11T14:19:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.517+0000 7f830cff9700 10 client.26782 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.860+0000 7f830cff9700 1 -- 172.21.15.3:0/1120128017 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df443b87c0 con 0x55df444d7360 2023-08-11T14:19:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.860+0000 7f830cff9700 10 client.26782 handle_command_reply: tid=0 2023-08-11T14:19:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.861+0000 7f833eef8b80 1 client.26782 shutdown 2023-08-11T14:19:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.862+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 msgr2=0x55df4447c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.862+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f831c0da8c0 tx=0x7f831c0aa330 comp rx=0 tx=0).stop 2023-08-11T14:19:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 shutdown_connections 2023-08-11T14:19:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55df4446e790 0x55df4447d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55df444d7360 0x55df444d9780 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df4446cf40 0x55df4447c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 --2- 172.21.15.3:0/1120128017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df4446d8a0 0x55df4447c5b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 >> 172.21.15.3:0/1120128017 conn(0x55df4439a8f0 msgr2=0x55df44471720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f830cff9700 0 client.26782 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 shutdown_connections 2023-08-11T14:19:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.863+0000 7f833eef8b80 1 -- 172.21.15.3:0/1120128017 wait complete. 2023-08-11T14:19:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.864+0000 7f833eef8b80 20 client.26782 trim_cache size 0 max 16384 2023-08-11T14:19:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:19:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:19:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:19:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:19:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 msgr2=0x7f83100a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 0x7f83100a2610 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55df444692f0 tx=0x7f8314006cd0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 msgr2=0x7f832419d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:19:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f831c01c7b0 tx=0x7f831c01ddd0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 shutdown_connections 2023-08-11T14:19:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f83100a0150 0x7f83100a2610 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832410cc60 0x7f832419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8324107920 0x7f832419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 --2- 172.21.15.3:0/3570327735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f832410a290 0x7f832419d320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:19:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.865+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 >> 172.21.15.3:0/3570327735 conn(0x7f832406d750 msgr2=0x7f832410ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:19:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.867+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 shutdown_connections 2023-08-11T14:19:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:19:56.867+0000 7f832b5ff700 1 -- 172.21.15.3:0/3570327735 wait complete. 2023-08-11T14:19:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 780.0 s, 656.6 tps, lat 152.426 ms stddev 340.044 2023-08-11T14:19:58.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:19:58 smithi103 ceph-mon[130434]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 901 op/s 2023-08-11T14:19:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:19:58 smithi003 ceph-mon[97195]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 901 op/s 2023-08-11T14:19:59.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:19:58 smithi187 ceph-mon[130973]: pgmap v828: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 901 op/s 2023-08-11T14:20:00.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:19:59.998+0000 7f7a41abb700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:20:00.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:00 smithi003 ceph-mon[97195]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-08-11T14:20:00.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:00 smithi003 ceph-mon[97195]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:20:00.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:00 smithi103 ceph-mon[130434]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-08-11T14:20:00.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:00 smithi103 ceph-mon[130434]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:20:00.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:00 smithi187 ceph-mon[130973]: pgmap v829: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.01k op/s 2023-08-11T14:20:00.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:00 smithi187 ceph-mon[130973]: overall HEALTH_ERR 1 MDSs report damaged metadata 2023-08-11T14:20:02.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:02 smithi003 ceph-mon[97195]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 909 op/s 2023-08-11T14:20:02.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:02 smithi187 ceph-mon[130973]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 909 op/s 2023-08-11T14:20:02.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:02 smithi103 ceph-mon[130434]: pgmap v830: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 13 MiB/s wr, 909 op/s 2023-08-11T14:20:03.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 785.0 s, 786.5 tps, lat 126.879 ms stddev 281.144 2023-08-11T14:20:04.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:04 smithi103 ceph-mon[130434]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-08-11T14:20:05.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:04 smithi003 ceph-mon[97195]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-08-11T14:20:05.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:04 smithi187 ceph-mon[130973]: pgmap v831: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 1.17k op/s 2023-08-11T14:20:06.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:06 smithi103 ceph-mon[130434]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:20:07.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:06 smithi187 ceph-mon[130973]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:20:07.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:06 smithi003 ceph-mon[97195]: pgmap v832: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 14 MiB/s wr, 1.04k op/s 2023-08-11T14:20:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 790.0 s, 481.7 tps, lat 205.927 ms stddev 498.629 2023-08-11T14:20:08.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:08 smithi103 ceph-mon[130434]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:20:09.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:08 smithi003 ceph-mon[97195]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:20:09.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:08 smithi187 ceph-mon[130973]: pgmap v833: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:20:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.179+0000 7f40a7dff700 1 -- 172.21.15.3:0/3159578937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a00ff190 msgr2=0x7f40a01015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.179+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3159578937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a00ff190 0x7f40a01015c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f409801c0d0 tx=0x7f409801c3e0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.201+0000 7f40a7dff700 1 -- 172.21.15.3:0/3159578937 shutdown_connections 2023-08-11T14:20:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.201+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3159578937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a01044d0 0x7f40a0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.201+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3159578937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a00ff190 0x7f40a01015c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.201+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3159578937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a0101b00 0x7f40a0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.201+0000 7f40a7dff700 1 -- 172.21.15.3:0/3159578937 >> 172.21.15.3:0/3159578937 conn(0x7f40a00fad70 msgr2=0x7f40a00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.208+0000 7f40a7dff700 1 -- 172.21.15.3:0/3159578937 shutdown_connections 2023-08-11T14:20:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.218+0000 7f40a7dff700 1 -- 172.21.15.3:0/3159578937 wait complete. 2023-08-11T14:20:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.220+0000 7f40a7dff700 1 Processor -- start 2023-08-11T14:20:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.221+0000 7f40a7dff700 1 -- start start 2023-08-11T14:20:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a00ff190 0x7f40a01b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 0x7f40a01bb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 0x7f40a01bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01be2e0 con 0x7f40a0101b00 2023-08-11T14:20:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01be450 con 0x7f40a01044d0 2023-08-11T14:20:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.236+0000 7f40a7dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01be610 con 0x7f40a00ff190 2023-08-11T14:20:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.237+0000 7f40a639c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 0x7f40a01bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.241+0000 7f40a539a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 0x7f40a01bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.241+0000 7f40a539a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 0x7f40a01bb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50642/0 (socket says 172.21.15.3:50642) 2023-08-11T14:20:10.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.241+0000 7f40a539a700 1 -- 172.21.15.3:0/3061688114 learned_addr learned my addr 172.21.15.3:0/3061688114 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.248+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a00ff190 0x7f40a01b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.254+0000 7f40a639c700 1 -- 172.21.15.3:0/3061688114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a00ff190 msgr2=0x7f40a01b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.254+0000 7f40a639c700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a00ff190 0x7f40a01b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.254+0000 7f40a639c700 1 -- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 msgr2=0x7f40a01bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.254+0000 7f40a639c700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 0x7f40a01bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.254+0000 7f40a639c700 1 -- 172.21.15.3:0/3061688114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409801ad10 con 0x7f40a01044d0 2023-08-11T14:20:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.264+0000 7f40a639c700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 0x7f40a01bdb10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b3c0 tx=0x7f409c00b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.279+0000 7f4096ffd700 1 -- 172.21.15.3:0/3061688114 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c011620 con 0x7f40a01044d0 2023-08-11T14:20:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.279+0000 7f4096ffd700 1 -- 172.21.15.3:0/3061688114 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409c011c40 con 0x7f40a01044d0 2023-08-11T14:20:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.279+0000 7f4096ffd700 1 -- 172.21.15.3:0/3061688114 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c010d40 con 0x7f40a01044d0 2023-08-11T14:20:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.279+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a01be8f0 con 0x7f40a01044d0 2023-08-11T14:20:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.280+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a01bed80 con 0x7f40a01044d0 2023-08-11T14:20:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.286+0000 7f40bb686b80 1 Processor -- start 2023-08-11T14:20:10.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.294+0000 7f40bb686b80 1 -- start start 2023-08-11T14:20:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328579b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 0x56328579c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579c790 0x56328579cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632856f45d0 con 0x56328579c790 2023-08-11T14:20:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632856f1ef0 con 0x56328579b8a0 2023-08-11T14:20:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.295+0000 7f40bb686b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632856ed7e0 con 0x56328579af40 2023-08-11T14:20:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.305+0000 7f40a5b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 0x56328579c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.305+0000 7f40a5b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 0x56328579c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49310/0 (socket says 172.21.15.3:49310) 2023-08-11T14:20:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.305+0000 7f40a5b9b700 1 -- 172.21.15.3:0/803809504 learned_addr learned my addr 172.21.15.3:0/803809504 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.306+0000 7f40a539a700 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328579b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.309+0000 7f4096ffd700 1 -- 172.21.15.3:0/3061688114 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f409c010380 con 0x7f40a01044d0 2023-08-11T14:20:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.312+0000 7f4096ffd700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 0x7f408c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.312+0000 7f4096ffd700 1 -- 172.21.15.3:0/3061688114 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f409c0cacc0 con 0x7f40a01044d0 2023-08-11T14:20:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 -- 172.21.15.3:0/803809504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 msgr2=0x56328579c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 0x56328579c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 -- 172.21.15.3:0/803809504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579c790 msgr2=0x56328579cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579c790 0x56328579cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 -- 172.21.15.3:0/803809504 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56328556d5d0 con 0x56328579af40 2023-08-11T14:20:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.325+0000 7f40a539a700 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328579b360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40a010f920 tx=0x7f4090009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174ff44fa721e8b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.338+0000 7f40a639c700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 0x7f408c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.349+0000 7f408bfff700 1 -- 172.21.15.3:0/803809504 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090010990 con 0x56328579af40 2023-08-11T14:20:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.349+0000 7f408bfff700 1 -- 172.21.15.3:0/803809504 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4090014470 con 0x56328579af40 2023-08-11T14:20:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.349+0000 7f408bfff700 1 -- 172.21.15.3:0/803809504 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090013630 con 0x56328579af40 2023-08-11T14:20:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.350+0000 7f40bb686b80 1 -- 172.21.15.3:0/803809504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 msgr2=0x56328579b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.350+0000 7f40bb686b80 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328579b360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40a010f920 tx=0x7f4090009e10 comp rx=0 tx=0).stop 2023-08-11T14:20:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.357+0000 7f40bb686b80 1 -- 172.21.15.3:0/803809504 shutdown_connections 2023-08-11T14:20:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.357+0000 7f40bb686b80 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579c790 0x56328579cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.357+0000 7f40bb686b80 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328579b360 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.357+0000 7f40bb686b80 1 --2- 172.21.15.3:0/803809504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579b8a0 0x56328579c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.357+0000 7f40bb686b80 1 -- 172.21.15.3:0/803809504 >> 172.21.15.3:0/803809504 conn(0x5632856c88f0 msgr2=0x563285797920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.366+0000 7f40bb686b80 1 -- 172.21.15.3:0/803809504 shutdown_connections 2023-08-11T14:20:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.366+0000 7f40bb686b80 1 -- 172.21.15.3:0/803809504 wait complete. 2023-08-11T14:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.367+0000 7f40bb686b80 1 -- start start 2023-08-11T14:20:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.367+0000 7f40bb686b80 1 Processor -- start 2023-08-11T14:20:10.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x5632857b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x5632857b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56328568ba60 con 0x56328579b8a0 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x56328568b760 con 0x56328579c790 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56328556f8c0 con 0x56328579af40 2023-08-11T14:20:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.394+0000 7f40bb686b80 10 client.0 _ll_register_callbacks cb 0x56328568ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.395+0000 7f40a539a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.395+0000 7f40a539a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:49326/0 (socket says 172.21.15.3:49326) 2023-08-11T14:20:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.395+0000 7f40a539a700 1 -- 172.21.15.3:0/2118565392 learned_addr learned my addr 172.21.15.3:0/2118565392 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.396+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x5632857b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x5632857b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 -- 172.21.15.3:0/2118565392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 msgr2=0x5632857b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x5632857b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 -- 172.21.15.3:0/2118565392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 msgr2=0x5632857b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x5632857b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.411+0000 7f40a539a700 1 -- 172.21.15.3:0/2118565392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4090009810 con 0x56328579c790 2023-08-11T14:20:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.413+0000 7f40a639c700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 0x7f408c0a2610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f409c094890 tx=0x7f409c094040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:10 smithi103 ceph-mon[130434]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:20:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.437+0000 7f40a539a700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4090014d00 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b073b1a6356f8b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.438+0000 7f40897fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:20:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.468+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090024030 con 0x56328579c790 2023-08-11T14:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.468+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4090025070 con 0x56328579c790 2023-08-11T14:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.468+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56328578e7e0 con 0x56328579c790 2023-08-11T14:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.473+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4090010d10 con 0x56328579c790 2023-08-11T14:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.474+0000 7f40bb686b80 10 client.26780 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.474+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x563285519d80 con 0x56328579c790 2023-08-11T14:20:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.489+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56328578e7e0 con 0x56328579c790 2023-08-11T14:20:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.489+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f409002b020 con 0x56328579c790 2023-08-11T14:20:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 10 client.26780 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 10 client.26780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 20 client.26780 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632857c1a10 0x5632857c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 4 client.26780 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:20:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.496+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56328576a480 con 0x5632857c1a10 2023-08-11T14:20:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.507+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632857c1a10 0x5632857c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.540+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632857c1a10 0x5632857c3e30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0253057727f81aa server_cookie=9da2e1776a77c048 in_seq=0 out_seq=0 2023-08-11T14:20:10.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:10 smithi187 ceph-mon[130973]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:20:10.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:10 smithi003 ceph-mon[97195]: pgmap v834: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 16 MiB/s wr, 1.24k op/s 2023-08-11T14:20:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.544+0000 7f40897fa700 10 client.26780 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.568+0000 7f40897fa700 1 -- 172.21.15.3:0/2118565392 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x56328576a480 con 0x5632857c1a10 2023-08-11T14:20:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.568+0000 7f40897fa700 10 client.26780 handle_command_reply: tid=0 2023-08-11T14:20:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.568+0000 7f40bb686b80 1 client.26780 shutdown 2023-08-11T14:20:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.576+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 msgr2=0x5632857bd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.576+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4090014d00 tx=0x7f4090008040 comp rx=0 tx=0).stop 2023-08-11T14:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 shutdown_connections 2023-08-11T14:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632857c1a10 0x5632857c3e30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632857bd210 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x5632857b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2118565392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x5632857b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.584+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 >> 172.21.15.3:0/2118565392 conn(0x5632856c88f0 msgr2=0x563285796710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.586+0000 7f40897fa700 0 client.26780 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.598+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 shutdown_connections 2023-08-11T14:20:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.607+0000 7f40bb686b80 1 -- 172.21.15.3:0/2118565392 wait complete. 2023-08-11T14:20:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.614+0000 7f40bb686b80 20 client.26780 trim_cache size 0 max 16384 2023-08-11T14:20:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.657+0000 7f40bb686b80 1 Processor -- start 2023-08-11T14:20:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 -- start start 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579af40 0x56328579f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x56328579fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x56328579c650 con 0x56328579af40 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632857b5210 con 0x56328579c790 2023-08-11T14:20:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.662+0000 7f40bb686b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632857b6cc0 con 0x56328579b8a0 2023-08-11T14:20:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.674+0000 7f40a5b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.674+0000 7f40a5b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42602/0 (socket says 172.21.15.3:42602) 2023-08-11T14:20:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.674+0000 7f40a5b9b700 1 -- 172.21.15.3:0/2795483244 learned_addr learned my addr 172.21.15.3:0/2795483244 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.676+0000 7f40a539a700 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x56328579fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.676+0000 7f40a539a700 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579af40 0x56328579f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 -- 172.21.15.3:0/2795483244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 msgr2=0x56328579fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x56328579fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 -- 172.21.15.3:0/2795483244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579af40 msgr2=0x56328579f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579af40 0x56328579f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 -- 172.21.15.3:0/2795483244 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563285502c00 con 0x56328579b8a0 2023-08-11T14:20:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.678+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f409801e010 tx=0x7f4098005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a06df0b6a21b4f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.679+0000 7f4094bff700 1 -- 172.21.15.3:0/2795483244 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801f040 con 0x56328579b8a0 2023-08-11T14:20:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.679+0000 7f4094bff700 1 -- 172.21.15.3:0/2795483244 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409801a2b0 con 0x56328579b8a0 2023-08-11T14:20:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.679+0000 7f4094bff700 1 -- 172.21.15.3:0/2795483244 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40980050b0 con 0x56328579b8a0 2023-08-11T14:20:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 -- 172.21.15.3:0/2795483244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 msgr2=0x5632857bd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f409801e010 tx=0x7f4098005040 comp rx=0 tx=0).stop 2023-08-11T14:20:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 -- 172.21.15.3:0/2795483244 shutdown_connections 2023-08-11T14:20:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x56328579fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579af40 0x56328579f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 --2- 172.21.15.3:0/2795483244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579b8a0 0x5632857bd1d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.680+0000 7f40bb686b80 1 -- 172.21.15.3:0/2795483244 >> 172.21.15.3:0/2795483244 conn(0x5632856c88f0 msgr2=0x5632857c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.692+0000 7f40bb686b80 1 -- 172.21.15.3:0/2795483244 shutdown_connections 2023-08-11T14:20:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.692+0000 7f40bb686b80 1 -- 172.21.15.3:0/2795483244 wait complete. 2023-08-11T14:20:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.692+0000 7f40bb686b80 1 -- start start 2023-08-11T14:20:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.693+0000 7f40bb686b80 1 Processor -- start 2023-08-11T14:20:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328580d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x56328580de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632858148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x563285814df0 con 0x56328579b8a0 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x563285814f60 con 0x56328579c790 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563285815120 con 0x56328579af40 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.701+0000 7f40bb686b80 10 client.0 _ll_register_callbacks cb 0x7f40900136e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.704+0000 7f40a539a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328580d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.704+0000 7f40a539a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328580d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:42614/0 (socket says 172.21.15.3:42614) 2023-08-11T14:20:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.704+0000 7f40a539a700 1 -- 172.21.15.3:0/1158754843 learned_addr learned my addr 172.21.15.3:0/1158754843 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.706+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x56328580de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.707+0000 7f40a5b9b700 1 -- 172.21.15.3:0/1158754843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 msgr2=0x56328580d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.707+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328580d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.707+0000 7f40a5b9b700 1 -- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 msgr2=0x5632858148b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.707+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632858148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.707+0000 7f40a5b9b700 1 -- 172.21.15.3:0/1158754843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a0105480 con 0x56328579b8a0 2023-08-11T14:20:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.708+0000 7f40a5b9b700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x56328580de00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4098018be0 tx=0x7f4098010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c770d108623456 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.708+0000 7f40897fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:20:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.709+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409801f040 con 0x56328579b8a0 2023-08-11T14:20:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.709+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f409800d040 con 0x56328579b8a0 2023-08-11T14:20:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.712+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4090014470 con 0x56328579b8a0 2023-08-11T14:20:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.724+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4098003a00 con 0x56328579b8a0 2023-08-11T14:20:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.729+0000 7f40bb686b80 10 client.16590 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.729+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5632855401f0 con 0x56328579b8a0 2023-08-11T14:20:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.739+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4090014470 con 0x56328579b8a0 2023-08-11T14:20:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.739+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f409802a070 con 0x56328579b8a0 2023-08-11T14:20:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 10 client.16590 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 20 client.16590 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632858165e0 0x5632858189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 4 client.16590 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:20:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.748+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632856e67c0 con 0x5632858165e0 2023-08-11T14:20:10.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.749+0000 7f40a539a700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632858165e0 0x5632858189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.787+0000 7f40a539a700 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632858165e0 0x5632858189d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c39b1840e9c03b server_cookie=54992219d0b886e4 in_seq=0 out_seq=0 2023-08-11T14:20:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:10.787+0000 7f40897fa700 10 client.16590 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.726+0000 7f40897fa700 1 -- 172.21.15.3:0/1158754843 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632856e67c0 con 0x5632858165e0 2023-08-11T14:20:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.726+0000 7f40897fa700 10 client.16590 handle_command_reply: tid=0 2023-08-11T14:20:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.726+0000 7f40bb686b80 1 client.16590 shutdown 2023-08-11T14:20:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.730+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 msgr2=0x56328580de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.730+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x56328580de00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4098018be0 tx=0x7f4098010ea0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 shutdown_connections 2023-08-11T14:20:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5632858165e0 0x5632858189d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56328579c790 0x5632858148b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56328579af40 0x56328580d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 --2- 172.21.15.3:0/1158754843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56328579b8a0 0x56328580de00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.738+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 >> 172.21.15.3:0/1158754843 conn(0x5632856c88f0 msgr2=0x56328579f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.739+0000 7f40897fa700 0 client.16590 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.745+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 shutdown_connections 2023-08-11T14:20:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.750+0000 7f40bb686b80 1 -- 172.21.15.3:0/1158754843 wait complete. 2023-08-11T14:20:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.751+0000 7f40bb686b80 20 client.16590 trim_cache size 0 max 16384 2023-08-11T14:20:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:20:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:20:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:20:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:20:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 msgr2=0x7f408c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 0x7f408c0a2610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f409c094890 tx=0x7f409c094040 comp rx=0 tx=0).stop 2023-08-11T14:20:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 msgr2=0x7f40a01bdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 0x7f40a01bdb10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b3c0 tx=0x7f409c00b6d0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 shutdown_connections 2023-08-11T14:20:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f408c0a0150 0x7f408c0a2610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a01044d0 0x7f40a01bdb10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a00ff190 0x7f40a01b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 --2- 172.21.15.3:0/3061688114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40a0101b00 0x7f40a01bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.756+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 >> 172.21.15.3:0/3061688114 conn(0x7f40a00fad70 msgr2=0x7f40a00fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.761+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 shutdown_connections 2023-08-11T14:20:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:12.763+0000 7f40a7dff700 1 -- 172.21.15.3:0/3061688114 wait complete. 2023-08-11T14:20:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:12 smithi103 ceph-mon[130434]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:20:13.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:12 smithi003 ceph-mon[97195]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:20:13.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:12 smithi187 ceph-mon[130973]: pgmap v835: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-08-11T14:20:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 795.0 s, 666.2 tps, lat 146.600 ms stddev 323.819 2023-08-11T14:20:14.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:14 smithi103 ceph-mon[130434]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:20:15.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:14 smithi187 ceph-mon[130973]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:20:15.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:14 smithi003 ceph-mon[97195]: pgmap v836: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:20:16.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:16 smithi103 ceph-mon[130434]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:20:17.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:16 smithi187 ceph-mon[130973]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:20:17.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:16 smithi003 ceph-mon[97195]: pgmap v837: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-08-11T14:20:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 800.0 s, 827.0 tps, lat 122.308 ms stddev 288.501 2023-08-11T14:20:18.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:18 smithi103 ceph-mon[130434]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-08-11T14:20:19.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:18 smithi003 ceph-mon[97195]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-08-11T14:20:19.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:18 smithi187 ceph-mon[130973]: pgmap v838: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.25k op/s 2023-08-11T14:20:20.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:20 smithi103 ceph-mon[130434]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:20.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:20 smithi003 ceph-mon[97195]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:20.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:20 smithi187 ceph-mon[130973]: pgmap v839: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:22.793 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:22 smithi187 ceph-mon[130973]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:22.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:22 smithi003 ceph-mon[97195]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:22.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:22 smithi103 ceph-mon[130434]: pgmap v840: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 805.0 s, 631.4 tps, lat 161.058 ms stddev 387.788 2023-08-11T14:20:24.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:24 smithi103 ceph-mon[130434]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:20:25.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:24 smithi003 ceph-mon[97195]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:20:25.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:24 smithi187 ceph-mon[130973]: pgmap v841: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-08-11T14:20:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.113+0000 7f8f34dff700 1 -- 172.21.15.3:0/683772968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010cc60 msgr2=0x7f8f3010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.113+0000 7f8f34dff700 1 --2- 172.21.15.3:0/683772968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010cc60 0x7f8f3010d0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f180098f0 tx=0x7f8f18009c00 comp rx=0 tx=0).stop 2023-08-11T14:20:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.123+0000 7f8f34dff700 1 -- 172.21.15.3:0/683772968 shutdown_connections 2023-08-11T14:20:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.123+0000 7f8f34dff700 1 --2- 172.21.15.3:0/683772968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010cc60 0x7f8f3010d0e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.124+0000 7f8f34dff700 1 --2- 172.21.15.3:0/683772968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f30107920 0x7f8f30109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.124+0000 7f8f34dff700 1 --2- 172.21.15.3:0/683772968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f3010a290 0x7f8f3010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.124+0000 7f8f34dff700 1 -- 172.21.15.3:0/683772968 >> 172.21.15.3:0/683772968 conn(0x7f8f3006d750 msgr2=0x7f8f3006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.125+0000 7f8f34dff700 1 -- 172.21.15.3:0/683772968 shutdown_connections 2023-08-11T14:20:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.136+0000 7f8f34dff700 1 -- 172.21.15.3:0/683772968 wait complete. 2023-08-11T14:20:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.137+0000 7f8f34dff700 1 Processor -- start 2023-08-11T14:20:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.137+0000 7f8f34dff700 1 -- start start 2023-08-11T14:20:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f30107920 0x7f8f301a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3010cc60 0x7f8f301a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301a6f50 con 0x7f8f30107920 2023-08-11T14:20:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301a70c0 con 0x7f8f3010a290 2023-08-11T14:20:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.138+0000 7f8f34dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301a7280 con 0x7f8f3010cc60 2023-08-11T14:20:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.144+0000 7f8f2ed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3010cc60 0x7f8f301a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.146+0000 7f8f2dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.146+0000 7f8f2dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51358/0 (socket says 172.21.15.3:51358) 2023-08-11T14:20:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.146+0000 7f8f2dd9b700 1 -- 172.21.15.3:0/3134927366 learned_addr learned my addr 172.21.15.3:0/3134927366 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.153+0000 7f8f2dd9b700 1 -- 172.21.15.3:0/3134927366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3010cc60 msgr2=0x7f8f301a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.153+0000 7f8f2dd9b700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3010cc60 0x7f8f301a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.153+0000 7f8f2dd9b700 1 -- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f30107920 msgr2=0x7f8f301a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.153+0000 7f8f2dd9b700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f30107920 0x7f8f301a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.153+0000 7f8f2dd9b700 1 -- 172.21.15.3:0/3134927366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18009520 con 0x7f8f3010a290 2023-08-11T14:20:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.157+0000 7f8f2dd9b700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9c0 tx=0x7f8f2000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.167+0000 7f8f277fe700 1 -- 172.21.15.3:0/3134927366 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000cbd0 con 0x7f8f3010a290 2023-08-11T14:20:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.167+0000 7f8f277fe700 1 -- 172.21.15.3:0/3134927366 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f2000cd30 con 0x7f8f3010a290 2023-08-11T14:20:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.167+0000 7f8f277fe700 1 -- 172.21.15.3:0/3134927366 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20005580 con 0x7f8f3010a290 2023-08-11T14:20:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.167+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f301a7560 con 0x7f8f3010a290 2023-08-11T14:20:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.167+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f30078d60 con 0x7f8f3010a290 2023-08-11T14:20:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.177+0000 7f8f48733b80 1 Processor -- start 2023-08-11T14:20:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.182+0000 7f8f48733b80 1 -- start start 2023-08-11T14:20:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c518a0 0x558912c521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912baa5d0 con 0x558912c518a0 2023-08-11T14:20:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912ba7ef0 con 0x558912c50f40 2023-08-11T14:20:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.183+0000 7f8f48733b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912ba37e0 con 0x558912c52790 2023-08-11T14:20:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51372/0 (socket says 172.21.15.3:51372) 2023-08-11T14:20:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/2360268342 learned_addr learned my addr 172.21.15.3:0/2360268342 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/2360268342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 msgr2=0x558912c52c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:20:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c52c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/2360268342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c518a0 msgr2=0x558912c521b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:20:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c518a0 0x558912c521b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.184+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/2360268342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558912a235d0 con 0x558912c50f40 2023-08-11T14:20:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.185+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f180098f0 tx=0x7f8f18009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d3d0af5b04379f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.187+0000 7f8f277fe700 1 -- 172.21.15.3:0/3134927366 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f8f20004750 con 0x7f8f3010a290 2023-08-11T14:20:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.190+0000 7f8f277fe700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 0x7f8f1c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.190+0000 7f8f277fe700 1 -- 172.21.15.3:0/3134927366 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8f20014070 con 0x7f8f3010a290 2023-08-11T14:20:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.191+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 0x7f8f1c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.191+0000 7f8f24ff9700 1 -- 172.21.15.3:0/2360268342 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801c040 con 0x558912c50f40 2023-08-11T14:20:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.191+0000 7f8f24ff9700 1 -- 172.21.15.3:0/2360268342 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f1800f7a0 con 0x558912c50f40 2023-08-11T14:20:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.191+0000 7f8f24ff9700 1 -- 172.21.15.3:0/2360268342 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f180046d0 con 0x558912c50f40 2023-08-11T14:20:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.192+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 0x7f8f1c0a2610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8f280060b0 tx=0x7f8f28006040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.193+0000 7f8f48733b80 1 -- 172.21.15.3:0/2360268342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 msgr2=0x558912c51360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.193+0000 7f8f48733b80 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f180098f0 tx=0x7f8f18009460 comp rx=0 tx=0).stop 2023-08-11T14:20:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.196+0000 7f8f48733b80 1 -- 172.21.15.3:0/2360268342 shutdown_connections 2023-08-11T14:20:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.196+0000 7f8f48733b80 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c52c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.196+0000 7f8f48733b80 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c50f40 0x558912c51360 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.196+0000 7f8f48733b80 1 --2- 172.21.15.3:0/2360268342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c518a0 0x558912c521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.196+0000 7f8f48733b80 1 -- 172.21.15.3:0/2360268342 >> 172.21.15.3:0/2360268342 conn(0x558912b7e8f0 msgr2=0x558912c4d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.197+0000 7f8f48733b80 1 -- 172.21.15.3:0/2360268342 shutdown_connections 2023-08-11T14:20:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.200+0000 7f8f48733b80 1 -- 172.21.15.3:0/2360268342 wait complete. 2023-08-11T14:20:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.201+0000 7f8f48733b80 1 -- start start 2023-08-11T14:20:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.201+0000 7f8f48733b80 1 Processor -- start 2023-08-11T14:20:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c6c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c73250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912b41a60 con 0x558912c50f40 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912b41760 con 0x558912c518a0 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.207+0000 7f8f48733b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912a258c0 con 0x558912c52790 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.208+0000 7f8f48733b80 10 client.0 _ll_register_callbacks cb 0x558912b44a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.209+0000 7f8f2e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.209+0000 7f8f2e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41490/0 (socket says 172.21.15.3:41490) 2023-08-11T14:20:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.209+0000 7f8f2e59c700 1 -- 172.21.15.3:0/3199782738 learned_addr learned my addr 172.21.15.3:0/3199782738 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 -- 172.21.15.3:0/3199782738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 msgr2=0x558912c73250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:20:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c73250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 -- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 msgr2=0x558912c6c210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:20:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c6c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 -- 172.21.15.3:0/3199782738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3010dc60 con 0x558912c50f40 2023-08-11T14:20:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.210+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f280179e0 tx=0x7f8f28009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d42e9b633170ae server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.211+0000 7f8f15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:20:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.217+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2801c070 con 0x558912c50f40 2023-08-11T14:20:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.217+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8f28020410 con 0x558912c50f40 2023-08-11T14:20:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.217+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558912c447e0 con 0x558912c50f40 2023-08-11T14:20:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.218+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f280205d0 con 0x558912c50f40 2023-08-11T14:20:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.221+0000 7f8f48733b80 10 client.16596 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.221+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5589129cfd80 con 0x558912c50f40 2023-08-11T14:20:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558912c447e0 con 0x558912c50f40 2023-08-11T14:20:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f28028070 con 0x558912c50f40 2023-08-11T14:20:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 20 client.16596 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c77b10 0x558912c79f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 4 client.16596 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:20:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.240+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558912c20480 con 0x558912c77b10 2023-08-11T14:20:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.241+0000 7f8f2dd9b700 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c77b10 0x558912c79f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.274+0000 7f8f2dd9b700 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c77b10 0x558912c79f30 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655e182c7393f5f server_cookie=baa8f877ec000758 in_seq=0 out_seq=0 2023-08-11T14:20:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.279+0000 7f8f15ffb700 10 client.16596 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.280+0000 7f8f15ffb700 1 -- 172.21.15.3:0/3199782738 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x7f8f200b6670 con 0x558912c77b10 2023-08-11T14:20:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.280+0000 7f8f15ffb700 10 client.16596 handle_command_reply: tid=0 2023-08-11T14:20:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.280+0000 7f8f48733b80 1 client.16596 shutdown 2023-08-11T14:20:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.284+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 msgr2=0x558912c6bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.284+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f280179e0 tx=0x7f8f28009620 comp rx=0 tx=0).stop 2023-08-11T14:20:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 shutdown_connections 2023-08-11T14:20:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c73250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c6bcd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c77b10 0x558912c79f30 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 --2- 172.21.15.3:0/3199782738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c6c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 >> 172.21.15.3:0/3199782738 conn(0x558912b7e8f0 msgr2=0x558912c4c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f15ffb700 0 client.16596 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 shutdown_connections 2023-08-11T14:20:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.285+0000 7f8f48733b80 1 -- 172.21.15.3:0/3199782738 wait complete. 2023-08-11T14:20:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.286+0000 7f8f48733b80 20 client.16596 trim_cache size 0 max 16384 2023-08-11T14:20:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.320+0000 7f8f48733b80 1 Processor -- start 2023-08-11T14:20:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.326+0000 7f8f48733b80 1 -- start start 2023-08-11T14:20:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c50f40 0x558912c4d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 0x558912c4d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c447b0 con 0x558912c52790 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c4cd50 con 0x558912c518a0 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f48733b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c6b020 con 0x558912c50f40 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f2dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f2dd9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:51392/0 (socket says 172.21.15.3:51392) 2023-08-11T14:20:26.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.327+0000 7f8f2dd9b700 1 -- 172.21.15.3:0/787086595 learned_addr learned my addr 172.21.15.3:0/787086595 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.330+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c50f40 0x558912c4d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.333+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 0x558912c4d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.338+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/787086595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c50f40 msgr2=0x558912c4d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.338+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c50f40 0x558912c4d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.338+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/787086595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 msgr2=0x558912c731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.338+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.338+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/787086595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589129b8c00 con 0x558912c52790 2023-08-11T14:20:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.339+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 0x558912c4d830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f180098f0 tx=0x7f8f18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d66ebe40a7d598 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.340+0000 7f8f257fa700 1 -- 172.21.15.3:0/787086595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801c040 con 0x558912c52790 2023-08-11T14:20:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.340+0000 7f8f257fa700 1 -- 172.21.15.3:0/787086595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f18005b00 con 0x558912c52790 2023-08-11T14:20:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.340+0000 7f8f257fa700 1 -- 172.21.15.3:0/787086595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18022720 con 0x558912c52790 2023-08-11T14:20:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.341+0000 7f8f48733b80 1 -- 172.21.15.3:0/787086595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 msgr2=0x558912c4d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.341+0000 7f8f48733b80 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 0x558912c4d830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f180098f0 tx=0x7f8f18009c00 comp rx=0 tx=0).stop 2023-08-11T14:20:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.343+0000 7f8f48733b80 1 -- 172.21.15.3:0/787086595 shutdown_connections 2023-08-11T14:20:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.343+0000 7f8f48733b80 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c52790 0x558912c4d830 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.343+0000 7f8f48733b80 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c50f40 0x558912c4d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.343+0000 7f8f48733b80 1 --2- 172.21.15.3:0/787086595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.343+0000 7f8f48733b80 1 -- 172.21.15.3:0/787086595 >> 172.21.15.3:0/787086595 conn(0x558912b7e8f0 msgr2=0x558912c58c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.344+0000 7f8f48733b80 1 -- 172.21.15.3:0/787086595 shutdown_connections 2023-08-11T14:20:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.344+0000 7f8f48733b80 1 -- 172.21.15.3:0/787086595 wait complete. 2023-08-11T14:20:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.345+0000 7f8f48733b80 1 -- start start 2023-08-11T14:20:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.345+0000 7f8f48733b80 1 Processor -- start 2023-08-11T14:20:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c5dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c60850 con 0x558912c50f40 2023-08-11T14:20:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c609c0 con 0x558912c518a0 2023-08-11T14:20:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558912c60b80 con 0x558912c52790 2023-08-11T14:20:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.346+0000 7f8f48733b80 10 client.0 _ll_register_callbacks cb 0x7f8f28007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.347+0000 7f8f2ed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.347+0000 7f8f2ed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:54960/0 (socket says 172.21.15.3:54960) 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.347+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/976592479 learned_addr learned my addr 172.21.15.3:0/976592479 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 msgr2=0x558912c5dc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c5dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/976592479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 msgr2=0x558912c5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 -- 172.21.15.3:0/976592479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558912a235d0 con 0x558912c52790 2023-08-11T14:20:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f2ed9d700 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009810 tx=0x7f8f1800b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffa6f42555359bb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.348+0000 7f8f15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:20:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.349+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801c040 con 0x558912c52790 2023-08-11T14:20:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.349+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f8f180226b0 con 0x558912c52790 2023-08-11T14:20:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.349+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558912b9c7c0 con 0x558912c52790 2023-08-11T14:20:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.350+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f18022870 con 0x558912c52790 2023-08-11T14:20:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.350+0000 7f8f48733b80 10 client.26806 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.350+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5589129f61f0 con 0x558912c52790 2023-08-11T14:20:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558912b9c7c0 con 0x558912c52790 2023-08-11T14:20:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8f18029070 con 0x558912c52790 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 10 client.26806 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 10 client.26806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 20 client.26806 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c62060 0x558912c644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 4 client.26806 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:20:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558912b5cff0 con 0x558912c62060 2023-08-11T14:20:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.352+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c62060 0x558912c644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.354+0000 7f8f2e59c700 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c62060 0x558912c644c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50d7b673c0e6953 server_cookie=6ddc851999dc3ece in_seq=0 out_seq=0 2023-08-11T14:20:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.355+0000 7f8f15ffb700 10 client.26806 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.423+0000 7f8f15ffb700 1 -- 172.21.15.3:0/976592479 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558912b5cff0 con 0x558912c62060 2023-08-11T14:20:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.423+0000 7f8f15ffb700 10 client.26806 handle_command_reply: tid=0 2023-08-11T14:20:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.423+0000 7f8f48733b80 1 client.26806 shutdown 2023-08-11T14:20:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 msgr2=0x558912c60170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009810 tx=0x7f8f1800b2e0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 shutdown_connections 2023-08-11T14:20:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558912c52790 0x558912c60170 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558912c50f40 0x558912c5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x558912c62060 0x558912c644c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 --2- 172.21.15.3:0/976592479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558912c518a0 0x558912c5dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.425+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 >> 172.21.15.3:0/976592479 conn(0x558912b7e8f0 msgr2=0x558912c4d210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.426+0000 7f8f15ffb700 0 client.26806 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.426+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 shutdown_connections 2023-08-11T14:20:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.426+0000 7f8f48733b80 1 -- 172.21.15.3:0/976592479 wait complete. 2023-08-11T14:20:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:20:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:20:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:20:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:20:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.432+0000 7f8f48733b80 20 client.26806 trim_cache size 0 max 16384 2023-08-11T14:20:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.437+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 msgr2=0x7f8f1c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.437+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 0x7f8f1c0a2610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8f280060b0 tx=0x7f8f28006040 comp rx=0 tx=0).stop 2023-08-11T14:20:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.437+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 msgr2=0x7f8f301a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.437+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9c0 tx=0x7f8f2000ed80 comp rx=0 tx=0).stop 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 shutdown_connections 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3010cc60 0x7f8f301a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f30107920 0x7f8f301a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f8f1c0a0150 0x7f8f1c0a2610 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 --2- 172.21.15.3:0/3134927366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3010a290 0x7f8f301a5a00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.438+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 >> 172.21.15.3:0/3134927366 conn(0x7f8f3006d750 msgr2=0x7f8f3010bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.440+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 shutdown_connections 2023-08-11T14:20:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:26.440+0000 7f8f34dff700 1 -- 172.21.15.3:0/3134927366 wait complete. 2023-08-11T14:20:26.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:26 smithi103 ceph-mon[130434]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-08-11T14:20:27.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:26 smithi003 ceph-mon[97195]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-08-11T14:20:27.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:26 smithi187 ceph-mon[130973]: pgmap v842: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2023-08-11T14:20:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 810.0 s, 710.6 tps, lat 141.445 ms stddev 376.184 2023-08-11T14:20:28.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:28 smithi103 ceph-mon[130434]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-08-11T14:20:29.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:28 smithi003 ceph-mon[97195]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-08-11T14:20:29.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:28 smithi187 ceph-mon[130973]: pgmap v843: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-08-11T14:20:30.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:30 smithi103 ceph-mon[130434]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:20:30.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:30 smithi003 ceph-mon[97195]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:20:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:30 smithi187 ceph-mon[130973]: pgmap v844: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-08-11T14:20:32.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:32 smithi103 ceph-mon[130434]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:20:33.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:32 smithi003 ceph-mon[97195]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:20:33.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:32 smithi187 ceph-mon[130973]: pgmap v845: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-08-11T14:20:33.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 815.0 s, 700.6 tps, lat 142.660 ms stddev 354.888 2023-08-11T14:20:35.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:34 smithi003 ceph-mon[97195]: pgmap v846: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-08-11T14:20:35.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:34 smithi187 ceph-mon[130973]: pgmap v846: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-08-11T14:20:35.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:34 smithi103 ceph-mon[130434]: pgmap v846: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.38k op/s 2023-08-11T14:20:37.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:36 smithi003 ceph-mon[97195]: pgmap v847: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:36 smithi187 ceph-mon[130973]: pgmap v847: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:37.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:36 smithi103 ceph-mon[130434]: pgmap v847: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:20:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 820.0 s, 321.0 tps, lat 303.074 ms stddev 623.785 2023-08-11T14:20:39.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:38 smithi003 ceph-mon[97195]: pgmap v848: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:20:39.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:38 smithi187 ceph-mon[130973]: pgmap v848: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:20:39.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:38 smithi103 ceph-mon[130434]: pgmap v848: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:20:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.762+0000 7f93bbbff700 1 -- 172.21.15.3:0/1764305224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b4107920 msgr2=0x7f93b4109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.762+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1764305224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b4107920 0x7f93b4109d50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009990 tx=0x7f93b0009ca0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.768+0000 7f93bbbff700 1 -- 172.21.15.3:0/1764305224 shutdown_connections 2023-08-11T14:20:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.768+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1764305224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b410cc60 0x7f93b410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.768+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1764305224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b4107920 0x7f93b4109d50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.768+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1764305224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b410a290 0x7f93b410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.768+0000 7f93bbbff700 1 -- 172.21.15.3:0/1764305224 >> 172.21.15.3:0/1764305224 conn(0x7f93b406d750 msgr2=0x7f93b406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.771+0000 7f93bbbff700 1 -- 172.21.15.3:0/1764305224 shutdown_connections 2023-08-11T14:20:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.782+0000 7f93bbbff700 1 -- 172.21.15.3:0/1764305224 wait complete. 2023-08-11T14:20:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.783+0000 7f93bbbff700 1 Processor -- start 2023-08-11T14:20:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.783+0000 7f93bbbff700 1 -- start start 2023-08-11T14:20:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b410a290 0x7f93b41a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b410cc60 0x7f93b41a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41a6f70 con 0x7f93b4107920 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41a70e0 con 0x7f93b410a290 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.784+0000 7f93bbbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41a72a0 con 0x7f93b410cc60 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.785+0000 7f93b999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.785+0000 7f93b999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54882/0 (socket says 172.21.15.3:54882) 2023-08-11T14:20:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.785+0000 7f93b999b700 1 -- 172.21.15.3:0/1545521712 learned_addr learned my addr 172.21.15.3:0/1545521712 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 -- 172.21.15.3:0/1545521712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b410cc60 msgr2=0x7f93b41a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b410cc60 0x7f93b41a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 -- 172.21.15.3:0/1545521712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b410a290 msgr2=0x7f93b41a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b410a290 0x7f93b41a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 -- 172.21.15.3:0/1545521712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00095f0 con 0x7f93b4107920 2023-08-11T14:20:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.786+0000 7f93b999b700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098e0 tx=0x7f93b000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.794+0000 7f93aaffd700 1 -- 172.21.15.3:0/1545521712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b001d070 con 0x7f93b4107920 2023-08-11T14:20:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.794+0000 7f93aaffd700 1 -- 172.21.15.3:0/1545521712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b000fe10 con 0x7f93b4107920 2023-08-11T14:20:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.794+0000 7f93aaffd700 1 -- 172.21.15.3:0/1545521712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000e640 con 0x7f93b4107920 2023-08-11T14:20:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.798+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b41a7520 con 0x7f93b4107920 2023-08-11T14:20:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.798+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b41a7770 con 0x7f93b4107920 2023-08-11T14:20:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.804+0000 7f93cf562b80 1 Processor -- start 2023-08-11T14:20:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 -- start start 2023-08-11T14:20:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f936e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f936ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f92c65d0 con 0x55a2f936d8a0 2023-08-11T14:20:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f92c3ef0 con 0x55a2f936e790 2023-08-11T14:20:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.805+0000 7f93cf562b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f92bf7e0 con 0x55a2f936cf40 2023-08-11T14:20:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.806+0000 7f93ba19c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.807+0000 7f93ba19c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:34352/0 (socket says 172.21.15.3:34352) 2023-08-11T14:20:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.807+0000 7f93ba19c700 1 -- 172.21.15.3:0/2904552647 learned_addr learned my addr 172.21.15.3:0/2904552647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.810+0000 7f93ba19c700 1 -- 172.21.15.3:0/2904552647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 msgr2=0x55a2f936ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.810+0000 7f93ba19c700 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f936ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.810+0000 7f93ba19c700 1 -- 172.21.15.3:0/2904552647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 msgr2=0x55a2f936e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.810+0000 7f93ba19c700 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f936e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.810+0000 7f93ba19c700 1 -- 172.21.15.3:0/2904552647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f913f5d0 con 0x55a2f936cf40 2023-08-11T14:20:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.827+0000 7f93ba19c700 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e400 tx=0x7f93ac00e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de95a5a4eb08c580 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.833+0000 7f93aaffd700 1 -- 172.21.15.3:0/1545521712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f93b00173f0 con 0x7f93b4107920 2023-08-11T14:20:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.833+0000 7f939ffff700 1 -- 172.21.15.3:0/2904552647 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac018070 con 0x55a2f936cf40 2023-08-11T14:20:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.833+0000 7f939ffff700 1 -- 172.21.15.3:0/2904552647 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac00f040 con 0x55a2f936cf40 2023-08-11T14:20:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.833+0000 7f939ffff700 1 -- 172.21.15.3:0/2904552647 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac004280 con 0x55a2f936cf40 2023-08-11T14:20:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.834+0000 7f93cf562b80 1 -- 172.21.15.3:0/2904552647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 msgr2=0x55a2f936d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.834+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e400 tx=0x7f93ac00e710 comp rx=0 tx=0).stop 2023-08-11T14:20:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.840+0000 7f93cf562b80 1 -- 172.21.15.3:0/2904552647 shutdown_connections 2023-08-11T14:20:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.840+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f936ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.840+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f936d360 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.840+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2904552647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f936e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.840+0000 7f93cf562b80 1 -- 172.21.15.3:0/2904552647 >> 172.21.15.3:0/2904552647 conn(0x55a2f929a8f0 msgr2=0x55a2f9369920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.841+0000 7f93cf562b80 1 -- 172.21.15.3:0/2904552647 shutdown_connections 2023-08-11T14:20:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.841+0000 7f93cf562b80 1 -- 172.21.15.3:0/2904552647 wait complete. 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.843+0000 7f93cf562b80 1 -- start start 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 Processor -- start 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 0x55a2f938f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f925da60 con 0x55a2f936e790 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f925d760 con 0x55a2f936cf40 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.844+0000 7f93cf562b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f91418c0 con 0x55a2f936d8a0 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.845+0000 7f93cf562b80 10 client.0 _ll_register_callbacks cb 0x55a2f9260a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.846+0000 7f93b919a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 0x55a2f938f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.846+0000 7f93b919a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 0x55a2f938f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54890/0 (socket says 172.21.15.3:54890) 2023-08-11T14:20:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.846+0000 7f93b919a700 1 -- 172.21.15.3:0/2962742228 learned_addr learned my addr 172.21.15.3:0/2962742228 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.854+0000 7f93aaffd700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 0x7f93a00a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.854+0000 7f93aaffd700 1 -- 172.21.15.3:0/1545521712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f93b00cd120 con 0x7f93b4107920 2023-08-11T14:20:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.857+0000 7f93b999b700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.857+0000 7f93ba19c700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.859+0000 7f93ba19c700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 0x7f93a00a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.860+0000 7f93b999b700 1 -- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 msgr2=0x55a2f9387c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.860+0000 7f93b999b700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.860+0000 7f93b999b700 1 -- 172.21.15.3:0/2962742228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 msgr2=0x55a2f938f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.860+0000 7f93b999b700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 0x55a2f938f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.860+0000 7f93b999b700 1 -- 172.21.15.3:0/2962742228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac00e140 con 0x55a2f936d8a0 2023-08-11T14:20:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.882+0000 7f93b999b700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f93b0000c00 tx=0x7f93b0017980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78455840406b1b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.882+0000 7f939d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:20:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.893+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0099ff0 con 0x55a2f936d8a0 2023-08-11T14:20:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.893+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f93b009a900 con 0x55a2f936d8a0 2023-08-11T14:20:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.893+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f93607e0 con 0x55a2f936d8a0 2023-08-11T14:20:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.896+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b009e730 con 0x55a2f936d8a0 2023-08-11T14:20:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.897+0000 7f93ba19c700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 0x7f93a00a2520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e3d0 tx=0x7f93ac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.897+0000 7f93cf562b80 10 client.26815 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.897+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55a2f90ebd80 con 0x55a2f936d8a0 2023-08-11T14:20:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2f93607e0 con 0x55a2f936d8a0 2023-08-11T14:20:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93b0096890 con 0x55a2f936d8a0 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 10 client.26815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 20 client.26815 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f93934b0 0x55a2f93958a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 4 client.26815 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:20:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2f933c480 con 0x55a2f93934b0 2023-08-11T14:20:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.923+0000 7f93b919a700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f93934b0 0x55a2f93958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.940+0000 7f93b919a700 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f93934b0 0x55a2f93958a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8b762d69cac4b7 server_cookie=12066eda80962ecd in_seq=0 out_seq=0 2023-08-11T14:20:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.940+0000 7f939d7fa700 10 client.26815 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.951+0000 7f939d7fa700 1 -- 172.21.15.3:0/2962742228 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a2f933c480 con 0x55a2f93934b0 2023-08-11T14:20:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.951+0000 7f939d7fa700 10 client.26815 handle_command_reply: tid=0 2023-08-11T14:20:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.952+0000 7f93cf562b80 1 client.26815 shutdown 2023-08-11T14:20:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.957+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 msgr2=0x55a2f93881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.957+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f93b0000c00 tx=0x7f93b0017980 comp rx=0 tx=0).stop 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 shutdown_connections 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936e790 0x55a2f938f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f93934b0 0x55a2f93958a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 --2- 172.21.15.3:0/2962742228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 >> 172.21.15.3:0/2962742228 conn(0x55a2f929a8f0 msgr2=0x55a2f9368710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f939d7fa700 0 client.26815 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.958+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 shutdown_connections 2023-08-11T14:20:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.959+0000 7f93cf562b80 1 -- 172.21.15.3:0/2962742228 wait complete. 2023-08-11T14:20:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.959+0000 7f93cf562b80 20 client.26815 trim_cache size 0 max 16384 2023-08-11T14:20:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.996+0000 7f93cf562b80 1 Processor -- start 2023-08-11T14:20:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 -- start start 2023-08-11T14:20:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 0x55a2f938f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f9362ea0 con 0x55a2f936d8a0 2023-08-11T14:20:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f93871c0 con 0x55a2f936cf40 2023-08-11T14:20:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.997+0000 7f93cf562b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f93607c0 con 0x55a2f936e790 2023-08-11T14:20:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.998+0000 7f93b999b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.998+0000 7f93b999b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50150/0 (socket says 172.21.15.3:50150) 2023-08-11T14:20:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.998+0000 7f93b999b700 1 -- 172.21.15.3:0/3398883243 learned_addr learned my addr 172.21.15.3:0/3398883243 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:39.998+0000 7f93ba19c700 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 0x55a2f938f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.001+0000 7f93ba19c700 1 -- 172.21.15.3:0/3398883243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 msgr2=0x55a2f9387c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.001+0000 7f93ba19c700 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.001+0000 7f93ba19c700 1 -- 172.21.15.3:0/3398883243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 msgr2=0x55a2f93881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.001+0000 7f93ba19c700 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.001+0000 7f93ba19c700 1 -- 172.21.15.3:0/3398883243 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f9364a10 con 0x55a2f936e790 2023-08-11T14:20:40.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.008+0000 7f93ba19c700 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 0x55a2f938f1b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009c10 tx=0x7f93ac0090d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb4d8802b1220dc server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.021+0000 7f93a8bff700 1 -- 172.21.15.3:0/3398883243 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac027070 con 0x55a2f936e790 2023-08-11T14:20:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.021+0000 7f93a8bff700 1 -- 172.21.15.3:0/3398883243 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac007bc0 con 0x55a2f936e790 2023-08-11T14:20:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.021+0000 7f93a8bff700 1 -- 172.21.15.3:0/3398883243 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac018040 con 0x55a2f936e790 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.022+0000 7f93cf562b80 1 -- 172.21.15.3:0/3398883243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 msgr2=0x55a2f938f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.022+0000 7f93cf562b80 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 0x55a2f938f1b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009c10 tx=0x7f93ac0090d0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.031+0000 7f93cf562b80 1 -- 172.21.15.3:0/3398883243 shutdown_connections 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.031+0000 7f93cf562b80 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936e790 0x55a2f938f1b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.031+0000 7f93cf562b80 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936cf40 0x55a2f9387c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.031+0000 7f93cf562b80 1 --2- 172.21.15.3:0/3398883243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f93881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.031+0000 7f93cf562b80 1 -- 172.21.15.3:0/3398883243 >> 172.21.15.3:0/3398883243 conn(0x55a2f929a8f0 msgr2=0x55a2f938fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.040+0000 7f93cf562b80 1 -- 172.21.15.3:0/3398883243 shutdown_connections 2023-08-11T14:20:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.040+0000 7f93cf562b80 1 -- 172.21.15.3:0/3398883243 wait complete. 2023-08-11T14:20:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.040+0000 7f93cf562b80 1 -- start start 2023-08-11T14:20:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 Processor -- start 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f938ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f938d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f938e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f938e8a0 con 0x55a2f936d8a0 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f938ea10 con 0x55a2f936e790 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.041+0000 7f93cf562b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f938ebd0 con 0x55a2f936cf40 2023-08-11T14:20:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.042+0000 7f93cf562b80 10 client.0 _ll_register_callbacks cb 0x55a2f9260a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.050+0000 7f93ba19c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f938e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.050+0000 7f93ba19c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f938e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:50156/0 (socket says 172.21.15.3:50156) 2023-08-11T14:20:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.050+0000 7f93ba19c700 1 -- 172.21.15.3:0/1328553368 learned_addr learned my addr 172.21.15.3:0/1328553368 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.050+0000 7f93b919a700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f938d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.051+0000 7f93b999b700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f938ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.060+0000 7f93b999b700 1 -- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 msgr2=0x55a2f938e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.060+0000 7f93b999b700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f938e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.060+0000 7f93b999b700 1 -- 172.21.15.3:0/1328553368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 msgr2=0x55a2f938d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.060+0000 7f93b999b700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f938d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.060+0000 7f93b999b700 1 -- 172.21.15.3:0/1328553368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2f913f5d0 con 0x55a2f936cf40 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.071+0000 7f93b999b700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f938ce10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93b0097320 tx=0x7f93b00094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc7e81e5d927c8e server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.072+0000 7f939d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:20:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.085+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0099ad0 con 0x55a2f936cf40 2023-08-11T14:20:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.085+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f93b009a900 con 0x55a2f936cf40 2023-08-11T14:20:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.087+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93ac00f040 con 0x55a2f936cf40 2023-08-11T14:20:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.096+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b009e730 con 0x55a2f936cf40 2023-08-11T14:20:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.099+0000 7f93cf562b80 10 client.26821 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.099+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55a2f9387000 con 0x55a2f936cf40 2023-08-11T14:20:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93ac00f040 con 0x55a2f936cf40 2023-08-11T14:20:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f93b00c49e0 con 0x55a2f936cf40 2023-08-11T14:20:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 20 client.26821 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f9375810 0x55a2f9377c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 4 client.26821 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:20:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.105+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a2f92b87c0 con 0x55a2f9375810 2023-08-11T14:20:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.109+0000 7f93b919a700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f9375810 0x55a2f9377c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.129+0000 7f93b919a700 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f9375810 0x55a2f9377c30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e728f152c6db7c server_cookie=61acc507a4c88af in_seq=0 out_seq=0 2023-08-11T14:20:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.129+0000 7f939d7fa700 10 client.26821 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:40.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:40 smithi003 ceph-mon[97195]: pgmap v849: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:20:40.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:40 smithi187 ceph-mon[130973]: pgmap v849: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:20:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:40 smithi103 ceph-mon[130434]: pgmap v849: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 1.26k op/s 2023-08-11T14:20:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.834+0000 7f939d7fa700 1 -- 172.21.15.3:0/1328553368 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a2f92b87c0 con 0x55a2f9375810 2023-08-11T14:20:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.834+0000 7f939d7fa700 10 client.26821 handle_command_reply: tid=0 2023-08-11T14:20:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.834+0000 7f93cf562b80 1 client.26821 shutdown 2023-08-11T14:20:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 msgr2=0x55a2f938ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f938ce10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93b0097320 tx=0x7f93b00094b0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 shutdown_connections 2023-08-11T14:20:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55a2f9375810 0x55a2f9377c30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a2f936e790 0x55a2f938e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2f936cf40 0x55a2f938ce10 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93b0097320 tx=0x7f93b00094b0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 --2- 172.21.15.3:0/1328553368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2f936d8a0 0x55a2f938d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.835+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 >> 172.21.15.3:0/1328553368 conn(0x55a2f929a8f0 msgr2=0x55a2f9371760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.836+0000 7f939d7fa700 0 client.26821 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.836+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 shutdown_connections 2023-08-11T14:20:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.836+0000 7f93cf562b80 1 -- 172.21.15.3:0/1328553368 wait complete. 2023-08-11T14:20:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.836+0000 7f93cf562b80 20 client.26821 trim_cache size 0 max 16384 2023-08-11T14:20:40.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:20:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:20:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:20:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:20:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 msgr2=0x7f93a00a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 0x7f93a00a2520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e3d0 tx=0x7f93ac00b040 comp rx=0 tx=0).stop 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 msgr2=0x7f93b41a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098e0 tx=0x7f93b000f740 comp rx=0 tx=0).stop 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 shutdown_connections 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b410cc60 0x7f93b41a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93b4107920 0x7f93b41a5550 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93a00a0060 0x7f93a00a2520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.837+0000 7f93bbbff700 1 --2- 172.21.15.3:0/1545521712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b410a290 0x7f93b41a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.838+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 >> 172.21.15.3:0/1545521712 conn(0x7f93b406d750 msgr2=0x7f93b410baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.838+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 shutdown_connections 2023-08-11T14:20:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:40.838+0000 7f93bbbff700 1 -- 172.21.15.3:0/1545521712 wait complete. 2023-08-11T14:20:42.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:42 smithi103 ceph-mon[130434]: pgmap v850: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:20:43.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:42 smithi003 ceph-mon[97195]: pgmap v850: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:20:43.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:42 smithi187 ceph-mon[130973]: pgmap v850: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-08-11T14:20:43.088 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 825.0 s, 521.7 tps, lat 196.444 ms stddev 260.587 2023-08-11T14:20:44.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:44 smithi103 ceph-mon[130434]: pgmap v851: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:20:45.046 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:44 smithi003 ceph-mon[97195]: pgmap v851: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:20:45.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:44 smithi187 ceph-mon[130973]: pgmap v851: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:20:47.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:46 smithi003 ceph-mon[97195]: pgmap v852: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 935 op/s 2023-08-11T14:20:47.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:46 smithi187 ceph-mon[130973]: pgmap v852: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 935 op/s 2023-08-11T14:20:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:46 smithi103 ceph-mon[130434]: pgmap v852: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 12 MiB/s wr, 935 op/s 2023-08-11T14:20:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 830.0 s, 657.9 tps, lat 151.975 ms stddev 248.619 2023-08-11T14:20:49.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:48 smithi003 ceph-mon[97195]: pgmap v853: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 906 op/s 2023-08-11T14:20:49.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:48 smithi187 ceph-mon[130973]: pgmap v853: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 906 op/s 2023-08-11T14:20:49.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:48 smithi103 ceph-mon[130434]: pgmap v853: 129 pgs: 129 active+clean; 8.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 906 op/s 2023-08-11T14:20:50.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:50 smithi103 ceph-mon[130434]: pgmap v854: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 971 op/s 2023-08-11T14:20:50.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:50 smithi187 ceph-mon[130973]: pgmap v854: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 971 op/s 2023-08-11T14:20:50.546 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:50 smithi003 ceph-mon[97195]: pgmap v854: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 971 op/s 2023-08-11T14:20:51.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:51 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:20:51.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:51 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:20:51.933 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:51 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:20:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:52 smithi003 ceph-mon[97195]: pgmap v855: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 800 op/s 2023-08-11T14:20:52.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:52 smithi187 ceph-mon[130973]: pgmap v855: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 800 op/s 2023-08-11T14:20:52.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:52 smithi103 ceph-mon[130434]: pgmap v855: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 800 op/s 2023-08-11T14:20:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 835.0 s, 714.7 tps, lat 91.291 ms stddev 96.123 2023-08-11T14:20:53.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:53 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:53 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.092 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:53 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 -- 172.21.15.3:0/350145529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10cc60 msgr2=0x7f1ecc10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/350145529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc10d0e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009990 tx=0x7f1ec4009ca0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 -- 172.21.15.3:0/350145529 shutdown_connections 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/350145529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc10d0e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/350145529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc107920 0x7f1ecc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.127+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/350145529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc10a290 0x7f1ecc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.128+0000 7f1ed0bff700 1 -- 172.21.15.3:0/350145529 >> 172.21.15.3:0/350145529 conn(0x7f1ecc06d750 msgr2=0x7f1ecc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.128+0000 7f1ed0bff700 1 -- 172.21.15.3:0/350145529 shutdown_connections 2023-08-11T14:20:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.128+0000 7f1ed0bff700 1 -- 172.21.15.3:0/350145529 wait complete. 2023-08-11T14:20:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.129+0000 7f1ed0bff700 1 Processor -- start 2023-08-11T14:20:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 -- start start 2023-08-11T14:20:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc107920 0x7f1ecc113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10a290 0x7f1ecc114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc115960 con 0x7f1ecc10a290 2023-08-11T14:20:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc115ad0 con 0x7f1ecc10cc60 2023-08-11T14:20:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.130+0000 7f1ed0bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc115c90 con 0x7f1ecc107920 2023-08-11T14:20:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.131+0000 7f1ecad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.131+0000 7f1ecad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45672/0 (socket says 172.21.15.3:45672) 2023-08-11T14:20:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.131+0000 7f1ecad9d700 1 -- 172.21.15.3:0/821729147 learned_addr learned my addr 172.21.15.3:0/821729147 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.131+0000 7f1eca59c700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc107920 0x7f1ecc113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.131+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10a290 0x7f1ecc114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.132+0000 7f1ecad9d700 1 -- 172.21.15.3:0/821729147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc107920 msgr2=0x7f1ecc113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.132+0000 7f1ecad9d700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc107920 0x7f1ecc113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.132+0000 7f1ecad9d700 1 -- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10a290 msgr2=0x7f1ecc114430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.132+0000 7f1ecad9d700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10a290 0x7f1ecc114430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.132+0000 7f1ecad9d700 1 -- 172.21.15.3:0/821729147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40095f0 con 0x7f1ecc10cc60 2023-08-11T14:20:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.133+0000 7f1ecad9d700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec4003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.135+0000 7f1ec3fff700 1 -- 172.21.15.3:0/821729147 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec401d070 con 0x7f1ecc10cc60 2023-08-11T14:20:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.135+0000 7f1ec3fff700 1 -- 172.21.15.3:0/821729147 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ec400f460 con 0x7f1ecc10cc60 2023-08-11T14:20:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.135+0000 7f1ec3fff700 1 -- 172.21.15.3:0/821729147 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4021670 con 0x7f1ecc10cc60 2023-08-11T14:20:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.136+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc115f10 con 0x7f1ecc10cc60 2023-08-11T14:20:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.136+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc116720 con 0x7f1ecc10cc60 2023-08-11T14:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.144+0000 7f1ec3fff700 1 -- 172.21.15.3:0/821729147 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1ec400bab0 con 0x7f1ecc10cc60 2023-08-11T14:20:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.145+0000 7f1ee4530b80 1 Processor -- start 2023-08-11T14:20:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 -- start start 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8396e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8396ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d838c65d0 con 0x555d8396e790 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d838c3ef0 con 0x555d8396d8a0 2023-08-11T14:20:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.152+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d838bf7e0 con 0x555d8396cf40 2023-08-11T14:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.153+0000 7f1ec1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.153+0000 7f1ec1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51548/0 (socket says 172.21.15.3:51548) 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.153+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/428495837 learned_addr learned my addr 172.21.15.3:0/428495837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/428495837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 msgr2=0x555d8396e1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8396e1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/428495837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 msgr2=0x555d8396ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8396ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/428495837 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d8373f5d0 con 0x555d8396cf40 2023-08-11T14:20:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.154+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005c50 tx=0x7f1eb4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fa411a73ab30a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.156+0000 7f1ec0d99700 1 -- 172.21.15.3:0/428495837 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4016070 con 0x555d8396cf40 2023-08-11T14:20:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.156+0000 7f1ec0d99700 1 -- 172.21.15.3:0/428495837 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb40044e0 con 0x555d8396cf40 2023-08-11T14:20:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.156+0000 7f1ec0d99700 1 -- 172.21.15.3:0/428495837 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4003cf0 con 0x555d8396cf40 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.156+0000 7f1ec3fff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 0x7f1eb80a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ec3fff700 1 -- 172.21.15.3:0/821729147 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1ec40cc7e0 con 0x7f1ecc10cc60 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 -- 172.21.15.3:0/428495837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 msgr2=0x555d8396d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005c50 tx=0x7f1eb4011720 comp rx=0 tx=0).stop 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 -- 172.21.15.3:0/428495837 shutdown_connections 2023-08-11T14:20:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8396ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d8396d360 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/428495837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8396e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1ee4530b80 1 -- 172.21.15.3:0/428495837 >> 172.21.15.3:0/428495837 conn(0x555d8389a8f0 msgr2=0x555d83969920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.157+0000 7f1eca59c700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 0x7f1eb80a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.159+0000 7f1eca59c700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 0x7f1eb80a2610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x555d839692f0 tx=0x7f1ebc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.160+0000 7f1ee4530b80 1 -- 172.21.15.3:0/428495837 shutdown_connections 2023-08-11T14:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.160+0000 7f1ee4530b80 1 -- 172.21.15.3:0/428495837 wait complete. 2023-08-11T14:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.161+0000 7f1ee4530b80 1 -- start start 2023-08-11T14:20:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.162+0000 7f1ee4530b80 1 Processor -- start 2023-08-11T14:20:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d839771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8397e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8385da60 con 0x555d8396e790 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8385d760 con 0x555d8396d8a0 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d837418c0 con 0x555d8396cf40 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1ee4530b80 10 client.0 _ll_register_callbacks cb 0x555d83860a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1eca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1eca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51556/0 (socket says 172.21.15.3:51556) 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.163+0000 7f1eca59c700 1 -- 172.21.15.3:0/3096636007 learned_addr learned my addr 172.21.15.3:0/3096636007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d839771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1eca59c700 1 -- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 msgr2=0x555d839771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1eca59c700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d839771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1eca59c700 1 -- 172.21.15.3:0/3096636007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 msgr2=0x555d8397e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1eca59c700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8397e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.164+0000 7f1eca59c700 1 -- 172.21.15.3:0/3096636007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb4009a90 con 0x555d8396cf40 2023-08-11T14:20:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.165+0000 7f1eca59c700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc016c00 tx=0x7f1ebc004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4381a6e2dbe170d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.165+0000 7f1eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:20:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.165+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc010040 con 0x555d8396cf40 2023-08-11T14:20:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.166+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1ebc015980 con 0x555d8396cf40 2023-08-11T14:20:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.166+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d839607e0 con 0x555d8396cf40 2023-08-11T14:20:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.167+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ebc015b40 con 0x555d8396cf40 2023-08-11T14:20:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.167+0000 7f1ee4530b80 10 client.26833 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.167+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x555d836ebd80 con 0x555d8396cf40 2023-08-11T14:20:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.167+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d839607e0 con 0x555d8396cf40 2023-08-11T14:20:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.167+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ebc01f070 con 0x555d8396cf40 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 10 client.26833 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 10 client.26833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 20 client.26833 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d83982c30 0x555d83985050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 4 client.26833 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d8393c480 con 0x555d83982c30 2023-08-11T14:20:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.168+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d83982c30 0x555d83985050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.169+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d83982c30 0x555d83985050 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc191dd9d9ea6d server_cookie=e081786d26104eee in_seq=0 out_seq=0 2023-08-11T14:20:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.169+0000 7f1eb1ffb700 10 client.26833 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.169+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/3096636007 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555d8393c480 con 0x555d83982c30 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.169+0000 7f1eb1ffb700 10 client.26833 handle_command_reply: tid=0 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 client.26833 shutdown 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 msgr2=0x555d83976ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc016c00 tx=0x7f1ebc004b60 comp rx=0 tx=0).stop 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 shutdown_connections 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396e790 0x555d8397e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396cf40 0x555d83976ca0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d83982c30 0x555d83985050 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/3096636007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d839771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 >> 172.21.15.3:0/3096636007 conn(0x555d8389a8f0 msgr2=0x555d839686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1eb1ffb700 0 client.26833 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.170+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 shutdown_connections 2023-08-11T14:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.171+0000 7f1ee4530b80 1 -- 172.21.15.3:0/3096636007 wait complete. 2023-08-11T14:20:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.171+0000 7f1ee4530b80 20 client.26833 trim_cache size 0 max 16384 2023-08-11T14:20:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.200+0000 7f1ee4530b80 1 Processor -- start 2023-08-11T14:20:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.200+0000 7f1ee4530b80 1 -- start start 2023-08-11T14:20:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396d8a0 0x555d8397e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d83969860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d83968d50 con 0x555d8396d8a0 2023-08-11T14:20:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8396de10 con 0x555d8396cf40 2023-08-11T14:20:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8396cb60 con 0x555d8396e790 2023-08-11T14:20:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.201+0000 7f1eca59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45706/0 (socket says 172.21.15.3:45706) 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 -- 172.21.15.3:0/4275790380 learned_addr learned my addr 172.21.15.3:0/4275790380 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1ecad9d700 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d83969860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396d8a0 0x555d8397e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 -- 172.21.15.3:0/4275790380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 msgr2=0x555d83969860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d83969860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 -- 172.21.15.3:0/4275790380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396d8a0 msgr2=0x555d8397e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396d8a0 0x555d8397e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 -- 172.21.15.3:0/4275790380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d836d4c00 con 0x555d8396cf40 2023-08-11T14:20:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.202+0000 7f1eca59c700 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009e30 tx=0x7f1ebc009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc069215b6e2b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.203+0000 7f1ec159a700 1 -- 172.21.15.3:0/4275790380 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc010040 con 0x555d8396cf40 2023-08-11T14:20:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.203+0000 7f1ec159a700 1 -- 172.21.15.3:0/4275790380 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc01c040 con 0x555d8396cf40 2023-08-11T14:20:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.203+0000 7f1ec159a700 1 -- 172.21.15.3:0/4275790380 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc015940 con 0x555d8396cf40 2023-08-11T14:20:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 -- 172.21.15.3:0/4275790380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 msgr2=0x555d83969320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009e30 tx=0x7f1ebc009e60 comp rx=0 tx=0).stop 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 -- 172.21.15.3:0/4275790380 shutdown_connections 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d83969860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396cf40 0x555d83969320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/4275790380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396d8a0 0x555d8397e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 -- 172.21.15.3:0/4275790380 >> 172.21.15.3:0/4275790380 conn(0x555d8389a8f0 msgr2=0x555d83971670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 -- 172.21.15.3:0/4275790380 shutdown_connections 2023-08-11T14:20:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.204+0000 7f1ee4530b80 1 -- 172.21.15.3:0/4275790380 wait complete. 2023-08-11T14:20:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.205+0000 7f1ee4530b80 1 -- start start 2023-08-11T14:20:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.205+0000 7f1ee4530b80 1 Processor -- start 2023-08-11T14:20:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396cf40 0x555d8397c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d8397d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8397db30 con 0x555d8396cf40 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8397dca0 con 0x555d8396d8a0 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d8397de60 con 0x555d8396e790 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.206+0000 7f1ee4530b80 10 client.0 _ll_register_callbacks cb 0x7f1ebc0167b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:20:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ecad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d8397d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:45718/0 (socket says 172.21.15.3:45718) 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/2972640775 learned_addr learned my addr 172.21.15.3:0/2972640775 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ecad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d8397d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51572/0 (socket says 172.21.15.3:51572) 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1eca59c700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396cf40 0x555d8397c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/2972640775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 msgr2=0x555d8397d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d8397d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/2972640775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396cf40 msgr2=0x555d8397c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:20:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396cf40 0x555d8397c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 -- 172.21.15.3:0/2972640775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d836ebd80 con 0x555d8396d8a0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1ec1d9b700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009df0 tx=0x7f1eb4005210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7423e00a6e05e1e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.207+0000 7f1eb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4016040 con 0x555d8396d8a0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f1eb4004850 con 0x555d8396d8a0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1eb40044e0 con 0x555d8396d8a0 2023-08-11T14:20:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb4004a10 con 0x555d8396d8a0 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1ee4530b80 10 client.26816 fetch_fsmap learned FSMap version 58 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.208+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x555d837121f0 con 0x555d8396d8a0 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb40044e0 con 0x555d8396d8a0 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1eb4011440 con 0x555d8396d8a0 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 20 client.26816 populate_metadata read hostname 'smithi003' 2023-08-11T14:20:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d839d7960 0x555d839d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:20:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 4 client.26816 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:20:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d838b87c0 con 0x555d839d7960 2023-08-11T14:20:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.209+0000 7f1eca59c700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d839d7960 0x555d839d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:20:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.210+0000 7f1eca59c700 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d839d7960 0x555d839d9d80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febe4a1f82f3630b server_cookie=be28c9a4d6fc88f1 in_seq=0 out_seq=0 2023-08-11T14:20:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.210+0000 7f1eb1ffb700 10 client.26816 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.228+0000 7f1eb1ffb700 1 -- 172.21.15.3:0/2972640775 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d838b87c0 con 0x555d839d7960 2023-08-11T14:20:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.228+0000 7f1eb1ffb700 10 client.26816 handle_command_reply: tid=0 2023-08-11T14:20:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.229+0000 7f1ee4530b80 1 client.26816 shutdown 2023-08-11T14:20:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 msgr2=0x555d8397c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009df0 tx=0x7f1eb4005210 comp rx=0 tx=0).stop 2023-08-11T14:20:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 shutdown_connections 2023-08-11T14:20:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555d8396e790 0x555d8397d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555d8396cf40 0x555d8397c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x555d839d7960 0x555d839d9d80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 --2- 172.21.15.3:0/2972640775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8396d8a0 0x555d8397c5e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 >> 172.21.15.3:0/2972640775 conn(0x555d8389a8f0 msgr2=0x555d83969320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1eb1ffb700 0 client.26816 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:20:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 shutdown_connections 2023-08-11T14:20:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.231+0000 7f1ee4530b80 1 -- 172.21.15.3:0/2972640775 wait complete. 2023-08-11T14:20:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:20:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:20:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:20:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:20:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.232+0000 7f1ee4530b80 20 client.26816 trim_cache size 0 max 16384 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 msgr2=0x7f1eb80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 0x7f1eb80a2610 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x555d839692f0 tx=0x7f1ebc00d040 comp rx=0 tx=0).stop 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 msgr2=0x7f1ecc115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec4003e50 comp rx=0 tx=0).stop 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 shutdown_connections 2023-08-11T14:20:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1eb80a0150 0x7f1eb80a2610 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ecc10cc60 0x7f1ecc115300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ecc107920 0x7f1ecc113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 --2- 172.21.15.3:0/821729147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ecc10a290 0x7f1ecc114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 >> 172.21.15.3:0/821729147 conn(0x7f1ecc06d750 msgr2=0x7f1ecc1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 shutdown_connections 2023-08-11T14:20:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:20:54.233+0000 7f1ed0bff700 1 -- 172.21.15.3:0/821729147 wait complete. 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: pgmap v856: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:20:54.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:20:54.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:20:54.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:20:54.931 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:20:55.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: pgmap v856: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:20:55.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:20:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:20:55.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: pgmap v856: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi003 to 222.0M 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi003 to 232785100: error parsing value: Value '232785100' is below minimum 939524096 2023-08-11T14:20:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:20:55.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:20:55.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:20:56.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:56 smithi103 ceph-mon[130434]: pgmap v857: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:57.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:56 smithi003 ceph-mon[97195]: pgmap v857: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:57.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:56 smithi187 ceph-mon[130973]: pgmap v857: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-08-11T14:20:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 840.0 s, 1025.9 tps, lat 130.358 ms stddev 331.906 2023-08-11T14:20:58.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:20:58 smithi103 ceph-mon[130434]: pgmap v858: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:20:59.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:20:58 smithi003 ceph-mon[97195]: pgmap v858: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:20:59.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:20:58 smithi187 ceph-mon[130973]: pgmap v858: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.10k op/s 2023-08-11T14:21:00.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:00 smithi003 ceph-mon[97195]: pgmap v859: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:21:00.551 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:00 smithi187 ceph-mon[130973]: pgmap v859: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:21:00.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:00 smithi103 ceph-mon[130434]: pgmap v859: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-08-11T14:21:02.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:02 smithi103 ceph-mon[130434]: pgmap v860: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-08-11T14:21:03.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:02 smithi003 ceph-mon[97195]: pgmap v860: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-08-11T14:21:03.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:02 smithi187 ceph-mon[130973]: pgmap v860: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 16 MiB/s wr, 1.09k op/s 2023-08-11T14:21:03.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 845.0 s, 380.0 tps, lat 266.140 ms stddev 443.471 2023-08-11T14:21:04.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:04 smithi103 ceph-mon[130434]: pgmap v861: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-08-11T14:21:05.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:04 smithi003 ceph-mon[97195]: pgmap v861: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-08-11T14:21:05.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:04 smithi187 ceph-mon[130973]: pgmap v861: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-08-11T14:21:07.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:06 smithi003 ceph-mon[97195]: pgmap v862: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:07.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:06 smithi187 ceph-mon[130973]: pgmap v862: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:07.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:06 smithi103 ceph-mon[130434]: pgmap v862: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.521+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/3531968999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410a290 msgr2=0x7f4a5410c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.521+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/3531968999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410a290 0x7f4a5410c720 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001a9a0 tx=0x7f4a5001acb0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.523+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/3531968999 shutdown_connections 2023-08-11T14:21:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.523+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/3531968999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5410cc60 0x7f4a5410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.523+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/3531968999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a54107920 0x7f4a54109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.523+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/3531968999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410a290 0x7f4a5410c720 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.523+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/3531968999 >> 172.21.15.3:0/3531968999 conn(0x7f4a5406d750 msgr2=0x7f4a5406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.529+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/3531968999 shutdown_connections 2023-08-11T14:21:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.529+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/3531968999 wait complete. 2023-08-11T14:21:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.530+0000 7f4a5c7ff700 1 Processor -- start 2023-08-11T14:21:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.531+0000 7f4a5c7ff700 1 -- start start 2023-08-11T14:21:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a5410a290 0x7f4a5419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 0x7f4a5419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419e860 con 0x7f4a5410a290 2023-08-11T14:21:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419e9d0 con 0x7f4a5410cc60 2023-08-11T14:21:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.538+0000 7f4a5c7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419eb90 con 0x7f4a54107920 2023-08-11T14:21:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.539+0000 7f4a5a59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.539+0000 7f4a5a59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:37332/0 (socket says 172.21.15.3:37332) 2023-08-11T14:21:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.539+0000 7f4a5a59b700 1 -- 172.21.15.3:0/393046931 learned_addr learned my addr 172.21.15.3:0/393046931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a59d9a700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a5410a290 0x7f4a5419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5a59b700 1 -- 172.21.15.3:0/393046931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 msgr2=0x7f4a5419e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 0x7f4a5419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 0x7f4a5419e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5a59b700 1 -- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a5410a290 msgr2=0x7f4a5419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a5410a290 0x7f4a5419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.540+0000 7f4a5a59b700 1 -- 172.21.15.3:0/393046931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5001a600 con 0x7f4a54107920 2023-08-11T14:21:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.541+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00ba50 tx=0x7f4a4c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.546+0000 7f4a4b7fe700 1 -- 172.21.15.3:0/393046931 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00cab0 con 0x7f4a54107920 2023-08-11T14:21:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.547+0000 7f4a4b7fe700 1 -- 172.21.15.3:0/393046931 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c00cc10 con 0x7f4a54107920 2023-08-11T14:21:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.547+0000 7f4a4b7fe700 1 -- 172.21.15.3:0/393046931 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c007660 con 0x7f4a54107920 2023-08-11T14:21:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.547+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5419ee70 con 0x7f4a54107920 2023-08-11T14:21:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.547+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a541a5600 con 0x7f4a54107920 2023-08-11T14:21:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.548+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 0x7f4a5419e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:21:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.554+0000 7f4a4b7fe700 1 -- 172.21.15.3:0/393046931 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4a4c014440 con 0x7f4a54107920 2023-08-11T14:21:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.555+0000 7f4a700d0b80 1 Processor -- start 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.558+0000 7f4a4b7fe700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 0x7f4a400a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.558+0000 7f4a4b7fe700 1 -- 172.21.15.3:0/393046931 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4a4c006e40 con 0x7f4a54107920 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 -- start start 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 0x55d86e204360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e2051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e205790 0x55d86e205c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e15d5d0 con 0x55d86e205790 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e15aef0 con 0x55d86e2048a0 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.559+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e1567e0 con 0x55d86e203f40 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.560+0000 7f4a59d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e2051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.560+0000 7f4a59d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e2051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:59878/0 (socket says 172.21.15.3:59878) 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.560+0000 7f4a59d9a700 1 -- 172.21.15.3:0/642662134 learned_addr learned my addr 172.21.15.3:0/642662134 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.562+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 0x55d86e204360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.563+0000 7f4a5a59b700 1 -- 172.21.15.3:0/642662134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 msgr2=0x55d86e2051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.563+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e2051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.563+0000 7f4a5a59b700 1 -- 172.21.15.3:0/642662134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e205790 msgr2=0x55d86e205c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.563+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e205790 0x55d86e205c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.563+0000 7f4a5a59b700 1 -- 172.21.15.3:0/642662134 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86dfd65d0 con 0x55d86e203f40 2023-08-11T14:21:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.565+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 0x7f4a400a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.567+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 0x7f4a400a2610 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55d86e204f70 tx=0x7f4a44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.570+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 0x55d86e204360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0c8510 tx=0x7f4a4c097dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26d715194a3829 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.578+0000 7f4a48ff9700 1 -- 172.21.15.3:0/642662134 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c09c290 con 0x55d86e203f40 2023-08-11T14:21:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.578+0000 7f4a48ff9700 1 -- 172.21.15.3:0/642662134 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c0a0270 con 0x55d86e203f40 2023-08-11T14:21:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.578+0000 7f4a48ff9700 1 -- 172.21.15.3:0/642662134 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c09fde0 con 0x55d86e203f40 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.579+0000 7f4a700d0b80 1 -- 172.21.15.3:0/642662134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 msgr2=0x55d86e204360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.579+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 0x55d86e204360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0c8510 tx=0x7f4a4c097dc0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.580+0000 7f4a700d0b80 1 -- 172.21.15.3:0/642662134 shutdown_connections 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.580+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e205790 0x55d86e205c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.580+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e203f40 0x55d86e204360 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0c8510 tx=0x7f4a4c097dc0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.580+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/642662134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e2051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.580+0000 7f4a700d0b80 1 -- 172.21.15.3:0/642662134 >> 172.21.15.3:0/642662134 conn(0x55d86e1318f0 msgr2=0x55d86e200920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.582+0000 7f4a700d0b80 1 -- 172.21.15.3:0/642662134 shutdown_connections 2023-08-11T14:21:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.582+0000 7f4a700d0b80 1 -- 172.21.15.3:0/642662134 wait complete. 2023-08-11T14:21:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.583+0000 7f4a700d0b80 1 -- start start 2023-08-11T14:21:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.583+0000 7f4a700d0b80 1 Processor -- start 2023-08-11T14:21:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e20dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e215840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e0f4a60 con 0x55d86e2048a0 2023-08-11T14:21:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e0f4760 con 0x55d86e213450 2023-08-11T14:21:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86dfd88c0 con 0x55d86e205790 2023-08-11T14:21:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.584+0000 7f4a700d0b80 10 client.0 _ll_register_callbacks cb 0x55d86e0f7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.585+0000 7f4a59d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.585+0000 7f4a59d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:37348/0 (socket says 172.21.15.3:37348) 2023-08-11T14:21:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.585+0000 7f4a59d9a700 1 -- 172.21.15.3:0/2673924463 learned_addr learned my addr 172.21.15.3:0/2673924463 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.585+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e215840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 -- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 msgr2=0x55d86e215840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e215840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 -- 172.21.15.3:0/2673924463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 msgr2=0x55d86e20dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e20dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 -- 172.21.15.3:0/2673924463 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a541a5600 con 0x55d86e205790 2023-08-11T14:21:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.586+0000 7f4a59d9a700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001c5d0 tx=0x7f4a5001cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c06d1397743eea server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.589+0000 7f4a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:21:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.589+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50005310 con 0x55d86e205790 2023-08-11T14:21:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.589+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4a50005470 con 0x55d86e205790 2023-08-11T14:21:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.590+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d86e1f77e0 con 0x55d86e205790 2023-08-11T14:21:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.590+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a50005670 con 0x55d86e205790 2023-08-11T14:21:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.591+0000 7f4a700d0b80 10 client.26848 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.591+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d86df82d80 con 0x55d86e205790 2023-08-11T14:21:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.592+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d86e1f77e0 con 0x55d86e205790 2023-08-11T14:21:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.592+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a50003c30 con 0x55d86e205790 2023-08-11T14:21:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.593+0000 7f4a700d0b80 10 client.26848 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.593+0000 7f4a700d0b80 10 client.26848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.594+0000 7f4a700d0b80 20 client.26848 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.594+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e21a1d0 0x55d86e21c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.594+0000 7f4a700d0b80 4 client.26848 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.594+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d86e1d3480 con 0x55d86e21a1d0 2023-08-11T14:21:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.594+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e21a1d0 0x55d86e21c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.600+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e21a1d0 0x55d86e21c5f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53a651924a0c17e server_cookie=5b86be2e6846296c in_seq=0 out_seq=0 2023-08-11T14:21:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.600+0000 7f4a3dffb700 10 client.26848 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.613+0000 7f4a3dffb700 1 -- 172.21.15.3:0/2673924463 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d86e1d3480 con 0x55d86e21a1d0 2023-08-11T14:21:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.613+0000 7f4a3dffb700 10 client.26848 handle_command_reply: tid=0 2023-08-11T14:21:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.614+0000 7f4a700d0b80 1 client.26848 shutdown 2023-08-11T14:21:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.624+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 msgr2=0x55d86e20e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.624+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001c5d0 tx=0x7f4a5001cda0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 shutdown_connections 2023-08-11T14:21:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e21a1d0 0x55d86e21c5f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e215840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e20dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/2673924463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e20e420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.635+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 >> 172.21.15.3:0/2673924463 conn(0x55d86e1318f0 msgr2=0x55d86e1ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.641+0000 7f4a3dffb700 0 client.26848 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.645+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 shutdown_connections 2023-08-11T14:21:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.656+0000 7f4a700d0b80 1 -- 172.21.15.3:0/2673924463 wait complete. 2023-08-11T14:21:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.659+0000 7f4a700d0b80 20 client.26848 trim_cache size 0 max 16384 2023-08-11T14:21:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 Processor -- start 2023-08-11T14:21:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 -- start start 2023-08-11T14:21:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e229c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e208900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e203be0 con 0x55d86e2048a0 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e205040 con 0x55d86e213450 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.691+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e200010 con 0x55d86e205790 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.692+0000 7f4a5a59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.692+0000 7f4a5a59b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34982/0 (socket says 172.21.15.3:34982) 2023-08-11T14:21:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.692+0000 7f4a5a59b700 1 -- 172.21.15.3:0/4035305517 learned_addr learned my addr 172.21.15.3:0/4035305517 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.692+0000 7f4a59d9a700 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e229c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 -- 172.21.15.3:0/4035305517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 msgr2=0x55d86e229c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e229c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 -- 172.21.15.3:0/4035305517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 msgr2=0x55d86e208900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:21:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e208900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 -- 172.21.15.3:0/4035305517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86df6bc00 con 0x55d86e2048a0 2023-08-11T14:21:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.693+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c094ed0 tx=0x7f4a4c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a30579b6eec534 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.698+0000 7f4a497fa700 1 -- 172.21.15.3:0/4035305517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c09c290 con 0x55d86e2048a0 2023-08-11T14:21:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.698+0000 7f4a497fa700 1 -- 172.21.15.3:0/4035305517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c00cd80 con 0x55d86e2048a0 2023-08-11T14:21:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.698+0000 7f4a497fa700 1 -- 172.21.15.3:0/4035305517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c097bb0 con 0x55d86e2048a0 2023-08-11T14:21:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.699+0000 7f4a700d0b80 1 -- 172.21.15.3:0/4035305517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 msgr2=0x55d86e215840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.699+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c094ed0 tx=0x7f4a4c004be0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.709+0000 7f4a700d0b80 1 -- 172.21.15.3:0/4035305517 shutdown_connections 2023-08-11T14:21:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.709+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e213450 0x55d86e208900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.709+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e2048a0 0x55d86e215840 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.709+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/4035305517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e229c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.709+0000 7f4a700d0b80 1 -- 172.21.15.3:0/4035305517 >> 172.21.15.3:0/4035305517 conn(0x55d86e1318f0 msgr2=0x55d86e20c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.710+0000 7f4a700d0b80 1 -- 172.21.15.3:0/4035305517 shutdown_connections 2023-08-11T14:21:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.711+0000 7f4a700d0b80 1 -- 172.21.15.3:0/4035305517 wait complete. 2023-08-11T14:21:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.711+0000 7f4a700d0b80 1 -- start start 2023-08-11T14:21:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 Processor -- start 2023-08-11T14:21:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e226bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e2270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e26ca00 con 0x55d86e213450 2023-08-11T14:21:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e26cb40 con 0x55d86e2048a0 2023-08-11T14:21:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e26ccb0 con 0x55d86e205790 2023-08-11T14:21:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.712+0000 7f4a700d0b80 10 client.0 _ll_register_callbacks cb 0x7f4a5001a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a59d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e2270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34996/0 (socket says 172.21.15.3:34996) 2023-08-11T14:21:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 -- 172.21.15.3:0/608462773 learned_addr learned my addr 172.21.15.3:0/608462773 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e226bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 -- 172.21.15.3:0/608462773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 msgr2=0x55d86e2270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e2270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 -- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 msgr2=0x55d86e226bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e226bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.714+0000 7f4a5ad9c700 1 -- 172.21.15.3:0/608462773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86dfd65d0 con 0x55d86e213450 2023-08-11T14:21:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.715+0000 7f4a5ad9c700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a44013d40 tx=0x7f4a44006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b0f57d60c78825 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.715+0000 7f4a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:21:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.719+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44013e30 con 0x55d86e213450 2023-08-11T14:21:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.719+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4a44011cd0 con 0x55d86e213450 2023-08-11T14:21:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.719+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a4c0a0270 con 0x55d86e213450 2023-08-11T14:21:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.720+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a44011e90 con 0x55d86e213450 2023-08-11T14:21:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.720+0000 7f4a700d0b80 10 client.16629 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.721+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d86dfa91f0 con 0x55d86e213450 2023-08-11T14:21:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c0a0270 con 0x55d86e213450 2023-08-11T14:21:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a44027070 con 0x55d86e213450 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 10 client.16629 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 20 client.16629 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e26e6f0 0x55d86e270b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 4 client.16629 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:21:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.722+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d86e14f7c0 con 0x55d86e26e6f0 2023-08-11T14:21:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.724+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e26e6f0 0x55d86e270b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.725+0000 7f4a5a59b700 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e26e6f0 0x55d86e270b10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aed51ddb980aea2 server_cookie=6d3b1fd4d4632c9 in_seq=0 out_seq=0 2023-08-11T14:21:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.725+0000 7f4a3dffb700 10 client.16629 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:07.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.766+0000 7f4a3dffb700 1 -- 172.21.15.3:0/608462773 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d86e14f7c0 con 0x55d86e26e6f0 2023-08-11T14:21:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.766+0000 7f4a3dffb700 10 client.16629 handle_command_reply: tid=0 2023-08-11T14:21:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.766+0000 7f4a700d0b80 1 client.16629 shutdown 2023-08-11T14:21:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.766+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 msgr2=0x55d86e26c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.766+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a44013d40 tx=0x7f4a44006ce0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 shutdown_connections 2023-08-11T14:21:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d86e213450 0x55d86e26c4c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d86e26e6f0 0x55d86e270b10 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d86e2048a0 0x55d86e226bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 --2- 172.21.15.3:0/608462773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d86e205790 0x55d86e2270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 >> 172.21.15.3:0/608462773 conn(0x55d86e1318f0 msgr2=0x55d86e208900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a3dffb700 0 client.16629 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 shutdown_connections 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 1 -- 172.21.15.3:0/608462773 wait complete. 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.767+0000 7f4a700d0b80 20 client.16629 trim_cache size 0 max 16384 2023-08-11T14:21:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:21:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:21:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:21:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:21:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 msgr2=0x7f4a400a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 0x7f4a400a2610 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55d86e204f70 tx=0x7f4a44008040 comp rx=0 tx=0).stop 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 msgr2=0x7f4a5419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00ba50 tx=0x7f4a4c00be10 comp rx=0 tx=0).stop 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 shutdown_connections 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4a400a0150 0x7f4a400a2610 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a5410cc60 0x7f4a5419e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a54107920 0x7f4a5419ce70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 --2- 172.21.15.3:0/393046931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a5410a290 0x7f4a5419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.768+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 >> 172.21.15.3:0/393046931 conn(0x7f4a5406d750 msgr2=0x7f4a5410acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.769+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 shutdown_connections 2023-08-11T14:21:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:07.769+0000 7f4a5c7ff700 1 -- 172.21.15.3:0/393046931 wait complete. 2023-08-11T14:21:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 850.0 s, 677.6 tps, lat 147.193 ms stddev 404.413 2023-08-11T14:21:09.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:08 smithi103 ceph-mon[130434]: pgmap v863: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-08-11T14:21:09.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:08 smithi003 ceph-mon[97195]: pgmap v863: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-08-11T14:21:09.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:08 smithi187 ceph-mon[130973]: pgmap v863: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.27k op/s 2023-08-11T14:21:10.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:10 smithi103 ceph-mon[130434]: pgmap v864: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.34k op/s 2023-08-11T14:21:10.546 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:10 smithi187 ceph-mon[130973]: pgmap v864: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.34k op/s 2023-08-11T14:21:10.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:10 smithi003 ceph-mon[97195]: pgmap v864: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.34k op/s 2023-08-11T14:21:12.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:12 smithi103 ceph-mon[130434]: pgmap v865: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-08-11T14:21:13.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:12 smithi003 ceph-mon[97195]: pgmap v865: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-08-11T14:21:13.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:12 smithi187 ceph-mon[130973]: pgmap v865: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-08-11T14:21:13.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 855.0 s, 990.4 tps, lat 100.010 ms stddev 182.143 2023-08-11T14:21:15.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:14 smithi003 ceph-mon[97195]: pgmap v866: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.29k op/s 2023-08-11T14:21:15.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:14 smithi187 ceph-mon[130973]: pgmap v866: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.29k op/s 2023-08-11T14:21:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:14 smithi103 ceph-mon[130434]: pgmap v866: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 16 MiB/s wr, 1.29k op/s 2023-08-11T14:21:17.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:16 smithi187 ceph-mon[130973]: pgmap v867: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:21:17.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:16 smithi003 ceph-mon[97195]: pgmap v867: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:21:17.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:16 smithi103 ceph-mon[130434]: pgmap v867: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-08-11T14:21:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 860.0 s, 767.2 tps, lat 132.213 ms stddev 327.346 2023-08-11T14:21:19.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:18 smithi103 ceph-mon[130434]: pgmap v868: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:19.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:18 smithi003 ceph-mon[97195]: pgmap v868: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:19.305 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:18 smithi187 ceph-mon[130973]: pgmap v868: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:20.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:20 smithi187 ceph-mon[130973]: pgmap v869: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:21:20.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:20 smithi003 ceph-mon[97195]: pgmap v869: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:21:20.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:20 smithi103 ceph-mon[130434]: pgmap v869: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-08-11T14:21:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.011+0000 7f11b07ff700 1 -- 172.21.15.3:0/1999871827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8108170 msgr2=0x7f11a81085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.011+0000 7f11b07ff700 1 --2- 172.21.15.3:0/1999871827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8108170 0x7f11a81085d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f119401a9d0 tx=0x7f119401ace0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 -- 172.21.15.3:0/1999871827 shutdown_connections 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 --2- 172.21.15.3:0/1999871827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108b10 0x7f11a810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 --2- 172.21.15.3:0/1999871827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8106f80 0x7f11a81073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 --2- 172.21.15.3:0/1999871827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8108170 0x7f11a81085d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 -- 172.21.15.3:0/1999871827 >> 172.21.15.3:0/1999871827 conn(0x7f11a8075f00 msgr2=0x7f11a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 -- 172.21.15.3:0/1999871827 shutdown_connections 2023-08-11T14:21:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.012+0000 7f11b07ff700 1 -- 172.21.15.3:0/1999871827 wait complete. 2023-08-11T14:21:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.013+0000 7f11b07ff700 1 Processor -- start 2023-08-11T14:21:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.013+0000 7f11b07ff700 1 -- start start 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108170 0x7f11a81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8108b10 0x7f11a819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819c260 con 0x7f11a8108170 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819c3d0 con 0x7f11a8106f80 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11b07ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a819c590 con 0x7f11a8108b10 2023-08-11T14:21:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11ae59b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11ae59b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:35752/0 (socket says 172.21.15.3:35752) 2023-08-11T14:21:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11ae59b700 1 -- 172.21.15.3:0/3189633769 learned_addr learned my addr 172.21.15.3:0/3189633769 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11add9a700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108170 0x7f11a81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.014+0000 7f11aed9c700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8108b10 0x7f11a819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 -- 172.21.15.3:0/3189633769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8108b10 msgr2=0x7f11a819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8108b10 0x7f11a819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 -- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108170 msgr2=0x7f11a81993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108170 0x7f11a81993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 -- 172.21.15.3:0/3189633769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119401a600 con 0x7f11a8106f80 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11ae59b700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f119c00efc0 tx=0x7f119c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.015+0000 7f11a77fe700 1 -- 172.21.15.3:0/3189633769 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c018070 con 0x7f11a8106f80 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.016+0000 7f11a77fe700 1 -- 172.21.15.3:0/3189633769 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119c00f040 con 0x7f11a8106f80 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.016+0000 7f11a77fe700 1 -- 172.21.15.3:0/3189633769 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c004280 con 0x7f11a8106f80 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.016+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a819c810 con 0x7f11a8106f80 2023-08-11T14:21:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.016+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a819cca0 con 0x7f11a8106f80 2023-08-11T14:21:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.021+0000 7f11c40d1b80 1 Processor -- start 2023-08-11T14:21:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 -- start start 2023-08-11T14:21:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edae9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae98a0 0x55e4edaea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4eda425d0 con 0x55e4edaea790 2023-08-11T14:21:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4eda3fef0 con 0x55e4edae8f40 2023-08-11T14:21:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.022+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4eda3b7e0 con 0x55e4edae98a0 2023-08-11T14:21:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.032+0000 7f11a77fe700 1 -- 172.21.15.3:0/3189633769 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f119c003a00 con 0x7f11a8106f80 2023-08-11T14:21:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.034+0000 7f11add9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.034+0000 7f11aed9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edae9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.034+0000 7f11add9a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57390/0 (socket says 172.21.15.3:57390) 2023-08-11T14:21:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.034+0000 7f11add9a700 1 -- 172.21.15.3:0/1875518304 learned_addr learned my addr 172.21.15.3:0/1875518304 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.035+0000 7f11a77fe700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 0x7f11980a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.035+0000 7f11a77fe700 1 -- 172.21.15.3:0/3189633769 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f119c0cb3b0 con 0x7f11a8106f80 2023-08-11T14:21:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.042+0000 7f11ae59b700 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae98a0 0x55e4edaea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.044+0000 7f11add9a700 1 -- 172.21.15.3:0/1875518304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae98a0 msgr2=0x55e4edaea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.044+0000 7f11add9a700 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae98a0 0x55e4edaea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.044+0000 7f11add9a700 1 -- 172.21.15.3:0/1875518304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 msgr2=0x55e4edae9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.044+0000 7f11add9a700 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edae9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.044+0000 7f11add9a700 1 -- 172.21.15.3:0/1875518304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ed8bb5d0 con 0x55e4edaea790 2023-08-11T14:21:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.045+0000 7f11add9a700 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f119401c590 tx=0x7f1194005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb47bae07771f93 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.046+0000 7f11a4ff9700 1 -- 172.21.15.3:0/1875518304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119402d050 con 0x55e4edaea790 2023-08-11T14:21:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.046+0000 7f11a4ff9700 1 -- 172.21.15.3:0/1875518304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119402e040 con 0x55e4edaea790 2023-08-11T14:21:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.046+0000 7f11a4ff9700 1 -- 172.21.15.3:0/1875518304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194032e30 con 0x55e4edaea790 2023-08-11T14:21:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.047+0000 7f11c40d1b80 1 -- 172.21.15.3:0/1875518304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 msgr2=0x55e4edaeac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.047+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f119401c590 tx=0x7f1194005ab0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.058+0000 7f11aed9c700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 0x7f11980a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.066+0000 7f11c40d1b80 1 -- 172.21.15.3:0/1875518304 shutdown_connections 2023-08-11T14:21:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.066+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaeac10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.066+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edae9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.066+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/1875518304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae98a0 0x55e4edaea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.066+0000 7f11c40d1b80 1 -- 172.21.15.3:0/1875518304 >> 172.21.15.3:0/1875518304 conn(0x55e4eda168f0 msgr2=0x55e4edae5920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.076+0000 7f11aed9c700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 0x7f11980a25e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009a70 tx=0x7f11a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.088+0000 7f11c40d1b80 1 -- 172.21.15.3:0/1875518304 shutdown_connections 2023-08-11T14:21:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.094+0000 7f11c40d1b80 1 -- 172.21.15.3:0/1875518304 wait complete. 2023-08-11T14:21:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.095+0000 7f11c40d1b80 1 -- start start 2023-08-11T14:21:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.095+0000 7f11c40d1b80 1 Processor -- start 2023-08-11T14:21:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae8f40 0x55e4edaf2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edaf3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edafa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed9d9a60 con 0x55e4edae8f40 2023-08-11T14:21:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed9d9760 con 0x55e4edae98a0 2023-08-11T14:21:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ed8bd8c0 con 0x55e4edaea790 2023-08-11T14:21:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.104+0000 7f11c40d1b80 10 client.0 _ll_register_callbacks cb 0x55e4ed9dca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.109+0000 7f11aed9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edafa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.119+0000 7f11add9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edaf3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.119+0000 7f11add9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edaf3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:35756/0 (socket says 172.21.15.3:35756) 2023-08-11T14:21:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.119+0000 7f11add9a700 1 -- 172.21.15.3:0/2335098602 learned_addr learned my addr 172.21.15.3:0/2335098602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.120+0000 7f11ae59b700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae8f40 0x55e4edaf2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.125+0000 7f11aed9c700 1 -- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 msgr2=0x55e4edaf3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.125+0000 7f11aed9c700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edaf3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.125+0000 7f11aed9c700 1 -- 172.21.15.3:0/2335098602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae8f40 msgr2=0x55e4edaf2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.125+0000 7f11aed9c700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae8f40 0x55e4edaf2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.125+0000 7f11aed9c700 1 -- 172.21.15.3:0/2335098602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a810fd10 con 0x55e4edaea790 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.126+0000 7f11aed9c700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edafa170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11a0011ec0 tx=0x7f11a0009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc31cb2d016023f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.126+0000 7f1191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.126+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0021450 con 0x55e4edaea790 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.126+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f11a001a030 con 0x55e4edaea790 2023-08-11T14:21:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.128+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4edadc7e0 con 0x55e4edaea790 2023-08-11T14:21:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.132+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11a00215b0 con 0x55e4edaea790 2023-08-11T14:21:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.132+0000 7f11c40d1b80 10 client.26866 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.132+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e4ed867d80 con 0x55e4edaea790 2023-08-11T14:21:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.133+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4edadc7e0 con 0x55e4edaea790 2023-08-11T14:21:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.133+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11a0020730 con 0x55e4edaea790 2023-08-11T14:21:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.133+0000 7f11c40d1b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.133+0000 7f11c40d1b80 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.133+0000 7f11c40d1b80 20 client.26866 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.134+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb0fb30 0x55e4edb11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.134+0000 7f11c40d1b80 4 client.26866 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:21:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.134+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4edab8480 con 0x55e4edb0fb30 2023-08-11T14:21:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.135+0000 7f11ae59b700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb0fb30 0x55e4edb11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.141+0000 7f11ae59b700 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb0fb30 0x55e4edb11f20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e80839536c5c server_cookie=1667433f5143477d in_seq=0 out_seq=0 2023-08-11T14:21:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.141+0000 7f1191ffb700 10 client.26866 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.147+0000 7f1191ffb700 1 -- 172.21.15.3:0/2335098602 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e4edab8480 con 0x55e4edb0fb30 2023-08-11T14:21:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.148+0000 7f1191ffb700 10 client.26866 handle_command_reply: tid=0 2023-08-11T14:21:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.148+0000 7f11c40d1b80 1 client.26866 shutdown 2023-08-11T14:21:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.153+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 msgr2=0x55e4edafa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.153+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edafa170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11a0011ec0 tx=0x7f11a0009110 comp rx=0 tx=0).stop 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.153+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 shutdown_connections 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edafa170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae8f40 0x55e4edaf2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb0fb30 0x55e4edb11f20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/2335098602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edaf3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 >> 172.21.15.3:0/2335098602 conn(0x55e4eda168f0 msgr2=0x55e4edae4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.154+0000 7f1191ffb700 0 client.26866 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.160+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 shutdown_connections 2023-08-11T14:21:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.161+0000 7f11c40d1b80 1 -- 172.21.15.3:0/2335098602 wait complete. 2023-08-11T14:21:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.165+0000 7f11c40d1b80 20 client.26866 trim_cache size 0 max 16384 2023-08-11T14:21:21.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 Processor -- start 2023-08-11T14:21:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 -- start start 2023-08-11T14:21:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae8f40 0x55e4edb16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaf2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edb0ce70 con 0x55e4edaea790 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edae4d50 con 0x55e4edae98a0 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.196+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edae9c80 con 0x55e4edae8f40 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:35770/0 (socket says 172.21.15.3:35770) 2023-08-11T14:21:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 -- 172.21.15.3:0/546675670 learned_addr learned my addr 172.21.15.3:0/546675670 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 -- 172.21.15.3:0/546675670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae8f40 msgr2=0x55e4edb16850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:21:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae8f40 0x55e4edb16850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 -- 172.21.15.3:0/546675670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 msgr2=0x55e4edaf2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.197+0000 7f11add9a700 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaf2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.198+0000 7f11add9a700 1 -- 172.21.15.3:0/546675670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ed850c00 con 0x55e4edae98a0 2023-08-11T14:21:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.198+0000 7f11add9a700 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1194005850 tx=0x7f1194000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21cad18a8e15d10 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.198+0000 7f11a57fa700 1 -- 172.21.15.3:0/546675670 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119401ca30 con 0x55e4edae98a0 2023-08-11T14:21:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.198+0000 7f11a57fa700 1 -- 172.21.15.3:0/546675670 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119401cb90 con 0x55e4edae98a0 2023-08-11T14:21:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.198+0000 7f11a57fa700 1 -- 172.21.15.3:0/546675670 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194032660 con 0x55e4edae98a0 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 -- 172.21.15.3:0/546675670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 msgr2=0x55e4edafa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1194005850 tx=0x7f1194000bc0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 -- 172.21.15.3:0/546675670 shutdown_connections 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edaea790 0x55e4edaf2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edae8f40 0x55e4edb16850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/546675670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae98a0 0x55e4edafa130 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.200+0000 7f11c40d1b80 1 -- 172.21.15.3:0/546675670 >> 172.21.15.3:0/546675670 conn(0x55e4eda168f0 msgr2=0x55e4edaf0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.201+0000 7f11c40d1b80 1 -- 172.21.15.3:0/546675670 shutdown_connections 2023-08-11T14:21:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.201+0000 7f11c40d1b80 1 -- 172.21.15.3:0/546675670 wait complete. 2023-08-11T14:21:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.202+0000 7f11c40d1b80 1 -- start start 2023-08-11T14:21:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.202+0000 7f11c40d1b80 1 Processor -- start 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edb5d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae98a0 0x55e4edb5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edb62800 con 0x55e4edae98a0 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edb62970 con 0x55e4edae8f40 2023-08-11T14:21:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4edb62b30 con 0x55e4edaea790 2023-08-11T14:21:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.203+0000 7f11c40d1b80 10 client.0 _ll_register_callbacks cb 0x7f11a0011af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:51424/0 (socket says 172.21.15.3:51424) 2023-08-11T14:21:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 -- 172.21.15.3:0/500156948 learned_addr learned my addr 172.21.15.3:0/500156948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 -- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 msgr2=0x55e4edb5d740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-11T14:21:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edb5d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 -- 172.21.15.3:0/500156948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae98a0 msgr2=0x55e4edb5fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.204+0000 7f11aed9c700 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae98a0 0x55e4edb5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.205+0000 7f11aed9c700 1 -- 172.21.15.3:0/500156948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ed8bb5d0 con 0x55e4edaea790 2023-08-11T14:21:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.205+0000 7f11aed9c700 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b010 tx=0x7f11a0011690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95d126d7d55827a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.206+0000 7f1191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:21:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.206+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0020ec0 con 0x55e4edaea790 2023-08-11T14:21:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.206+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f11a001a040 con 0x55e4edaea790 2023-08-11T14:21:21.208 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.206+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f119402e040 con 0x55e4edaea790 2023-08-11T14:21:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.207+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11a002b450 con 0x55e4edaea790 2023-08-11T14:21:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.207+0000 7f11c40d1b80 10 client.26869 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.207+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55e4ed88e1f0 con 0x55e4edaea790 2023-08-11T14:21:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119402e040 con 0x55e4edaea790 2023-08-11T14:21:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11a00216f0 con 0x55e4edaea790 2023-08-11T14:21:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 10 client.26869 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 10 client.26869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 20 client.26869 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb64440 0x55e4edb66860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 4 client.26869 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:21:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.209+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4eda347c0 con 0x55e4edb64440 2023-08-11T14:21:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.210+0000 7f11ae59b700 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb64440 0x55e4edb66860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.210+0000 7f11ae59b700 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb64440 0x55e4edb66860 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161aa127842a243e server_cookie=9cb275db1de7d03a in_seq=0 out_seq=0 2023-08-11T14:21:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.210+0000 7f1191ffb700 10 client.26869 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.380+0000 7f1191ffb700 1 -- 172.21.15.3:0/500156948 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4eda347c0 con 0x55e4edb64440 2023-08-11T14:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.380+0000 7f1191ffb700 10 client.26869 handle_command_reply: tid=0 2023-08-11T14:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.383+0000 7f11c40d1b80 1 client.26869 shutdown 2023-08-11T14:21:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 msgr2=0x55e4edb621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b010 tx=0x7f11a0011690 comp rx=0 tx=0).stop 2023-08-11T14:21:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 shutdown_connections 2023-08-11T14:21:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4edaea790 0x55e4edb621e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55e4edb64440 0x55e4edb66860 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e4edae8f40 0x55e4edb5d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 --2- 172.21.15.3:0/500156948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e4edae98a0 0x55e4edb5fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.387+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 >> 172.21.15.3:0/500156948 conn(0x55e4eda168f0 msgr2=0x55e4edaed6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.388+0000 7f1191ffb700 0 client.26869 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.389+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 shutdown_connections 2023-08-11T14:21:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.389+0000 7f11c40d1b80 1 -- 172.21.15.3:0/500156948 wait complete. 2023-08-11T14:21:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:21:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:21:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:21:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:21:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.392+0000 7f11c40d1b80 20 client.26869 trim_cache size 0 max 16384 2023-08-11T14:21:21.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.400+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 msgr2=0x7f11980a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.400+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 0x7f11980a25e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009a70 tx=0x7f11a0008040 comp rx=0 tx=0).stop 2023-08-11T14:21:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.400+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 msgr2=0x7f11a8196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.400+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f119c00efc0 tx=0x7f119c00c3a0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 shutdown_connections 2023-08-11T14:21:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11a8108b10 0x7f11a819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f11980a0120 0x7f11980a25e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11a8106f80 0x7f11a8196e60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 --2- 172.21.15.3:0/3189633769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11a8108170 0x7f11a81993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.403+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 >> 172.21.15.3:0/3189633769 conn(0x7f11a8075f00 msgr2=0x7f11a80781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.404+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 shutdown_connections 2023-08-11T14:21:21.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:21.404+0000 7f11b07ff700 1 -- 172.21.15.3:0/3189633769 wait complete. 2023-08-11T14:21:22.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:22 smithi103 ceph-mon[130434]: pgmap v870: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.28k op/s 2023-08-11T14:21:23.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:22 smithi003 ceph-mon[97195]: pgmap v870: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.28k op/s 2023-08-11T14:21:23.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:22 smithi187 ceph-mon[130973]: pgmap v870: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 1.28k op/s 2023-08-11T14:21:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 865.0 s, 465.9 tps, lat 209.082 ms stddev 482.324 2023-08-11T14:21:25.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:24 smithi187 ceph-mon[130973]: pgmap v871: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:25.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:24 smithi003 ceph-mon[97195]: pgmap v871: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:25.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:24 smithi103 ceph-mon[130434]: pgmap v871: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-08-11T14:21:27.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:26 smithi003 ceph-mon[97195]: pgmap v872: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.28k op/s 2023-08-11T14:21:27.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:26 smithi187 ceph-mon[130973]: pgmap v872: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.28k op/s 2023-08-11T14:21:27.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:26 smithi103 ceph-mon[130434]: pgmap v872: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 1.28k op/s 2023-08-11T14:21:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 870.0 s, 817.2 tps, lat 125.273 ms stddev 286.371 2023-08-11T14:21:29.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:28 smithi003 ceph-mon[97195]: pgmap v873: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:29.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:28 smithi187 ceph-mon[130973]: pgmap v873: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:29.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:28 smithi103 ceph-mon[130434]: pgmap v873: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 1.35k op/s 2023-08-11T14:21:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:30 smithi003 ceph-mon[97195]: pgmap v874: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-08-11T14:21:30.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:30 smithi187 ceph-mon[130973]: pgmap v874: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-08-11T14:21:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:30 smithi103 ceph-mon[130434]: pgmap v874: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-08-11T14:21:33.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:32 smithi003 ceph-mon[97195]: pgmap v875: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-08-11T14:21:33.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:32 smithi187 ceph-mon[130973]: pgmap v875: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-08-11T14:21:33.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 875.0 s, 281.2 tps, lat 312.557 ms stddev 567.865 2023-08-11T14:21:33.189 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:32 smithi103 ceph-mon[130434]: pgmap v875: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 18 MiB/s wr, 1.41k op/s 2023-08-11T14:21:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 -- 172.21.15.3:0/2509756281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010a290 msgr2=0x7fbe9010c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2509756281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010a290 0x7fbe9010c720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0098f0 tx=0x7fbe8c009c00 comp rx=0 tx=0).stop 2023-08-11T14:21:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 -- 172.21.15.3:0/2509756281 shutdown_connections 2023-08-11T14:21:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2509756281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9010cc60 0x7fbe9010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2509756281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 0x7fbe90109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2509756281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010a290 0x7fbe9010c720 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0098f0 tx=0x7fbe8c009c00 comp rx=0 tx=0).stop 2023-08-11T14:21:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.729+0000 7fbe987ff700 1 -- 172.21.15.3:0/2509756281 >> 172.21.15.3:0/2509756281 conn(0x7fbe9006d750 msgr2=0x7fbe9006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.737+0000 7fbe987ff700 1 -- 172.21.15.3:0/2509756281 shutdown_connections 2023-08-11T14:21:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.737+0000 7fbe987ff700 1 -- 172.21.15.3:0/2509756281 wait complete. 2023-08-11T14:21:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.739+0000 7fbe987ff700 1 Processor -- start 2023-08-11T14:21:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.739+0000 7fbe987ff700 1 -- start start 2023-08-11T14:21:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 0x7fbe9019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010cc60 0x7fbe9019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019ee40 con 0x7fbe90107920 2023-08-11T14:21:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019efb0 con 0x7fbe9010cc60 2023-08-11T14:21:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.740+0000 7fbe987ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019f170 con 0x7fbe9019e3f0 2023-08-11T14:21:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.741+0000 7fbe96d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.741+0000 7fbe96d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57664/0 (socket says 172.21.15.3:57664) 2023-08-11T14:21:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.741+0000 7fbe96d9c700 1 -- 172.21.15.3:0/2879078539 learned_addr learned my addr 172.21.15.3:0/2879078539 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.742+0000 7fbe9659b700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 0x7fbe9019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.743+0000 7fbe96d9c700 1 -- 172.21.15.3:0/2879078539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010cc60 msgr2=0x7fbe9019d570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-11T14:21:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.743+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010cc60 0x7fbe9019d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.743+0000 7fbe96d9c700 1 -- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 msgr2=0x7fbe9019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.743+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 0x7fbe9019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.743+0000 7fbe96d9c700 1 -- 172.21.15.3:0/2879078539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009520 con 0x7fbe9019e3f0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe8001e9f0 tx=0x7fbe8001ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe877fe700 1 -- 172.21.15.3:0/2879078539 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8001a950 con 0x7fbe9019e3f0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe877fe700 1 -- 172.21.15.3:0/2879078539 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe80004830 con 0x7fbe9019e3f0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe877fe700 1 -- 172.21.15.3:0/2879078539 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe800054e0 con 0x7fbe9019e3f0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe901a56a0 con 0x7fbe9019e3f0 2023-08-11T14:21:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.744+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe901a5be0 con 0x7fbe9019e3f0 2023-08-11T14:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.760+0000 7fbeac09bb80 1 Processor -- start 2023-08-11T14:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.760+0000 7fbeac09bb80 1 -- start start 2023-08-11T14:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd48ff40 0x5645fd490360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd491790 0x5645fd491c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd3e95d0 con 0x5645fd48ff40 2023-08-11T14:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd3e6ef0 con 0x5645fd4908a0 2023-08-11T14:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd3e27e0 con 0x5645fd491790 2023-08-11T14:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbe9659b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.761+0000 7fbe9659b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33430/0 (socket says 172.21.15.3:33430) 2023-08-11T14:21:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 -- 172.21.15.3:0/1491660380 learned_addr learned my addr 172.21.15.3:0/1491660380 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 -- 172.21.15.3:0/1491660380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd491790 msgr2=0x5645fd491c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:21:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd491790 0x5645fd491c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 -- 172.21.15.3:0/1491660380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd48ff40 msgr2=0x5645fd490360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-11T14:21:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd48ff40 0x5645fd490360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe9659b700 1 -- 172.21.15.3:0/1491660380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd2625d0 con 0x5645fd4908a0 2023-08-11T14:21:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.762+0000 7fbe877fe700 1 -- 172.21.15.3:0/2879078539 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbe80021470 con 0x7fbe9019e3f0 2023-08-11T14:21:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.764+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800a4b0 tx=0x7fbe880096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4653a967241bfd4 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.769+0000 7fbe84ff9700 1 -- 172.21.15.3:0/1491660380 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88010040 con 0x5645fd4908a0 2023-08-11T14:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.769+0000 7fbe84ff9700 1 -- 172.21.15.3:0/1491660380 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8800ae80 con 0x5645fd4908a0 2023-08-11T14:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.769+0000 7fbe84ff9700 1 -- 172.21.15.3:0/1491660380 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88014c00 con 0x5645fd4908a0 2023-08-11T14:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.770+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1491660380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 msgr2=0x5645fd4911b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.770+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800a4b0 tx=0x7fbe880096d0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.775+0000 7fbe877fe700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 0x7fbe7c0a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.776+0000 7fbe877fe700 1 -- 172.21.15.3:0/2879078539 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbe800dcf40 con 0x7fbe9019e3f0 2023-08-11T14:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.777+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1491660380 shutdown_connections 2023-08-11T14:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.777+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd491790 0x5645fd491c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.777+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd48ff40 0x5645fd490360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.777+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1491660380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd4911b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.777+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1491660380 >> 172.21.15.3:0/1491660380 conn(0x5645fd3bd8f0 msgr2=0x5645fd48c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.790+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 0x7fbe7c0a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.794+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 0x7fbe7c0a25e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005950 tx=0x7fbe8c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.796+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1491660380 shutdown_connections 2023-08-11T14:21:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.802+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1491660380 wait complete. 2023-08-11T14:21:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.802+0000 7fbeac09bb80 1 -- start start 2023-08-11T14:21:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.803+0000 7fbeac09bb80 1 Processor -- start 2023-08-11T14:21:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd499c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd4908a0 0x5645fd49a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd380a60 con 0x5645fd4908a0 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd380760 con 0x5645fd491790 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd2648c0 con 0x5645fd48ff40 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.804+0000 7fbeac09bb80 10 client.0 _ll_register_callbacks cb 0x5645fd383a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.805+0000 7fbe96d9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.805+0000 7fbe96d9c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:33440/0 (socket says 172.21.15.3:33440) 2023-08-11T14:21:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.805+0000 7fbe96d9c700 1 -- 172.21.15.3:0/3273312815 learned_addr learned my addr 172.21.15.3:0/3273312815 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.805+0000 7fbe9659b700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd499c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.806+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd4908a0 0x5645fd49a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 -- 172.21.15.3:0/3273312815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 msgr2=0x5645fd499c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd499c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 -- 172.21.15.3:0/3273312815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd4908a0 msgr2=0x5645fd49a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd4908a0 0x5645fd49a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 -- 172.21.15.3:0/3273312815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe88009ce0 con 0x5645fd491790 2023-08-11T14:21:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.807+0000 7fbe96d9c700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbe8001cfd0 tx=0x7fbe800aab50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e82cb748febb1664 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.809+0000 7fbe79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:21:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.809+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe800ae440 con 0x5645fd491790 2023-08-11T14:21:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.809+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbe800b2420 con 0x5645fd491790 2023-08-11T14:21:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.809+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645fd45f480 con 0x5645fd491790 2023-08-11T14:21:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.810+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe80005640 con 0x5645fd491790 2023-08-11T14:21:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.810+0000 7fbeac09bb80 10 client.26852 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.810+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5645fd20ed80 con 0x5645fd491790 2023-08-11T14:21:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.810+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645fd45f480 con 0x5645fd491790 2023-08-11T14:21:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe800dab80 con 0x5645fd491790 2023-08-11T14:21:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 10 client.26852 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 10 client.26852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 20 client.26852 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4a5a70 0x5645fd4a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 4 client.26852 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:21:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.812+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645fd3db7c0 con 0x5645fd4a5a70 2023-08-11T14:21:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.813+0000 7fbe9659b700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4a5a70 0x5645fd4a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.816+0000 7fbe9659b700 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4a5a70 0x5645fd4a7e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8829dd274847397c server_cookie=c2af517efd64e3e6 in_seq=0 out_seq=0 2023-08-11T14:21:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.816+0000 7fbe79ffb700 10 client.26852 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.822+0000 7fbe79ffb700 1 -- 172.21.15.3:0/3273312815 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5645fd3db7c0 con 0x5645fd4a5a70 2023-08-11T14:21:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.822+0000 7fbe79ffb700 10 client.26852 handle_command_reply: tid=0 2023-08-11T14:21:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.822+0000 7fbeac09bb80 1 client.26852 shutdown 2023-08-11T14:21:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 msgr2=0x5645fd4a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbe8001cfd0 tx=0x7fbe800aab50 comp rx=0 tx=0).stop 2023-08-11T14:21:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 shutdown_connections 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4a5a70 0x5645fd4a7e90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd491790 0x5645fd4a11f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd499c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3273312815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd4908a0 0x5645fd49a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.828+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 >> 172.21.15.3:0/3273312815 conn(0x5645fd3bd8f0 msgr2=0x5645fd48b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.829+0000 7fbe79ffb700 0 client.26852 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.829+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 shutdown_connections 2023-08-11T14:21:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.829+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3273312815 wait complete. 2023-08-11T14:21:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.837+0000 7fbeac09bb80 20 client.26852 trim_cache size 0 max 16384 2023-08-11T14:21:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.870+0000 7fbeac09bb80 1 Processor -- start 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.871+0000 7fbeac09bb80 1 -- start start 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd48ff40 0x5645fd494650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd48bd50 con 0x5645fd491790 2023-08-11T14:21:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd49adf0 con 0x5645fd48ff40 2023-08-11T14:21:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd490fa0 con 0x5645fd4908a0 2023-08-11T14:21:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbe95d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbe95d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57672/0 (socket says 172.21.15.3:57672) 2023-08-11T14:21:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.872+0000 7fbe95d9a700 1 -- 172.21.15.3:0/3532034393 learned_addr learned my addr 172.21.15.3:0/3532034393 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe9659b700 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd48ff40 0x5645fd494650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe95d9a700 1 -- 172.21.15.3:0/3532034393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd48ff40 msgr2=0x5645fd494650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd48ff40 0x5645fd494650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe95d9a700 1 -- 172.21.15.3:0/3532034393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 msgr2=0x5645fd4a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.873+0000 7fbe95d9a700 1 -- 172.21.15.3:0/3532034393 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fd1f7c00 con 0x5645fd4908a0 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.879+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009230 tx=0x7fbe8c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51cb26ce99963db0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.892+0000 7fbe857fa700 1 -- 172.21.15.3:0/3532034393 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c029070 con 0x5645fd4908a0 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.892+0000 7fbe857fa700 1 -- 172.21.15.3:0/3532034393 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8c00f9e0 con 0x5645fd4908a0 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.892+0000 7fbe857fa700 1 -- 172.21.15.3:0/3532034393 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c01c070 con 0x5645fd4908a0 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.893+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3532034393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 msgr2=0x5645fd494b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.893+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009230 tx=0x7fbe8c01c000 comp rx=0 tx=0).stop 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.901+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3532034393 shutdown_connections 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.901+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.901+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd48ff40 0x5645fd494650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.901+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/3532034393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd4908a0 0x5645fd494b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.901+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3532034393 >> 172.21.15.3:0/3532034393 conn(0x5645fd3bd8f0 msgr2=0x5645fd497b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.918+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3532034393 shutdown_connections 2023-08-11T14:21:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.918+0000 7fbeac09bb80 1 -- 172.21.15.3:0/3532034393 wait complete. 2023-08-11T14:21:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.918+0000 7fbeac09bb80 1 -- start start 2023-08-11T14:21:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 Processor -- start 2023-08-11T14:21:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd49f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd4a0700 con 0x5645fd491790 2023-08-11T14:21:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd4a0870 con 0x5645fd4908a0 2023-08-11T14:21:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fd4a0a30 con 0x5645fd48ff40 2023-08-11T14:21:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.919+0000 7fbeac09bb80 10 client.0 _ll_register_callbacks cb 0x7fbe800aa150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.920+0000 7fbe9659b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.920+0000 7fbe9659b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57682/0 (socket says 172.21.15.3:57682) 2023-08-11T14:21:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.920+0000 7fbe9659b700 1 -- 172.21.15.3:0/1082938052 learned_addr learned my addr 172.21.15.3:0/1082938052 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.920+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd49f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.922+0000 7fbe9659b700 1 -- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 msgr2=0x5645fd49f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.922+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd49f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.922+0000 7fbe9659b700 1 -- 172.21.15.3:0/1082938052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 msgr2=0x5645fd4a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.922+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.922+0000 7fbe9659b700 1 -- 172.21.15.3:0/1082938052 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9010dc60 con 0x5645fd48ff40 2023-08-11T14:21:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.927+0000 7fbe9659b700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbe88004b10 tx=0x7fbe880278f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0b9598e8e8e46c server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.928+0000 7fbe79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:21:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.930+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88024040 con 0x5645fd48ff40 2023-08-11T14:21:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.930+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbe88006e40 con 0x5645fd48ff40 2023-08-11T14:21:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.930+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645fd39bff0 con 0x5645fd48ff40 2023-08-11T14:21:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.930+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe88010040 con 0x5645fd48ff40 2023-08-11T14:21:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.933+0000 7fbeac09bb80 10 client.26884 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.933+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5645fd2351f0 con 0x5645fd48ff40 2023-08-11T14:21:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645fd39bff0 con 0x5645fd48ff40 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbe88027050 con 0x5645fd48ff40 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 10 client.26884 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 10 client.26884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 20 client.26884 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4fa3a0 0x5645fd4fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 4 client.26884 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:21:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.939+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645fd22ff10 con 0x5645fd4fa3a0 2023-08-11T14:21:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.940+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4fa3a0 0x5645fd4fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.942+0000 7fbe95d9a700 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4fa3a0 0x5645fd4fc7c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f7314501ace2ee server_cookie=2498290470fb19f in_seq=0 out_seq=0 2023-08-11T14:21:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:34.943+0000 7fbe79ffb700 10 client.26884 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbe79ffb700 1 -- 172.21.15.3:0/1082938052 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645fd22ff10 con 0x5645fd4fa3a0 2023-08-11T14:21:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbe79ffb700 10 client.26884 handle_command_reply: tid=0 2023-08-11T14:21:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 client.26884 shutdown 2023-08-11T14:21:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 msgr2=0x5645fd49ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbe88004b10 tx=0x7fbe880278f0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 shutdown_connections 2023-08-11T14:21:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5645fd491790 0x5645fd4a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645fd48ff40 0x5645fd49ec70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x5645fd4fa3a0 0x5645fd4fc7c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 --2- 172.21.15.3:0/1082938052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645fd4908a0 0x5645fd49f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.028+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 >> 172.21.15.3:0/1082938052 conn(0x5645fd3bd8f0 msgr2=0x5645fd494650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.029+0000 7fbe79ffb700 0 client.26884 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.029+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 shutdown_connections 2023-08-11T14:21:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.029+0000 7fbeac09bb80 1 -- 172.21.15.3:0/1082938052 wait complete. 2023-08-11T14:21:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.030+0000 7fbeac09bb80 20 client.26884 trim_cache size 0 max 16384 2023-08-11T14:21:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:21:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:21:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:21:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:21:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 msgr2=0x7fbe7c0a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 0x7fbe7c0a25e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005950 tx=0x7fbe8c0058e0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 msgr2=0x7fbe9019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe8001e9f0 tx=0x7fbe8001ed00 comp rx=0 tx=0).stop 2023-08-11T14:21:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 shutdown_connections 2023-08-11T14:21:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9019e3f0 0x7fbe9019e870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe90107920 0x7fbe9019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbe7c0a0120 0x7fbe7c0a25e0 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c005950 tx=0x7fbe8c0058e0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 --2- 172.21.15.3:0/2879078539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe9010cc60 0x7fbe9019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.031+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 >> 172.21.15.3:0/2879078539 conn(0x7fbe9006d750 msgr2=0x7fbe9010ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.032+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 shutdown_connections 2023-08-11T14:21:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:35.032+0000 7fbe987ff700 1 -- 172.21.15.3:0/2879078539 wait complete. 2023-08-11T14:21:35.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:34 smithi187 ceph-mon[130973]: pgmap v876: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:21:35.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:34 smithi103 ceph-mon[130434]: pgmap v876: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:21:35.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:34 smithi003 ceph-mon[97195]: pgmap v876: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 1.17k op/s 2023-08-11T14:21:37.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:36 smithi103 ceph-mon[130434]: pgmap v877: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:21:37.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:36 smithi187 ceph-mon[130973]: pgmap v877: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:21:37.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:36 smithi003 ceph-mon[97195]: pgmap v877: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-08-11T14:21:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 880.0 s, 462.5 tps, lat 237.279 ms stddev 347.465 2023-08-11T14:21:39.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:38 smithi103 ceph-mon[130434]: pgmap v878: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:39.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:38 smithi003 ceph-mon[97195]: pgmap v878: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:39.293 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:38 smithi187 ceph-mon[130973]: pgmap v878: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:40.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:40 smithi003 ceph-mon[97195]: pgmap v879: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 928 op/s 2023-08-11T14:21:40.543 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:40 smithi187 ceph-mon[130973]: pgmap v879: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 928 op/s 2023-08-11T14:21:40.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:40 smithi103 ceph-mon[130434]: pgmap v879: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 928 op/s 2023-08-11T14:21:43.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:42 smithi003 ceph-mon[97195]: pgmap v880: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:21:43.050 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:42 smithi187 ceph-mon[130973]: pgmap v880: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:21:43.081 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 885.0 s, 973.4 tps, lat 103.865 ms stddev 173.698 2023-08-11T14:21:43.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:42 smithi103 ceph-mon[130434]: pgmap v880: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-08-11T14:21:45.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:44 smithi003 ceph-mon[97195]: pgmap v881: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 876 op/s 2023-08-11T14:21:45.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:44 smithi187 ceph-mon[130973]: pgmap v881: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 876 op/s 2023-08-11T14:21:45.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:44 smithi103 ceph-mon[130434]: pgmap v881: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 876 op/s 2023-08-11T14:21:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:46 smithi103 ceph-mon[130434]: pgmap v882: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-08-11T14:21:47.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:46 smithi003 ceph-mon[97195]: pgmap v882: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-08-11T14:21:47.302 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:46 smithi187 ceph-mon[130973]: pgmap v882: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 12 MiB/s wr, 862 op/s 2023-08-11T14:21:48.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 890.0 s, 722.3 tps, lat 139.785 ms stddev 334.116 2023-08-11T14:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.404+0000 7f937bbff700 1 -- 172.21.15.3:0/2055567605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740756a0 msgr2=0x7f9374077b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.404+0000 7f937bbff700 1 --2- 172.21.15.3:0/2055567605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740756a0 0x7f9374077b30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f936c0090c0 tx=0x7f936c00c640 comp rx=0 tx=0).stop 2023-08-11T14:21:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.416+0000 7f937bbff700 1 -- 172.21.15.3:0/2055567605 shutdown_connections 2023-08-11T14:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.416+0000 7f937bbff700 1 --2- 172.21.15.3:0/2055567605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f937407a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.416+0000 7f937bbff700 1 --2- 172.21.15.3:0/2055567605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740727c0 0x7f9374072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.416+0000 7f937bbff700 1 --2- 172.21.15.3:0/2055567605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740756a0 0x7f9374077b30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.416+0000 7f937bbff700 1 -- 172.21.15.3:0/2055567605 >> 172.21.15.3:0/2055567605 conn(0x7f937406d750 msgr2=0x7f937406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.438+0000 7f937bbff700 1 -- 172.21.15.3:0/2055567605 shutdown_connections 2023-08-11T14:21:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.446+0000 7f937bbff700 1 -- 172.21.15.3:0/2055567605 wait complete. 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.447+0000 7f937bbff700 1 Processor -- start 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 -- start start 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740727c0 0x7f937412b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 0x7f937412de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374130cd0 con 0x7f9374078070 2023-08-11T14:21:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374130e40 con 0x7f93740756a0 2023-08-11T14:21:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.448+0000 7f937bbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374131000 con 0x7f93740727c0 2023-08-11T14:21:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.458+0000 7f937919a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 0x7f937412de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.460+0000 7f937a19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.460+0000 7f937a19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44032/0 (socket says 172.21.15.3:44032) 2023-08-11T14:21:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.460+0000 7f937a19c700 1 -- 172.21.15.3:0/1356130282 learned_addr learned my addr 172.21.15.3:0/1356130282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.467+0000 7f937999b700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740727c0 0x7f937412b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 -- 172.21.15.3:0/1356130282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740727c0 msgr2=0x7f937412b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740727c0 0x7f937412b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 -- 172.21.15.3:0/1356130282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 msgr2=0x7f937412de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 0x7f937412de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 -- 172.21.15.3:0/1356130282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c008cf0 con 0x7f9374078070 2023-08-11T14:21:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.486+0000 7f937a19c700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f937000a8e0 tx=0x7f937000abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.487+0000 7f937919a700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 0x7f937412de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:21:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.490+0000 7f936affd700 1 -- 172.21.15.3:0/1356130282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370010070 con 0x7f9374078070 2023-08-11T14:21:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.490+0000 7f936affd700 1 -- 172.21.15.3:0/1356130282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9370004cf0 con 0x7f9374078070 2023-08-11T14:21:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.490+0000 7f936affd700 1 -- 172.21.15.3:0/1356130282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700078b0 con 0x7f9374078070 2023-08-11T14:21:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.501+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9374131280 con 0x7f9374078070 2023-08-11T14:21:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.501+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9374131710 con 0x7f9374078070 2023-08-11T14:21:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.508+0000 7f938f5eab80 1 Processor -- start 2023-08-11T14:21:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.511+0000 7f936affd700 1 -- 172.21.15.3:0/1356130282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f9370005020 con 0x7f9374078070 2023-08-11T14:21:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.513+0000 7f936affd700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 0x7f93600a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.514+0000 7f936affd700 1 -- 172.21.15.3:0/1356130282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f93700ca570 con 0x7f9374078070 2023-08-11T14:21:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.515+0000 7f937919a700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 0x7f93600a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.516+0000 7f937919a700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 0x7f93600a2610 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f936c000f80 tx=0x7f936c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.516+0000 7f938f5eab80 1 -- start start 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c277979360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c27797a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27797ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2778d25d0 con 0x55c27797a790 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2778cfef0 con 0x55c2779798a0 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f938f5eab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2778cb7e0 con 0x55c277978f40 2023-08-11T14:21:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f937919a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c27797a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f937919a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c27797a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47424/0 (socket says 172.21.15.3:47424) 2023-08-11T14:21:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.517+0000 7f937919a700 1 -- 172.21.15.3:0/3084092603 learned_addr learned my addr 172.21.15.3:0/3084092603 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.518+0000 7f937a19c700 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27797ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 -- 172.21.15.3:0/3084092603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 msgr2=0x55c277979360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c277979360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 -- 172.21.15.3:0/3084092603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 msgr2=0x55c27797a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c27797a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 -- 172.21.15.3:0/3084092603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c27774b5d0 con 0x55c27797a790 2023-08-11T14:21:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.519+0000 7f937a19c700 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27797ac10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f93700c7cf0 tx=0x7f9370097bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6a6c21601b2648 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.522+0000 7f935ffff700 1 -- 172.21.15.3:0/3084092603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700b70d0 con 0x55c27797a790 2023-08-11T14:21:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.522+0000 7f935ffff700 1 -- 172.21.15.3:0/3084092603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93700983d0 con 0x55c27797a790 2023-08-11T14:21:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.522+0000 7f935ffff700 1 -- 172.21.15.3:0/3084092603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700971c0 con 0x55c27797a790 2023-08-11T14:21:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 -- 172.21.15.3:0/3084092603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 msgr2=0x55c27797ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27797ac10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f93700c7cf0 tx=0x7f9370097bd0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 -- 172.21.15.3:0/3084092603 shutdown_connections 2023-08-11T14:21:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27797ac10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c277979360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3084092603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c27797a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 -- 172.21.15.3:0/3084092603 >> 172.21.15.3:0/3084092603 conn(0x55c2778a68f0 msgr2=0x55c277975920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 -- 172.21.15.3:0/3084092603 shutdown_connections 2023-08-11T14:21:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.525+0000 7f938f5eab80 1 -- 172.21.15.3:0/3084092603 wait complete. 2023-08-11T14:21:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.526+0000 7f938f5eab80 1 -- start start 2023-08-11T14:21:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.526+0000 7f938f5eab80 1 Processor -- start 2023-08-11T14:21:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c277983210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27798a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277869a60 con 0x55c27797a790 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277869760 con 0x55c277978f40 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c27774d8c0 con 0x55c2779798a0 2023-08-11T14:21:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.527+0000 7f938f5eab80 10 client.0 _ll_register_callbacks cb 0x55c27786ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.528+0000 7f937999b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.528+0000 7f937999b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:47432/0 (socket says 172.21.15.3:47432) 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.528+0000 7f937999b700 1 -- 172.21.15.3:0/3655699503 learned_addr learned my addr 172.21.15.3:0/3655699503 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.529+0000 7f937a19c700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27798a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.529+0000 7f937999b700 1 -- 172.21.15.3:0/3655699503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 msgr2=0x55c277983210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.529+0000 7f937999b700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c277983210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.529+0000 7f937999b700 1 -- 172.21.15.3:0/3655699503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 msgr2=0x55c27798a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.529+0000 7f937999b700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27798a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.530+0000 7f937999b700 1 -- 172.21.15.3:0/3655699503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9374131710 con 0x55c277978f40 2023-08-11T14:21:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.538+0000 7f937999b700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f936400c5f0 tx=0x7f936400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d94fb73c9be721f server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.539+0000 7f935d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:21:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.557+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364004300 con 0x55c277978f40 2023-08-11T14:21:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.557+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9364004460 con 0x55c277978f40 2023-08-11T14:21:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.558+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c27796c7e0 con 0x55c277978f40 2023-08-11T14:21:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.587+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9364003680 con 0x55c277978f40 2023-08-11T14:21:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.587+0000 7f938f5eab80 10 client.26870 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.587+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55c2776f7d80 con 0x55c277978f40 2023-08-11T14:21:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c27796c7e0 con 0x55c277978f40 2023-08-11T14:21:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f936401f070 con 0x55c277978f40 2023-08-11T14:21:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 10 client.26870 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 20 client.26870 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c27798ea80 0x55c277990ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 4 client.26870 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:21:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.600+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c277948480 con 0x55c27798ea80 2023-08-11T14:21:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.603+0000 7f937919a700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c27798ea80 0x55c277990ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.638+0000 7f937919a700 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c27798ea80 0x55c277990ea0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7fece95d395b47 server_cookie=968a09a426d85d03 in_seq=0 out_seq=0 2023-08-11T14:21:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.638+0000 7f935d7fa700 10 client.26870 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.649+0000 7f935d7fa700 1 -- 172.21.15.3:0/3655699503 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c277948480 con 0x55c27798ea80 2023-08-11T14:21:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.649+0000 7f935d7fa700 10 client.26870 handle_command_reply: tid=0 2023-08-11T14:21:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.649+0000 7f938f5eab80 1 client.26870 shutdown 2023-08-11T14:21:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.673+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 msgr2=0x55c277982cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.673+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f936400c5f0 tx=0x7f936400c900 comp rx=0 tx=0).stop 2023-08-11T14:21:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 shutdown_connections 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c27798a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c27798ea80 0x55c277990ea0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c277978f40 0x55c277982cd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 --2- 172.21.15.3:0/3655699503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c277983210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.678+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 >> 172.21.15.3:0/3655699503 conn(0x55c2778a68f0 msgr2=0x55c277974710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.679+0000 7f935d7fa700 0 client.26870 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.681+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 shutdown_connections 2023-08-11T14:21:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.681+0000 7f938f5eab80 1 -- 172.21.15.3:0/3655699503 wait complete. 2023-08-11T14:21:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.682+0000 7f938f5eab80 20 client.26870 trim_cache size 0 max 16384 2023-08-11T14:21:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 Processor -- start 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 -- start start 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c27798a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c27797a790 0x55c2779751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277974f50 con 0x55c277978f40 2023-08-11T14:21:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2779809a0 con 0x55c27797a790 2023-08-11T14:21:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.712+0000 7f938f5eab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277981fe0 con 0x55c2779798a0 2023-08-11T14:21:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.715+0000 7f937999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.715+0000 7f937999b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44058/0 (socket says 172.21.15.3:44058) 2023-08-11T14:21:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.715+0000 7f937999b700 1 -- 172.21.15.3:0/550796527 learned_addr learned my addr 172.21.15.3:0/550796527 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.715+0000 7f937a19c700 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c27797a790 0x55c2779751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.721+0000 7f937999b700 1 -- 172.21.15.3:0/550796527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 msgr2=0x55c27798a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.721+0000 7f937999b700 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c27798a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.721+0000 7f937999b700 1 -- 172.21.15.3:0/550796527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c27797a790 msgr2=0x55c2779751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.721+0000 7f937999b700 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c27797a790 0x55c2779751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.721+0000 7f937999b700 1 -- 172.21.15.3:0/550796527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2776e0c00 con 0x55c277978f40 2023-08-11T14:21:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.723+0000 7f937999b700 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f936400cd00 tx=0x7f9364005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abe31deae7ff4f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.723+0000 7f9368bff700 1 -- 172.21.15.3:0/550796527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364005580 con 0x55c277978f40 2023-08-11T14:21:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.724+0000 7f9368bff700 1 -- 172.21.15.3:0/550796527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9364013040 con 0x55c277978f40 2023-08-11T14:21:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.724+0000 7f9368bff700 1 -- 172.21.15.3:0/550796527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364003ce0 con 0x55c277978f40 2023-08-11T14:21:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.725+0000 7f938f5eab80 1 -- 172.21.15.3:0/550796527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 msgr2=0x55c27797ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.725+0000 7f938f5eab80 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f936400cd00 tx=0x7f9364005ce0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.733+0000 7f938f5eab80 1 -- 172.21.15.3:0/550796527 shutdown_connections 2023-08-11T14:21:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.733+0000 7f938f5eab80 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c27797a790 0x55c2779751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.733+0000 7f938f5eab80 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c277978f40 0x55c27797ab90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.733+0000 7f938f5eab80 1 --2- 172.21.15.3:0/550796527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2779798a0 0x55c27798a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.733+0000 7f938f5eab80 1 -- 172.21.15.3:0/550796527 >> 172.21.15.3:0/550796527 conn(0x55c2778a68f0 msgr2=0x55c27797d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.745+0000 7f938f5eab80 1 -- 172.21.15.3:0/550796527 shutdown_connections 2023-08-11T14:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.745+0000 7f938f5eab80 1 -- 172.21.15.3:0/550796527 wait complete. 2023-08-11T14:21:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.746+0000 7f938f5eab80 1 -- start start 2023-08-11T14:21:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.746+0000 7f938f5eab80 1 Processor -- start 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c2779881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c277988700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277989c50 con 0x55c27797a790 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277989dc0 con 0x55c2779798a0 2023-08-11T14:21:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c277989f80 con 0x55c277978f40 2023-08-11T14:21:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.747+0000 7f938f5eab80 10 client.0 _ll_register_callbacks cb 0x55c27786ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:21:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.750+0000 7f937a19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.750+0000 7f937a19c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44062/0 (socket says 172.21.15.3:44062) 2023-08-11T14:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.750+0000 7f937a19c700 1 -- 172.21.15.3:0/960553160 learned_addr learned my addr 172.21.15.3:0/960553160 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:21:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.751+0000 7f937999b700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c2779881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.752+0000 7f937a19c700 1 -- 172.21.15.3:0/960553160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 msgr2=0x55c2779881c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.752+0000 7f937a19c700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c2779881c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.752+0000 7f937a19c700 1 -- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 msgr2=0x55c277988700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:21:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.752+0000 7f937a19c700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c277988700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.752+0000 7f937a19c700 1 -- 172.21.15.3:0/960553160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2776f7d80 con 0x55c27797a790 2023-08-11T14:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.753+0000 7f937a19c700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f937000a8b0 tx=0x7f9370097aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aae34401b2c9aa server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:21:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.754+0000 7f935d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-08-11T14:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.754+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937009fa50 con 0x55c27797a790 2023-08-11T14:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.754+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9370097cd0 con 0x55c27797a790 2023-08-11T14:21:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.755+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93700983d0 con 0x55c27797a790 2023-08-11T14:21:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.756+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937009f5c0 con 0x55c27797a790 2023-08-11T14:21:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.761+0000 7f938f5eab80 10 client.16662 fetch_fsmap learned FSMap version 58 2023-08-11T14:21:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.761+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55c27771e1f0 con 0x55c27797a790 2023-08-11T14:21:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93700983d0 con 0x55c27797a790 2023-08-11T14:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9370004750 con 0x55c27797a790 2023-08-11T14:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 10 client.16662 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:21:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:21:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 20 client.16662 populate_metadata read hostname 'smithi003' 2023-08-11T14:21:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c2779e3510 0x55c2779e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:21:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 4 client.16662 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:21:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.772+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2778c47c0 con 0x55c2779e3510 2023-08-11T14:21:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.786+0000 7f937999b700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c2779e3510 0x55c2779e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:21:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.813+0000 7f937999b700 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c2779e3510 0x55c2779e5930 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcaeb7ddf29ea95 server_cookie=b2569b43de7f71c7 in_seq=0 out_seq=0 2023-08-11T14:21:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:48.813+0000 7f935d7fa700 10 client.16662 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:49.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:48 smithi103 ceph-mon[130434]: pgmap v883: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:21:49.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:48 smithi003 ceph-mon[97195]: pgmap v883: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:21:49.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:48 smithi187 ceph-mon[130973]: pgmap v883: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2023-08-11T14:21:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.471+0000 7f935d7fa700 1 -- 172.21.15.3:0/960553160 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2778c47c0 con 0x55c2779e3510 2023-08-11T14:21:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.471+0000 7f935d7fa700 10 client.16662 handle_command_reply: tid=0 2023-08-11T14:21:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.471+0000 7f938f5eab80 1 client.16662 shutdown 2023-08-11T14:21:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.473+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 msgr2=0x55c277989710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.473+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f937000a8b0 tx=0x7f9370097aa0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 shutdown_connections 2023-08-11T14:21:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c27797a790 0x55c277989710 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c277978f40 0x55c2779881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55c2779e3510 0x55c2779e5930 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 --2- 172.21.15.3:0/960553160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2779798a0 0x55c277988700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 >> 172.21.15.3:0/960553160 conn(0x55c2778a68f0 msgr2=0x55c2779751e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.476+0000 7f935d7fa700 0 client.16662 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:21:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.486+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 shutdown_connections 2023-08-11T14:21:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.486+0000 7f938f5eab80 1 -- 172.21.15.3:0/960553160 wait complete. 2023-08-11T14:21:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.495+0000 7f938f5eab80 20 client.16662 trim_cache size 0 max 16384 2023-08-11T14:21:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:21:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:21:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:21:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:21:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.503+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 msgr2=0x7f93600a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.503+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 0x7f93600a2610 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f936c000f80 tx=0x7f936c01a040 comp rx=0 tx=0).stop 2023-08-11T14:21:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.503+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 msgr2=0x7f93741304b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:21:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.503+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f937000a8e0 tx=0x7f937000abf0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 shutdown_connections 2023-08-11T14:21:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9374078070 0x7f93741304b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93740727c0 0x7f937412b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f93600a0150 0x7f93600a2610 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 --2- 172.21.15.3:0/1356130282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93740756a0 0x7f937412de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:21:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.513+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 >> 172.21.15.3:0/1356130282 conn(0x7f937406d750 msgr2=0x7f9374079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:21:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.519+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 shutdown_connections 2023-08-11T14:21:49.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:21:49.525+0000 7f937bbff700 1 -- 172.21.15.3:0/1356130282 wait complete. 2023-08-11T14:21:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:50 smithi103 ceph-mon[130434]: pgmap v884: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 907 op/s 2023-08-11T14:21:50.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:50 smithi003 ceph-mon[97195]: pgmap v884: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 907 op/s 2023-08-11T14:21:50.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:50 smithi187 ceph-mon[130973]: pgmap v884: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 907 op/s 2023-08-11T14:21:53.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:52 smithi187 ceph-mon[130973]: pgmap v885: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-08-11T14:21:53.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:52 smithi003 ceph-mon[97195]: pgmap v885: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-08-11T14:21:53.098 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 895.0 s, 692.7 tps, lat 143.978 ms stddev 331.717 2023-08-11T14:21:53.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:52 smithi103 ceph-mon[130434]: pgmap v885: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 1.29k op/s 2023-08-11T14:21:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:54 smithi003 ceph-mon[97195]: pgmap v886: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:21:55.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:54 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:21:55.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:54 smithi187 ceph-mon[130973]: pgmap v886: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:21:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:54 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:21:55.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:54 smithi103 ceph-mon[130434]: pgmap v886: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 1.21k op/s 2023-08-11T14:21:55.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:54 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:21:56.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:55 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.296 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:55 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:56.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:55 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: pgmap v887: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:57.187 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:21:57.188 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:21:57.188 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:56 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:21:57.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: pgmap v887: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:21:57.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:56 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:21:57.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi103 to 196.2M 2023-08-11T14:21:57.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi103 to 205771059: error parsing value: Value '205771059' is below minimum 939524096 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: pgmap v887: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:21:57.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:56 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:21:58.080 INFO:teuthology.orchestra.run.smithi003.stderr:progress: 900.0 s, 323.1 tps, lat 311.695 ms stddev 469.543 2023-08-11T14:21:58.207 INFO:teuthology.orchestra.run.smithi003.stdout:transaction type: 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:scaling factor: 500 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:query mode: simple 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:number of clients: 100 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:number of threads: 4 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:duration: 900 s 2023-08-11T14:21:58.208 INFO:teuthology.orchestra.run.smithi003.stdout:number of transactions actually processed: 703719 2023-08-11T14:21:58.209 INFO:teuthology.orchestra.run.smithi003.stdout:latency average = 127.871 ms 2023-08-11T14:21:58.209 INFO:teuthology.orchestra.run.smithi003.stdout:latency stddev = 295.140 ms 2023-08-11T14:21:58.209 INFO:teuthology.orchestra.run.smithi003.stdout:tps = 781.799267 (including connections establishing) 2023-08-11T14:21:58.209 INFO:teuthology.orchestra.run.smithi003.stdout:tps = 781.806061 (excluding connections establishing) 2023-08-11T14:21:58.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-08-11T14:21:59.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:21:58 smithi103 ceph-mon[130434]: pgmap v888: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-08-11T14:21:59.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:21:58 smithi003 ceph-mon[97195]: pgmap v888: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-08-11T14:21:59.303 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:21:58 smithi187 ceph-mon[130973]: pgmap v888: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 1.46k op/s 2023-08-11T14:22:00.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:00 smithi103 ceph-mon[130434]: pgmap v889: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:22:00.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:00 smithi003 ceph-mon[97195]: pgmap v889: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:22:00.805 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:00 smithi187 ceph-mon[130973]: pgmap v889: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-08-11T14:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.880+0000 7fd43adff700 1 -- 172.21.15.3:0/3974654541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43410cc60 msgr2=0x7fd43410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.880+0000 7fd43adff700 1 --2- 172.21.15.3:0/3974654541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43410cc60 0x7fd43410d0e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b0a0 tx=0x7fd42c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.897+0000 7fd43adff700 1 -- 172.21.15.3:0/3974654541 shutdown_connections 2023-08-11T14:22:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.897+0000 7fd43adff700 1 --2- 172.21.15.3:0/3974654541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd43410cc60 0x7fd43410d0e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.897+0000 7fd43adff700 1 --2- 172.21.15.3:0/3974654541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd434107920 0x7fd434109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.897+0000 7fd43adff700 1 --2- 172.21.15.3:0/3974654541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410a290 0x7fd43410c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.897+0000 7fd43adff700 1 -- 172.21.15.3:0/3974654541 >> 172.21.15.3:0/3974654541 conn(0x7fd43406d750 msgr2=0x7fd43406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.917+0000 7fd43adff700 1 -- 172.21.15.3:0/3974654541 shutdown_connections 2023-08-11T14:22:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.923+0000 7fd43adff700 1 -- 172.21.15.3:0/3974654541 wait complete. 2023-08-11T14:22:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.924+0000 7fd43adff700 1 Processor -- start 2023-08-11T14:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 -- start start 2023-08-11T14:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 0x7fd434113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 0x7fd434114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410cc60 0x7fd434115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434115960 con 0x7fd434107920 2023-08-11T14:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434115ad0 con 0x7fd43410a290 2023-08-11T14:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.925+0000 7fd43adff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434115c90 con 0x7fd43410cc60 2023-08-11T14:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.927+0000 7fd43939c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410cc60 0x7fd434115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.929+0000 7fd438b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 0x7fd434113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.929+0000 7fd438b9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 0x7fd434113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46994/0 (socket says 172.21.15.3:46994) 2023-08-11T14:22:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.929+0000 7fd438b9b700 1 -- 172.21.15.3:0/4032978099 learned_addr learned my addr 172.21.15.3:0/4032978099 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.941+0000 7fd433fff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 0x7fd434114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.955+0000 7fd433fff700 1 -- 172.21.15.3:0/4032978099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410cc60 msgr2=0x7fd434115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.955+0000 7fd433fff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410cc60 0x7fd434115300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.955+0000 7fd433fff700 1 -- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 msgr2=0x7fd434113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.955+0000 7fd433fff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 0x7fd434113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.955+0000 7fd433fff700 1 -- 172.21.15.3:0/4032978099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c009d00 con 0x7fd43410a290 2023-08-11T14:22:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.966+0000 7fd433fff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 0x7fd434114430 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd42800e9f0 tx=0x7fd42800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.968+0000 7fd431ffb700 1 -- 172.21.15.3:0/4032978099 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800c9c0 con 0x7fd43410a290 2023-08-11T14:22:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.969+0000 7fd431ffb700 1 -- 172.21.15.3:0/4032978099 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd428010430 con 0x7fd43410a290 2023-08-11T14:22:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.969+0000 7fd431ffb700 1 -- 172.21.15.3:0/4032978099 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428018680 con 0x7fd43410a290 2023-08-11T14:22:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.970+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd434115f70 con 0x7fd43410a290 2023-08-11T14:22:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.970+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4341167a0 con 0x7fd43410a290 2023-08-11T14:22:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.976+0000 7fd44e748b80 1 Processor -- start 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 -- start start 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e575d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 0x55d8e575e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 0x55d8e575ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e56b65d0 con 0x55d8e575d8a0 2023-08-11T14:22:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e56b3ef0 con 0x55d8e575cf40 2023-08-11T14:22:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.977+0000 7fd44e748b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e56af7e0 con 0x55d8e575e790 2023-08-11T14:22:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.984+0000 7fd433fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 0x55d8e575ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.984+0000 7fd433fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 0x55d8e575ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57756/0 (socket says 172.21.15.3:57756) 2023-08-11T14:22:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.984+0000 7fd433fff700 1 -- 172.21.15.3:0/504031950 learned_addr learned my addr 172.21.15.3:0/504031950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.990+0000 7fd438b9b700 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 0x55d8e575e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.991+0000 7fd438b9b700 1 -- 172.21.15.3:0/504031950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 msgr2=0x55d8e575ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.991+0000 7fd438b9b700 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 0x55d8e575ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.991+0000 7fd438b9b700 1 -- 172.21.15.3:0/504031950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 msgr2=0x55d8e575d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.991+0000 7fd438b9b700 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e575d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.991+0000 7fd438b9b700 1 -- 172.21.15.3:0/504031950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e552f5d0 con 0x55d8e575d8a0 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.992+0000 7fd438b9b700 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 0x55d8e575e1b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd424009ab0 tx=0x7fd424009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104254eba4a160e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.992+0000 7fd41affd700 1 -- 172.21.15.3:0/504031950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400e8d0 con 0x55d8e575d8a0 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.992+0000 7fd41affd700 1 -- 172.21.15.3:0/504031950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424013470 con 0x55d8e575d8a0 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.992+0000 7fd41affd700 1 -- 172.21.15.3:0/504031950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424012630 con 0x55d8e575d8a0 2023-08-11T14:22:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.993+0000 7fd44e748b80 1 -- 172.21.15.3:0/504031950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 msgr2=0x55d8e575e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:02.993+0000 7fd44e748b80 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 0x55d8e575e1b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd424009ab0 tx=0x7fd424009dc0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.000+0000 7fd431ffb700 1 -- 172.21.15.3:0/4032978099 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd42800ccd0 con 0x7fd43410a290 2023-08-11T14:22:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.013+0000 7fd431ffb700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 0x7fd41c0a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.013+0000 7fd431ffb700 1 -- 172.21.15.3:0/4032978099 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd428014070 con 0x7fd43410a290 2023-08-11T14:22:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.016+0000 7fd44e748b80 1 -- 172.21.15.3:0/504031950 shutdown_connections 2023-08-11T14:22:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.016+0000 7fd44e748b80 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575e790 0x55d8e575ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.016+0000 7fd44e748b80 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e575d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.016+0000 7fd44e748b80 1 --2- 172.21.15.3:0/504031950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575d8a0 0x55d8e575e1b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.016+0000 7fd44e748b80 1 -- 172.21.15.3:0/504031950 >> 172.21.15.3:0/504031950 conn(0x55d8e568a8f0 msgr2=0x55d8e5759920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.022+0000 7fd43939c700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 0x7fd41c0a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.023+0000 7fd44e748b80 1 -- 172.21.15.3:0/504031950 shutdown_connections 2023-08-11T14:22:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.023+0000 7fd44e748b80 1 -- 172.21.15.3:0/504031950 wait complete. 2023-08-11T14:22:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.023+0000 7fd44e748b80 1 -- start start 2023-08-11T14:22:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.024+0000 7fd44e748b80 1 Processor -- start 2023-08-11T14:22:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.024+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575cf40 0x55d8e5766c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.024+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.025+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575e790 0x55d8e576e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.025+0000 7fd44e748b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e564da60 con 0x55d8e575cf40 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.025+0000 7fd44e748b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e564d760 con 0x55d8e575e790 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.025+0000 7fd44e748b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e55318c0 con 0x55d8e575d8a0 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.025+0000 7fd44e748b80 10 client.0 _ll_register_callbacks cb 0x55d8e5650a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.026+0000 7fd433fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.026+0000 7fd433fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:57760/0 (socket says 172.21.15.3:57760) 2023-08-11T14:22:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.026+0000 7fd433fff700 1 -- 172.21.15.3:0/1662288657 learned_addr learned my addr 172.21.15.3:0/1662288657 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.040+0000 7fd43939c700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575e790 0x55d8e576e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.045+0000 7fd438b9b700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575cf40 0x55d8e5766c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 -- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575e790 msgr2=0x55d8e576e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575e790 0x55d8e576e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 -- 172.21.15.3:0/1662288657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575cf40 msgr2=0x55d8e5766c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575cf40 0x55d8e5766c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 -- 172.21.15.3:0/1662288657 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd424009710 con 0x55d8e575d8a0 2023-08-11T14:22:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.051+0000 7fd433fff700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd428095d30 tx=0x7fd4280992c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a71f961978fa985 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.061+0000 7fd413fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-08-11T14:22:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.062+0000 7fd43939c700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 0x7fd41c0a2610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b780 tx=0x7fd42c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.072+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4280a10d0 con 0x55d8e575d8a0 2023-08-11T14:22:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.072+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd4280b1250 con 0x55d8e575d8a0 2023-08-11T14:22:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.074+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8e57507e0 con 0x55d8e575d8a0 2023-08-11T14:22:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.091+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4280b1410 con 0x55d8e575d8a0 2023-08-11T14:22:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.091+0000 7fd44e748b80 10 client.26905 fetch_fsmap learned FSMap version 58 2023-08-11T14:22:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.091+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d8e54dbd80 con 0x55d8e575d8a0 2023-08-11T14:22:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8e57507e0 con 0x55d8e575d8a0 2023-08-11T14:22:03.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4280a0c40 con 0x55d8e575d8a0 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 10 client.26905 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 20 client.26905 populate_metadata read hostname 'smithi003' 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e5794a30 0x55d8e5796e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 4 client.26905 mds_command: new command op to 24569 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-11T14:22:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.106+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8e572c480 con 0x55d8e5794a30 2023-08-11T14:22:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.126+0000 7fd438b9b700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e5794a30 0x55d8e5796e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.160+0000 7fd438b9b700 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e5794a30 0x55d8e5796e50 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a45a084394596ff server_cookie=e2f6dd1cee853aae in_seq=0 out_seq=0 2023-08-11T14:22:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.161+0000 7fd413fff700 10 client.26905 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:22:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.176+0000 7fd413fff700 1 -- 172.21.15.3:0/1662288657 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d8e572c480 con 0x55d8e5794a30 2023-08-11T14:22:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.176+0000 7fd413fff700 10 client.26905 handle_command_reply: tid=0 2023-08-11T14:22:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.176+0000 7fd44e748b80 1 client.26905 shutdown 2023-08-11T14:22:03.184 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:02 smithi103 ceph-mon[130434]: pgmap v890: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 msgr2=0x55d8e5767160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd428095d30 tx=0x7fd4280992c0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 shutdown_connections 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e5794a30 0x55d8e5796e50 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575e790 0x55d8e576e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575cf40 0x55d8e5766c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 --2- 172.21.15.3:0/1662288657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e5767160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 >> 172.21.15.3:0/1662288657 conn(0x55d8e568a8f0 msgr2=0x55d8e5758710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.182+0000 7fd413fff700 0 client.26905 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:22:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.196+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 shutdown_connections 2023-08-11T14:22:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.197+0000 7fd44e748b80 1 -- 172.21.15.3:0/1662288657 wait complete. 2023-08-11T14:22:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.203+0000 7fd44e748b80 20 client.26905 trim_cache size 0 max 16384 2023-08-11T14:22:03.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.242+0000 7fd44e748b80 1 Processor -- start 2023-08-11T14:22:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 -- start start 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 0x55d8e575d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575d8a0 0x55d8e576c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5766170 con 0x55d8e575e790 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5752ea0 con 0x55d8e575d8a0 2023-08-11T14:22:03.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.253+0000 7fd44e748b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5764890 con 0x55d8e575cf40 2023-08-11T14:22:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.254+0000 7fd43939c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.254+0000 7fd43939c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47020/0 (socket says 172.21.15.3:47020) 2023-08-11T14:22:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.254+0000 7fd43939c700 1 -- 172.21.15.3:0/4122211355 learned_addr learned my addr 172.21.15.3:0/4122211355 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd433fff700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575d8a0 0x55d8e576c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd438b9b700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 0x55d8e575d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd43939c700 1 -- 172.21.15.3:0/4122211355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 msgr2=0x55d8e575d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd43939c700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 0x55d8e575d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd43939c700 1 -- 172.21.15.3:0/4122211355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575d8a0 msgr2=0x55d8e576c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd43939c700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575d8a0 0x55d8e576c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.264+0000 7fd43939c700 1 -- 172.21.15.3:0/4122211355 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e54c4c00 con 0x55d8e575e790 2023-08-11T14:22:03.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.266+0000 7fd438b9b700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 0x55d8e575d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-11T14:22:03.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:02 smithi003 ceph-mon[97195]: pgmap v890: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:22:03.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:02 smithi187 ceph-mon[130973]: pgmap v890: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-08-11T14:22:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.295+0000 7fd43939c700 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007910 tx=0x7fd42c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49faf77e4f3256e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.314+0000 7fd41b7fe700 1 -- 172.21.15.3:0/4122211355 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c027070 con 0x55d8e575e790 2023-08-11T14:22:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.314+0000 7fd41b7fe700 1 -- 172.21.15.3:0/4122211355 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd42c003d90 con 0x55d8e575e790 2023-08-11T14:22:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.314+0000 7fd41b7fe700 1 -- 172.21.15.3:0/4122211355 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00e040 con 0x55d8e575e790 2023-08-11T14:22:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.317+0000 7fd44e748b80 1 -- 172.21.15.3:0/4122211355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 msgr2=0x55d8e5766c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.317+0000 7fd44e748b80 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007910 tx=0x7fd42c01e000 comp rx=0 tx=0).stop 2023-08-11T14:22:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.319+0000 7fd44e748b80 1 -- 172.21.15.3:0/4122211355 shutdown_connections 2023-08-11T14:22:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.319+0000 7fd44e748b80 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e5766c20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.319+0000 7fd44e748b80 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575cf40 0x55d8e575d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.319+0000 7fd44e748b80 1 --2- 172.21.15.3:0/4122211355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575d8a0 0x55d8e576c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.319+0000 7fd44e748b80 1 -- 172.21.15.3:0/4122211355 >> 172.21.15.3:0/4122211355 conn(0x55d8e568a8f0 msgr2=0x55d8e5766370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.334+0000 7fd44e748b80 1 -- 172.21.15.3:0/4122211355 shutdown_connections 2023-08-11T14:22:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.334+0000 7fd44e748b80 1 -- 172.21.15.3:0/4122211355 wait complete. 2023-08-11T14:22:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.334+0000 7fd44e748b80 1 -- start start 2023-08-11T14:22:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 Processor -- start 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e576a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e576b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e576bae0 con 0x55d8e575e790 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e576bc50 con 0x55d8e575cf40 2023-08-11T14:22:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e576be10 con 0x55d8e575d8a0 2023-08-11T14:22:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.335+0000 7fd44e748b80 10 client.0 _ll_register_callbacks cb 0x7fd428098e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-11T14:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.344+0000 7fd438b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.344+0000 7fd438b9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:36410/0 (socket says 172.21.15.3:36410) 2023-08-11T14:22:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.344+0000 7fd438b9b700 1 -- 172.21.15.3:0/2801934454 learned_addr learned my addr 172.21.15.3:0/2801934454 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.344+0000 7fd433fff700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e576a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.353+0000 7fd43939c700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e576b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.354+0000 7fd438b9b700 1 -- 172.21.15.3:0/2801934454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 msgr2=0x55d8e576a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.354+0000 7fd438b9b700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e576a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.354+0000 7fd438b9b700 1 -- 172.21.15.3:0/2801934454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 msgr2=0x55d8e576b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-11T14:22:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.354+0000 7fd438b9b700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e576b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.354+0000 7fd438b9b700 1 -- 172.21.15.3:0/2801934454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43410dc60 con 0x55d8e575cf40 2023-08-11T14:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.361+0000 7fd438b9b700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e00 tx=0x7fd424006a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0302bfb7476bf0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.361+0000 7fd413fff700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2023-08-11T14:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.372+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424022040 con 0x55d8e575cf40 2023-08-11T14:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.372+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd4240137e0 con 0x55d8e575cf40 2023-08-11T14:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.373+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd424013470 con 0x55d8e575cf40 2023-08-11T14:22:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.386+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4240139a0 con 0x55d8e575cf40 2023-08-11T14:22:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.386+0000 7fd44e748b80 10 client.26885 fetch_fsmap learned FSMap version 58 2023-08-11T14:22:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.386+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55d8e55021f0 con 0x55d8e575cf40 2023-08-11T14:22:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424013470 con 0x55d8e575cf40 2023-08-11T14:22:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 58) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd424025050 con 0x55d8e575cf40 2023-08-11T14:22:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 10 client.26885 fetch_fsmap finished waiting for FSMap version 58 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 10 client.26885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 20 client.26885 populate_metadata read hostname 'smithi003' 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e57e9680 0x55d8e57ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 4 client.26885 mds_command: new command op to 24569 tid=0[{"prefix": "flush journal"}] 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.408+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 --> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8e56a87c0 con 0x55d8e57e9680 2023-08-11T14:22:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.412+0000 7fd433fff700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e57e9680 0x55d8e57ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.453+0000 7fd433fff700 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e57e9680 0x55d8e57ebaa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f02ea64f4c9d1a8 server_cookie=fea11f118d7d0887 in_seq=0 out_seq=0 2023-08-11T14:22:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:03.454+0000 7fd413fff700 10 client.26885 ms_handle_connect on v2:172.21.15.103:6838/2494449018 2023-08-11T14:22:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.786+0000 7fd413fff700 1 -- 172.21.15.3:0/2801934454 <== mds.0 v2:172.21.15.103:6838/2494449018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8e56a87c0 con 0x55d8e57e9680 2023-08-11T14:22:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.786+0000 7fd413fff700 10 client.26885 handle_command_reply: tid=0 2023-08-11T14:22:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.786+0000 7fd44e748b80 1 client.26885 shutdown 2023-08-11T14:22:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 msgr2=0x55d8e576a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e00 tx=0x7fd424006a70 comp rx=0 tx=0).stop 2023-08-11T14:22:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 shutdown_connections 2023-08-11T14:22:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d8e575e790 0x55d8e576b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:6838/2494449018,v1:172.21.15.103:6839/2494449018] conn(0x55d8e57e9680 0x55d8e57ebaa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e575cf40 0x55d8e576a050 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e00 tx=0x7fd424006a70 comp rx=0 tx=0).stop 2023-08-11T14:22:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 --2- 172.21.15.3:0/2801934454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e575d8a0 0x55d8e576a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.798+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 >> 172.21.15.3:0/2801934454 conn(0x55d8e568a8f0 msgr2=0x55d8e5761610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.799+0000 7fd413fff700 0 client.26885 ms_handle_reset on v2:172.21.15.103:6838/2494449018 2023-08-11T14:22:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.806+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 shutdown_connections 2023-08-11T14:22:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.806+0000 7fd44e748b80 1 -- 172.21.15.3:0/2801934454 wait complete. 2023-08-11T14:22:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.817+0000 7fd44e748b80 20 client.26885 trim_cache size 0 max 16384 2023-08-11T14:22:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2023-08-11T14:22:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2023-08-11T14:22:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2023-08-11T14:22:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2023-08-11T14:22:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.845+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 msgr2=0x7fd41c0a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.845+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 0x7fd41c0a2610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b780 tx=0x7fd42c0094b0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.845+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 msgr2=0x7fd434114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.845+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 0x7fd434114430 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd42800e9f0 tx=0x7fd42800ed00 comp rx=0 tx=0).stop 2023-08-11T14:22:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 shutdown_connections 2023-08-11T14:22:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43410cc60 0x7fd434115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd434107920 0x7fd434113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd41c0a0150 0x7fd41c0a2610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 --2- 172.21.15.3:0/4032978099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43410a290 0x7fd434114430 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.856+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 >> 172.21.15.3:0/4032978099 conn(0x7fd43406d750 msgr2=0x7fd4341090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.864+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 shutdown_connections 2023-08-11T14:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2023-08-11T14:22:04.864+0000 7fd43adff700 1 -- 172.21.15.3:0/4032978099 wait complete. 2023-08-11T14:22:05.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:04 smithi103 ceph-mon[130434]: pgmap v891: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-08-11T14:22:05.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:04 smithi003 ceph-mon[97195]: pgmap v891: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-08-11T14:22:05.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:04 smithi187 ceph-mon[130973]: pgmap v891: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 1.05k op/s 2023-08-11T14:22:07.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:06 smithi187 ceph-mon[130973]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-08-11T14:22:07.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:06 smithi003 ceph-mon[97195]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-08-11T14:22:07.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:06 smithi103 ceph-mon[130434]: pgmap v892: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-08-11T14:22:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:08 smithi003 ceph-mon[97195]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-08-11T14:22:09.299 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:09 smithi187 ceph-mon[130973]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-08-11T14:22:09.428 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:08 smithi103 ceph-mon[130434]: pgmap v893: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-08-11T14:22:10.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:10 smithi103 ceph-mon[130434]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 766 op/s 2023-08-11T14:22:10.794 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:10 smithi187 ceph-mon[130973]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 766 op/s 2023-08-11T14:22:10.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:10 smithi003 ceph-mon[97195]: pgmap v894: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 766 op/s 2023-08-11T14:22:13.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:12 smithi003 ceph-mon[97195]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.6 MiB/s wr, 865 op/s 2023-08-11T14:22:13.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:12 smithi187 ceph-mon[130973]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.6 MiB/s wr, 865 op/s 2023-08-11T14:22:13.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:12 smithi103 ceph-mon[130434]: pgmap v895: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.6 MiB/s wr, 865 op/s 2023-08-11T14:22:15.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:14 smithi003 ceph-mon[97195]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 6.1 MiB/s wr, 762 op/s 2023-08-11T14:22:15.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:14 smithi187 ceph-mon[130973]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 6.1 MiB/s wr, 762 op/s 2023-08-11T14:22:15.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:14 smithi103 ceph-mon[130434]: pgmap v896: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 6.1 MiB/s wr, 762 op/s 2023-08-11T14:22:15.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/: 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 . 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 data 2023-08-11T14:22:15.391 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data: 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:total 31 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 . 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:03 .. 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 base 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Aug 11 14:04 current_logfiles 2023-08-11T14:22:15.392 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Aug 11 14:04 global 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 log 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_commit_ts 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_dynshmem 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Aug 11 14:04 pg_hba.conf 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 11 14:04 pg_ident.conf 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 pg_logical 2023-08-11T14:22:15.393 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 pg_multixact 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 pg_notify 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_replslot 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_serial 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_snapshots 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 pg_stat 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:22 pg_stat_tmp 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:21 pg_subtrans 2023-08-11T14:22:15.394 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_tblspc 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 pg_twophase 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Aug 11 14:22 pg_wal 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 pg_xact 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 11 14:04 postgresql.auto.conf 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Aug 11 14:04 postgresql.conf 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Aug 11 14:04 postmaster.opts 2023-08-11T14:22:15.395 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base: 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 . 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 1 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 12346 2023-08-11T14:22:15.396 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Aug 11 14:07 12347 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:06 pgsql_tmp 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/1: 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout:total 7209 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 . 2023-08-11T14:22:15.397 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 .. 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:22:15.398 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:22:15.399 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:22:15.399 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:22:15.399 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:22:15.399 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:22:15.399 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:22:15.400 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:22:15.401 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:22:15.402 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:22:15.403 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:22:15.403 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:22:15.403 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:22:15.403 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:22:15.403 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:04 1249 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:22:15.404 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:04 1259 2023-08-11T14:22:15.405 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:22:15.408 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:22:15.409 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:22:15.410 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:22:15.411 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:22:15.412 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:22:15.413 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:22:15.413 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:22:15.413 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606 2023-08-11T14:22:15.413 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606_vm 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 11 14:04 2608 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2608_fsm 2023-08-11T14:22:15.414 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2608_vm 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610_fsm 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2610_vm 2023-08-11T14:22:15.415 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:22:15.416 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:22:15.417 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 11 14:04 2619 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2619_fsm 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2619_vm 2023-08-11T14:22:15.418 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:22:15.419 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 11 14:04 2658 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2659 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:22:15.420 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2662 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2663 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2664 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2665 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2666 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2667 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:22:15.421 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:22:15.422 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:04 2673 2023-08-11T14:22:15.422 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:04 2674 2023-08-11T14:22:15.422 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:22:15.422 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2678 2023-08-11T14:22:15.422 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2679 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:22:15.423 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:22:15.424 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2696 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:22:15.425 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:22:15.426 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:22:15.427 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:22:15.428 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:22:15.429 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:22:15.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:22:15.431 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:22:15.432 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 3455 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:22:15.433 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:22:15.434 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:22:15.435 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:22:15.436 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:22:15.437 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:22:15.438 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:22:15.440 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:22:15.440 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:22:15.440 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:22:15.440 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:22:15.441 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:22:15.442 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:22:15.442 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:22:15.442 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:22:15.442 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:22:15.443 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:22:15.444 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:total 7209 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Aug 11 14:04 . 2023-08-11T14:22:15.445 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 .. 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:22:15.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:22:15.447 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:22:15.448 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:22:15.449 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:22:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:04 1249 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:22:15.451 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:04 1259 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:22:15.452 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:22:15.453 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:22:15.454 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:22:15.455 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:22:15.456 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2606_vm 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:22:15.457 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Aug 11 14:04 2608 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2608_fsm 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2608_vm 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610 2023-08-11T14:22:15.458 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2610_fsm 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2610_vm 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:22:15.459 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 11 14:04 2619 2023-08-11T14:22:15.461 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2619_fsm 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2619_vm 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:22:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 11 14:04 2658 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2659 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:22:15.463 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2662 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2663 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2664 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2665 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2666 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2667 2023-08-11T14:22:15.464 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:04 2673 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:04 2674 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2678 2023-08-11T14:22:15.465 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2679 2023-08-11T14:22:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:22:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:22:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:22:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:22:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:22:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2696 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:22:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:22:15.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:22:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:22:15.471 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:22:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:22:15.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:22:15.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:22:15.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:22:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:22:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:22:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:22:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:22:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 3455 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:22:15.477 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:22:15.478 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:22:15.479 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:22:15.479 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:22:15.479 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:22:15.479 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:22:15.479 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:22:15.480 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:22:15.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:22:15.482 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:22:15.483 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:22:15.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:22:15.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:22:15.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.489 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:total 7767696 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 315 Aug 11 14:07 . 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 .. 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 112 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 113 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 11 14:04 12182 2023-08-11T14:22:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12182_fsm 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12182_vm 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12184 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12186 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12187_fsm 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12187_vm 2023-08-11T14:22:15.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12189 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12191 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12192_fsm 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12192_vm 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12194 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12196 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197 2023-08-11T14:22:15.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12197_fsm 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12197_vm 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12199 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12201 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12202_fsm 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12202_vm 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12204 2023-08-11T14:22:15.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12206 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 12207_fsm 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12207_vm 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12209 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12211 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12212 2023-08-11T14:22:15.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 12214 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 12216 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Aug 11 14:04 1247 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1247_fsm 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1247_vm 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Aug 11 14:07 1249 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1249_fsm 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1249_vm 2023-08-11T14:22:15.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Aug 11 14:04 1255 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1255_fsm 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1255_vm 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 11 14:21 1259 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1259_fsm 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1259_vm 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1417_vm 2023-08-11T14:22:15.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 1418_vm 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3948544 Aug 11 14:21 16387 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:21 16387_fsm 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:21 16387_vm 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 712704 Aug 11 14:21 16393 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:21 16393_fsm 2023-08-11T14:22:15.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:21 16393_vm 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396.1 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396.2 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396.3 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396.4 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16396.5 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 337035264 Aug 11 14:21 16396.6 2023-08-11T14:22:15.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1687552 Aug 11 14:21 16396_fsm 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Aug 11 14:21 16396_vm 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:21 16397 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Aug 11 14:21 16399 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Aug 11 14:21 16401 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Aug 11 14:21 16401.1 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 36962304 Aug 11 14:21 16403 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:21 16403_fsm 2023-08-11T14:22:15.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 174 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 175 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2187 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2224_vm 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2328_vm 2023-08-11T14:22:15.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2336_vm 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2337 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2600 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2600_fsm 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2600_vm 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601 2023-08-11T14:22:15.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2601_fsm 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2601_vm 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2602 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2602_fsm 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2602_vm 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2603 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2603_fsm 2023-08-11T14:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2603_vm 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2604_vm 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2605 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2605_fsm 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2605_vm 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:06 2606 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2606_fsm 2023-08-11T14:22:15.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:06 2606_vm 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2607_fsm 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2607_vm 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Aug 11 14:06 2608 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:06 2608_fsm 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:06 2608_vm 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Aug 11 14:04 2609 2023-08-11T14:22:15.504 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2609_fsm 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2609_vm 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:07 2610 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:06 2610_fsm 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:06 2610_vm 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2611_vm 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612 2023-08-11T14:22:15.505 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2612_fsm 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2612_vm 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2613_vm 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2615_fsm 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2615_vm 2023-08-11T14:22:15.506 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2616_fsm 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2616_vm 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 11 14:04 2617 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2617_fsm 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2617_vm 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 11 14:04 2618 2023-08-11T14:22:15.507 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2618_fsm 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2618_vm 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Aug 11 14:21 2619 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:21 2619_fsm 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:06 2619_vm 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620 2023-08-11T14:22:15.508 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2620_vm 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2650 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2651 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2652 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2653 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2654 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2655 2023-08-11T14:22:15.509 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2656 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2657 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 11 14:06 2658 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:06 2659 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2660 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2661 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:07 2662 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:07 2663 2023-08-11T14:22:15.510 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2664 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2665 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2666 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2667 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2668 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2669 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2670 2023-08-11T14:22:15.511 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Aug 11 14:06 2673 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Aug 11 14:06 2674 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:04 2675 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2678 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:06 2679 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2680 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2681 2023-08-11T14:22:15.512 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2682 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2683 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2684 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2685 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2686 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2687 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2688 2023-08-11T14:22:15.513 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 11 14:04 2689 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 11 14:04 2690 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Aug 11 14:04 2691 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2692 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2693 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:21 2696 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2699 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2701 2023-08-11T14:22:15.514 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2702 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2703 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:04 2704 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2753 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2753_fsm 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2753_vm 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2754 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2755 2023-08-11T14:22:15.515 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2756 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 2757 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2830_vm 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2831 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2832_vm 2023-08-11T14:22:15.516 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2833 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2834_vm 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2835 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2836_vm 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2837 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Aug 11 14:04 2838 2023-08-11T14:22:15.517 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2838_fsm 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2838_vm 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2839 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2840_fsm 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2840_vm 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2841 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995 2023-08-11T14:22:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2995_vm 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2996 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3079_fsm 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3079_vm 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3080 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3081 2023-08-11T14:22:15.519 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3085 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3118_vm 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3119 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3164 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3256_vm 2023-08-11T14:22:15.520 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3257 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3258 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3350_vm 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3351 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3379 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3380 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381 2023-08-11T14:22:15.521 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3381_vm 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3394 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3394_fsm 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3394_vm 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3395 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3439_vm 2023-08-11T14:22:15.522 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3440 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 11 14:07 3455 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3456 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3456_fsm 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3456_vm 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466 2023-08-11T14:22:15.523 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3466_vm 2023-08-11T14:22:15.524 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3467 2023-08-11T14:22:15.524 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3468 2023-08-11T14:22:15.524 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501 2023-08-11T14:22:15.525 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3501_vm 2023-08-11T14:22:15.525 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3502 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3503 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3534 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3541_fsm 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3541_vm 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3542 2023-08-11T14:22:15.526 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3574 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3575 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3576_vm 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3596_vm 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3597 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598 2023-08-11T14:22:15.527 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3598_vm 2023-08-11T14:22:15.528 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3599 2023-08-11T14:22:15.528 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600 2023-08-11T14:22:15.528 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3600_fsm 2023-08-11T14:22:15.528 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3600_vm 2023-08-11T14:22:15.528 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3601_fsm 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3601_vm 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3602_fsm 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3602_vm 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3603 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3603_fsm 2023-08-11T14:22:15.529 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3603_vm 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3604 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3605 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3606 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3607 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3608 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 11 14:04 3609 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3712 2023-08-11T14:22:15.530 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 3764_fsm 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3764_vm 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3766 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 3767 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3997 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 5002 2023-08-11T14:22:15.531 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 548 2023-08-11T14:22:15.532 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 549 2023-08-11T14:22:15.532 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102 2023-08-11T14:22:15.532 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6102_vm 2023-08-11T14:22:15.532 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104 2023-08-11T14:22:15.533 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6104_vm 2023-08-11T14:22:15.533 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106 2023-08-11T14:22:15.533 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6106_vm 2023-08-11T14:22:15.533 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6110 2023-08-11T14:22:15.533 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6111 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6112 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6113 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6117 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 826_vm 2023-08-11T14:22:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 827 2023-08-11T14:22:15.535 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 828 2023-08-11T14:22:15.535 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:22:15.536 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Aug 11 14:04 pg_internal.init 2023-08-11T14:22:15.536 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:04 PG_VERSION 2023-08-11T14:22:15.536 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.536 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-08-11T14:22:15.536 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:06 . 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 11 14:06 .. 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/global: 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:total 574 2023-08-11T14:22:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Aug 11 14:04 . 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1136 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1136_fsm 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1136_vm 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1137 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1213 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1213_fsm 2023-08-11T14:22:15.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1213_vm 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1214 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1214_fsm 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1214_vm 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1232 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 1233 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1260 2023-08-11T14:22:15.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1260_fsm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1260_vm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1261 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1261_fsm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1261_vm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1262 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 1262_fsm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 1262_vm 2023-08-11T14:22:15.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2396 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 11 14:04 2396_fsm 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2396_vm 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2397 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2671 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2672 2023-08-11T14:22:15.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2676 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2677 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2694 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2695 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2697 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 11 14:04 2698 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2846 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2846_vm 2023-08-11T14:22:15.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2847 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2964 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2964_vm 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2965 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2966 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 2966_vm 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 2967 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3592 2023-08-11T14:22:15.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 3592_vm 2023-08-11T14:22:15.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 3593 2023-08-11T14:22:15.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 4060 2023-08-11T14:22:15.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 4060_vm 2023-08-11T14:22:15.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 4061 2023-08-11T14:22:15.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6000 2023-08-11T14:22:15.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6000_vm 2023-08-11T14:22:15.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6001 2023-08-11T14:22:15.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6002 2023-08-11T14:22:15.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6100 2023-08-11T14:22:15.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 6100_vm 2023-08-11T14:22:15.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6114 2023-08-11T14:22:15.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 6115 2023-08-11T14:22:15.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:22 pg_control 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 11 14:04 pg_filenode.map 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Aug 11 14:04 pg_internal.init 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/log: 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:total 5 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 5014 Aug 11 14:22 postgresql-Fri.log 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.548 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:total 1 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 . 2023-08-11T14:22:15.549 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 mappings 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 11 14:22 replorigin_checkpoint 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 snapshots 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.550 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 .. 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 .. 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.551 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 . 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 members 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 offsets 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-08-11T14:22:15.552 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 .. 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:22:15.553 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 11 14:04 .. 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout:total 8 2023-08-11T14:22:15.554 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 11 14:04 0000 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-08-11T14:22:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.556 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.557 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-08-11T14:22:15.557 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.557 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.557 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:total 13 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 11 14:22 . 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Aug 11 14:22 db_0.stat 2023-08-11T14:22:15.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Aug 11 14:22 db_12347.stat 2023-08-11T14:22:15.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Aug 11 14:22 global.stat 2023-08-11T14:22:15.559 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.560 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-08-11T14:22:15.560 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.560 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:22 . 2023-08-11T14:22:15.560 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:total 192 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:21 . 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 11 14:21 000A 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-08-11T14:22:15.561 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.562 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:total 1048576 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Aug 11 14:22 . 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 00000001000000030000009E 2023-08-11T14:22:15.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 00000001000000030000009F 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A0 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A1 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A2 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A3 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A4 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A5 2023-08-11T14:22:15.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A6 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A7 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A8 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000A9 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000AA 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:22 0000000100000003000000AB 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000AC 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000AD 2023-08-11T14:22:15.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000AE 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000AF 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B0 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B1 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B2 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B3 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B4 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000B5 2023-08-11T14:22:15.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000B6 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B7 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000B8 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000B9 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000BA 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000BB 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000BC 2023-08-11T14:22:15.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000BD 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000BE 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000BF 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:19 0000000100000003000000C0 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C1 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C2 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C3 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C4 2023-08-11T14:22:15.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C5 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C6 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000C7 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000C8 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000C9 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000CA 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000CB 2023-08-11T14:22:15.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000CC 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000CD 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000CE 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000CF 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000D0 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000D1 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000D2 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000D3 2023-08-11T14:22:15.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000D4 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000D5 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000D6 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000D7 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000D8 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000D9 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000DA 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000DB 2023-08-11T14:22:15.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:21 0000000100000003000000DC 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 11 14:20 0000000100000003000000DD 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 archive_status 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:total 0 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 11 14:04 . 2023-08-11T14:22:15.572 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Aug 11 14:22 .. 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:total 176 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 11 14:04 . 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 11 14:22 .. 2023-08-11T14:22:15.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Aug 11 14:21 0000 2023-08-11T14:22:15.574 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-08-11T14:22:15.591 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:22:15.605 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-08-11T14:22:15.699 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-08-11T14:22:15.700 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_92684807f016ce045ac66e37fdde6f8c47857bcc/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-08-11T14:22:15.700 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-08-11T14:22:15.729 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:22:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 -- 172.21.15.3:0/11503893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b0074960 msgr2=0x7fd8b0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/11503893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b0074960 0x7fd8b0072db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40098f0 tx=0x7fd8a4009c00 comp rx=0 tx=0).stop 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 -- 172.21.15.3:0/11503893 shutdown_connections 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/11503893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b01087f0 0x7fd8b010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/11503893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b0074960 0x7fd8b0072db0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/11503893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 -- 172.21.15.3:0/11503893 >> 172.21.15.3:0/11503893 conn(0x7fd8b00fbe60 msgr2=0x7fd8b00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.962+0000 7fd8b6bff700 1 -- 172.21.15.3:0/11503893 shutdown_connections 2023-08-11T14:22:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.963+0000 7fd8b6bff700 1 -- 172.21.15.3:0/11503893 wait complete. 2023-08-11T14:22:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.963+0000 7fd8b6bff700 1 Processor -- start 2023-08-11T14:22:15.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 -- start start 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b01087f0 0x7fd8b019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01a03c0 con 0x7fd8b0074960 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01a0500 con 0x7fd8b01087f0 2023-08-11T14:22:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b6bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01a0690 con 0x7fd8b0073380 2023-08-11T14:22:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42578/0 (socket says 172.21.15.3:42578) 2023-08-11T14:22:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8affff700 1 -- 172.21.15.3:0/45434024 learned_addr learned my addr 172.21.15.3:0/45434024 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b499b700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.964+0000 7fd8b519c700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b01087f0 0x7fd8b019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:15.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 -- 172.21.15.3:0/45434024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b01087f0 msgr2=0x7fd8b019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b01087f0 0x7fd8b019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 -- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 msgr2=0x7fd8b0198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 -- 172.21.15.3:0/45434024 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4009520 con 0x7fd8b0073380 2023-08-11T14:22:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8affff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b499b700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0198840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009230 tx=0x7fd8a400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a401c070 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b01a0910 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8a400f7d0 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4003a00 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.965+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b01a0d40 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.966+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0106ea0 con 0x7fd8b0073380 2023-08-11T14:22:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.969+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd8a4020030 con 0x7fd8b0073380 2023-08-11T14:22:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.970+0000 7fd8adffb700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 0x7fd8a00a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.970+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fd8a40cc600 con 0x7fd8b0073380 2023-08-11T14:22:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.970+0000 7fd8affff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 0x7fd8a00a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.971+0000 7fd8affff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 0x7fd8a00a2460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:15.972+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd8a40960a0 con 0x7fd8b0073380 2023-08-11T14:22:16.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.185+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8b010e4f0 con 0x7fd8b0073380 2023-08-11T14:22:16.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.187+0000 7fd8adffb700 1 -- 172.21.15.3:0/45434024 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7fd8a40996c0 con 0x7fd8b0073380 2023-08-11T14:22:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:16.190 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:22:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 msgr2=0x7fd8a00a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 0x7fd8a00a2460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009fd0 tx=0x7fd89c009310 comp rx=0 tx=0).stop 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 msgr2=0x7fd8b0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0198840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009230 tx=0x7fd8a400b790 comp rx=0 tx=0).stop 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 shutdown_connections 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd8a009ffa0 0x7fd8a00a2460 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8b01087f0 0x7fd8b019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8b0073380 0x7fd8b0198840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 --2- 172.21.15.3:0/45434024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8b0074960 0x7fd8b0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 >> 172.21.15.3:0/45434024 conn(0x7fd8b00fbe60 msgr2=0x7fd8b00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 shutdown_connections 2023-08-11T14:22:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.190+0000 7fd8b6bff700 1 -- 172.21.15.3:0/45434024 wait complete. 2023-08-11T14:22:16.211 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:22:16.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:22:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 -- 172.21.15.3:0/1277624485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c100910 msgr2=0x7f4d1c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 --2- 172.21.15.3:0/1277624485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c100910 0x7f4d1c100d30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000b0d0 tx=0x7f4d1000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 -- 172.21.15.3:0/1277624485 shutdown_connections 2023-08-11T14:22:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 --2- 172.21.15.3:0/1277624485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c1024a0 0x7f4d1c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 --2- 172.21.15.3:0/1277624485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c100910 0x7f4d1c100d30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 --2- 172.21.15.3:0/1277624485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 -- 172.21.15.3:0/1277624485 >> 172.21.15.3:0/1277624485 conn(0x7f4d1c0fbe60 msgr2=0x7f4d1c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 -- 172.21.15.3:0/1277624485 shutdown_connections 2023-08-11T14:22:16.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.418+0000 7f4d22fff700 1 -- 172.21.15.3:0/1277624485 wait complete. 2023-08-11T14:22:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.420+0000 7f4d22fff700 1 Processor -- start 2023-08-11T14:22:16.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.420+0000 7f4d22fff700 1 -- start start 2023-08-11T14:22:16.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.420+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 0x7f4d1c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d22fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c1024a0 0x7f4d1c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d22fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c19e370 con 0x7f4d1c101b00 2023-08-11T14:22:16.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d22fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c19e4b0 con 0x7f4d1c1024a0 2023-08-11T14:22:16.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d22fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c19e640 con 0x7f4d1c100910 2023-08-11T14:22:16.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42586/0 (socket says 172.21.15.3:42586) 2023-08-11T14:22:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d1bfff700 1 -- 172.21.15.3:0/116423439 learned_addr learned my addr 172.21.15.3:0/116423439 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 0x7f4d1c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d2159c700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c1024a0 0x7f4d1c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 -- 172.21.15.3:0/116423439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c1024a0 msgr2=0x7f4d1c19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c1024a0 0x7f4d1c19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 -- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 msgr2=0x7f4d1c196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d20d9b700 1 -- 172.21.15.3:0/116423439 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10009d00 con 0x7f4d1c100910 2023-08-11T14:22:16.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.421+0000 7f4d1bfff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d20d9b700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 0x7f4d1c1967f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000f80 tx=0x7f4d10008ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000e040 con 0x7f4d1c100910 2023-08-11T14:22:16.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c19e8c0 con 0x7f4d1c100910 2023-08-11T14:22:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d10005070 con 0x7f4d1c100910 2023-08-11T14:22:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1001d6f0 con 0x7f4d1c100910 2023-08-11T14:22:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.422+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c19ed40 con 0x7f4d1c100910 2023-08-11T14:22:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.423+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c10c170 con 0x7f4d1c100910 2023-08-11T14:22:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.425+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4d100051e0 con 0x7f4d1c100910 2023-08-11T14:22:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.427+0000 7f4d19ffb700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 0x7f4d0c0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.427+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4d100cc910 con 0x7f4d1c100910 2023-08-11T14:22:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.427+0000 7f4d1bfff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 0x7f4d0c0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.427+0000 7f4d1bfff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 0x7f4d0c0a2490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4d08011550 tx=0x7f4d08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.429+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4d10096440 con 0x7f4d1c100910 2023-08-11T14:22:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.640+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d1c107890 con 0x7f4d1c100910 2023-08-11T14:22:16.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.640+0000 7f4d19ffb700 1 -- 172.21.15.3:0/116423439 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d10099a60 con 0x7f4d1c100910 2023-08-11T14:22:16.642 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:22:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 msgr2=0x7f4d0c0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 0x7f4d0c0a2490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4d08011550 tx=0x7f4d08009260 comp rx=0 tx=0).stop 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 msgr2=0x7f4d1c1967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 0x7f4d1c1967f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000f80 tx=0x7f4d10008ec0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 shutdown_connections 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f4d0c09ffd0 0x7f4d0c0a2490 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d1c1024a0 0x7f4d1c19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d1c100910 0x7f4d1c1967f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 --2- 172.21.15.3:0/116423439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d1c101b00 0x7f4d1c196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 >> 172.21.15.3:0/116423439 conn(0x7f4d1c0fbe60 msgr2=0x7f4d1c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:16.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 shutdown_connections 2023-08-11T14:22:16.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.643+0000 7f4d22fff700 1 -- 172.21.15.3:0/116423439 wait complete. 2023-08-11T14:22:16.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:22:16.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 -- 172.21.15.3:0/1999746968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92381024a0 msgr2=0x7f9238108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 --2- 172.21.15.3:0/1999746968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92381024a0 0x7f9238108d50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f923000b0d0 tx=0x7f923000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 -- 172.21.15.3:0/1999746968 shutdown_connections 2023-08-11T14:22:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 --2- 172.21.15.3:0/1999746968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92381024a0 0x7f9238108d50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 --2- 172.21.15.3:0/1999746968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 0x7f9238100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 --2- 172.21.15.3:0/1999746968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9238101b00 0x7f9238101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 -- 172.21.15.3:0/1999746968 >> 172.21.15.3:0/1999746968 conn(0x7f92380fbe60 msgr2=0x7f92380fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:16.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 -- 172.21.15.3:0/1999746968 shutdown_connections 2023-08-11T14:22:16.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.871+0000 7f923fdff700 1 -- 172.21.15.3:0/1999746968 wait complete. 2023-08-11T14:22:16.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 Processor -- start 2023-08-11T14:22:16.875 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 -- start start 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 0x7f92380746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92381024a0 0x7f9238073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92380739f0 con 0x7f9238101b00 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238073b60 con 0x7f9238100910 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.872+0000 7f923fdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92381a0780 con 0x7f92381024a0 2023-08-11T14:22:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923db9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 0x7f92380746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42610/0 (socket says 172.21.15.3:42610) 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 -- 172.21.15.3:0/3286828672 learned_addr learned my addr 172.21.15.3:0/3286828672 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923e39c700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92381024a0 0x7f9238073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 -- 172.21.15.3:0/3286828672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92381024a0 msgr2=0x7f9238073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92381024a0 0x7f9238073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 -- 172.21.15.3:0/3286828672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 msgr2=0x7f92380746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 0x7f92380746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 -- 172.21.15.3:0/3286828672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009d00 con 0x7f9238101b00 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.873+0000 7f923d39a700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f922800e9a0 tx=0x7f922800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.874+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228009790 con 0x7f9238101b00 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.874+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92381a0a60 con 0x7f9238101b00 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.874+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9228010430 con 0x7f9238101b00 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.874+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228018680 con 0x7f9238101b00 2023-08-11T14:22:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.874+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92381a0fc0 con 0x7f9238101b00 2023-08-11T14:22:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.875+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9238107e60 con 0x7f9238101b00 2023-08-11T14:22:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.877+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f92280105a0 con 0x7f9238101b00 2023-08-11T14:22:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.878+0000 7f922effd700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 0x7f92240a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.878+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f9228014070 con 0x7f9238101b00 2023-08-11T14:22:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.878+0000 7f923db9b700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 0x7f92240a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:16.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.879+0000 7f923db9b700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 0x7f92240a2490 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9238101960 tx=0x7f9234009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:16.882+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f9228051c30 con 0x7f9238101b00 2023-08-11T14:22:17.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:16 smithi003 ceph-mon[97195]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2023-08-11T14:22:17.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:16 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/45434024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:17.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:16 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/116423439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:17.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.092+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f923802cda0 con 0x7f9238101b00 2023-08-11T14:22:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.092+0000 7f922effd700 1 -- 172.21.15.3:0/3286828672 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9228095100 con 0x7f9238101b00 2023-08-11T14:22:17.094 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:22:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 msgr2=0x7f92240a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 0x7f92240a2490 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9238101960 tx=0x7f9234009260 comp rx=0 tx=0).stop 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 msgr2=0x7f9238072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f922800e9a0 tx=0x7f922800ecb0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 shutdown_connections 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92381024a0 0x7f9238073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f922409ffd0 0x7f92240a2490 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9238100910 0x7f92380746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 --2- 172.21.15.3:0/3286828672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9238101b00 0x7f9238072d10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 >> 172.21.15.3:0/3286828672 conn(0x7f92380fbe60 msgr2=0x7f9238104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 shutdown_connections 2023-08-11T14:22:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.095+0000 7f923fdff700 1 -- 172.21.15.3:0/3286828672 wait complete. 2023-08-11T14:22:17.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:22:17.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:16 smithi103 ceph-mon[130434]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2023-08-11T14:22:17.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:16 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/45434024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:17.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:16 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/116423439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:16 smithi187 ceph-mon[130973]: pgmap v897: 129 pgs: 129 active+clean; 8.4 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 5.2 MiB/s wr, 644 op/s 2023-08-11T14:22:17.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:16 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/45434024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:17.295 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:16 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/116423439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 -- 172.21.15.3:0/201531439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c100910 msgr2=0x7fcb6c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 --2- 172.21.15.3:0/201531439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c100910 0x7fcb6c100d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009920 tx=0x7fcb5c009c30 comp rx=0 tx=0).stop 2023-08-11T14:22:17.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 -- 172.21.15.3:0/201531439 shutdown_connections 2023-08-11T14:22:17.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 --2- 172.21.15.3:0/201531439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c1024a0 0x7fcb6c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 --2- 172.21.15.3:0/201531439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c100910 0x7fcb6c100d30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 --2- 172.21.15.3:0/201531439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c101b00 0x7fcb6c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.322+0000 7fcb71dff700 1 -- 172.21.15.3:0/201531439 >> 172.21.15.3:0/201531439 conn(0x7fcb6c0fbe60 msgr2=0x7fcb6c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:17.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.323+0000 7fcb71dff700 1 -- 172.21.15.3:0/201531439 shutdown_connections 2023-08-11T14:22:17.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.323+0000 7fcb71dff700 1 -- 172.21.15.3:0/201531439 wait complete. 2023-08-11T14:22:17.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 Processor -- start 2023-08-11T14:22:17.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 -- start start 2023-08-11T14:22:17.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c1024a0 0x7fcb6c073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0739f0 con 0x7fcb6c101b00 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c073b60 con 0x7fcb6c100910 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb71dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1a0800 con 0x7fcb6c1024a0 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb6affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:44614/0 (socket says 172.21.15.3:44614) 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb6affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42630/0 (socket says 172.21.15.3:42630) 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.324+0000 7fcb6b7fe700 1 -- 172.21.15.3:0/3454842556 learned_addr learned my addr 172.21.15.3:0/3454842556 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6bfff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c1024a0 0x7fcb6c073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 -- 172.21.15.3:0/3454842556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c1024a0 msgr2=0x7fcb6c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c1024a0 0x7fcb6c073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 -- 172.21.15.3:0/3454842556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 msgr2=0x7fcb6c0746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 -- 172.21.15.3:0/3454842556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009520 con 0x7fcb6c101b00 2023-08-11T14:22:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6b7fe700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:17.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb6affd700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000c9c0 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb60010430 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c1a0ae0 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.325+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018680 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.326+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c1a0ff0 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.327+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c108060 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.329+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fcb60010860 con 0x7fcb6c101b00 2023-08-11T14:22:17.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.331+0000 7fcb68ff9700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 0x7fcb540a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.331+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fcb60014070 con 0x7fcb6c101b00 2023-08-11T14:22:17.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.331+0000 7fcb6b7fe700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 0x7fcb540a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.332+0000 7fcb6b7fe700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 0x7fcb540a2410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c010 tx=0x7fcb5c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.335+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fcb60051c30 con 0x7fcb6c101b00 2023-08-11T14:22:17.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.548+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb6c02cdf0 con 0x7fcb6c101b00 2023-08-11T14:22:17.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.548+0000 7fcb68ff9700 1 -- 172.21.15.3:0/3454842556 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb600950e0 con 0x7fcb6c101b00 2023-08-11T14:22:17.551 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 msgr2=0x7fcb540a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 0x7fcb540a2410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c010 tx=0x7fcb5c019040 comp rx=0 tx=0).stop 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 msgr2=0x7fcb6c072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000e9f0 tx=0x7fcb6000ed00 comp rx=0 tx=0).stop 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 shutdown_connections 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb6c1024a0 0x7fcb6c073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fcb5409ff50 0x7fcb540a2410 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb6c100910 0x7fcb6c0746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 --2- 172.21.15.3:0/3454842556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb6c101b00 0x7fcb6c072d10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 >> 172.21.15.3:0/3454842556 conn(0x7fcb6c0fbe60 msgr2=0x7fcb6c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 shutdown_connections 2023-08-11T14:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.551+0000 7fcb71dff700 1 -- 172.21.15.3:0/3454842556 wait complete. 2023-08-11T14:22:17.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:22:17.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 -- 172.21.15.3:0/2924612016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 msgr2=0x7fe68010af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 --2- 172.21.15.3:0/2924612016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68010af80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b0d0 tx=0x7fe67800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 -- 172.21.15.3:0/2924612016 shutdown_connections 2023-08-11T14:22:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 --2- 172.21.15.3:0/2924612016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68010af80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 --2- 172.21.15.3:0/2924612016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 0x7fe680102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 --2- 172.21.15.3:0/2924612016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 0x7fe680104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 -- 172.21.15.3:0/2924612016 >> 172.21.15.3:0/2924612016 conn(0x7fe6800fe0d0 msgr2=0x7fe6801004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 -- 172.21.15.3:0/2924612016 shutdown_connections 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.779+0000 7fe6867ff700 1 -- 172.21.15.3:0/2924612016 wait complete. 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.780+0000 7fe6867ff700 1 Processor -- start 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.780+0000 7fe6867ff700 1 -- start start 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 0x7fe680198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 0x7fe680198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801a04b0 con 0x7fe6801046d0 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801a05f0 con 0x7fe680103d30 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe6867ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801a0780 con 0x7fe680102b40 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe67ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 0x7fe680198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42636/0 (socket says 172.21.15.3:42636) 2023-08-11T14:22:17.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 -- 172.21.15.3:0/3306442452 learned_addr learned my addr 172.21.15.3:0/3306442452 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:17.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe67f7fe700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 0x7fe680198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 -- 172.21.15.3:0/3306442452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 msgr2=0x7fe680198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 0x7fe680198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 -- 172.21.15.3:0/3306442452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 msgr2=0x7fe680198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 0x7fe680198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.781+0000 7fe684d9c700 1 -- 172.21.15.3:0/3306442452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674009520 con 0x7fe6801046d0 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe684d9c700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe678008eb0 tx=0x7fe678008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800e040 con 0x7fe6801046d0 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe67800bdb0 con 0x7fe6801046d0 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67801d470 con 0x7fe6801046d0 2023-08-11T14:22:17.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe678009d00 con 0x7fe6801046d0 2023-08-11T14:22:17.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.782+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6801a0ca0 con 0x7fe6801046d0 2023-08-11T14:22:17.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.783+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe680109a70 con 0x7fe6801046d0 2023-08-11T14:22:17.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.785+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fe678005020 con 0x7fe6801046d0 2023-08-11T14:22:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.787+0000 7fe67d7fa700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 0x7fe6680a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.787+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fe6780cc530 con 0x7fe6801046d0 2023-08-11T14:22:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.787+0000 7fe67ffff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 0x7fe6680a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.788+0000 7fe67ffff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 0x7fe6680a2410 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe680103b90 tx=0x7fe670009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:17.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:17.790+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fe678095fd0 con 0x7fe6801046d0 2023-08-11T14:22:18.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.006+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe68002cd70 con 0x7fe6801046d0 2023-08-11T14:22:18.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.007+0000 7fe67d7fa700 1 -- 172.21.15.3:0/3306442452 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe6780995f0 con 0x7fe6801046d0 2023-08-11T14:22:18.010 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 msgr2=0x7fe6680a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 0x7fe6680a2410 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe680103b90 tx=0x7fe670009260 comp rx=0 tx=0).stop 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 msgr2=0x7fe68019fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe678008eb0 tx=0x7fe678008ee0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 shutdown_connections 2023-08-11T14:22:18.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6801046d0 0x7fe68019fee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe680102b40 0x7fe680198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fe66809ff50 0x7fe6680a2410 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 --2- 172.21.15.3:0/3306442452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe680103d30 0x7fe680198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.009+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 >> 172.21.15.3:0/3306442452 conn(0x7fe6800fe0d0 msgr2=0x7fe680100300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.010+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 shutdown_connections 2023-08-11T14:22:18.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.010+0000 7fe6867ff700 1 -- 172.21.15.3:0/3306442452 wait complete. 2023-08-11T14:22:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:17 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3286828672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:17 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3454842556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-08-11T14:22:18.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:17 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3286828672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:17 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3454842556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2001365279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 msgr2=0x7f6c24100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2001365279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c24100910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b0a0 tx=0x7f6c1c00b3b0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2001365279 shutdown_connections 2023-08-11T14:22:18.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2001365279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c24100910 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2001365279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 0x7f6c240ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2001365279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 0x7f6c240fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.233+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2001365279 >> 172.21.15.3:0/2001365279 conn(0x7f6c240fad50 msgr2=0x7f6c240fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.234+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2001365279 shutdown_connections 2023-08-11T14:22:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.234+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2001365279 wait complete. 2023-08-11T14:22:18.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 Processor -- start 2023-08-11T14:22:18.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 -- start start 2023-08-11T14:22:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 0x7f6c241988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:18.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 0x7f6c24198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:18.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:18.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241a0440 con 0x7f6c24100490 2023-08-11T14:22:18.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241a0580 con 0x7f6c240ff170 2023-08-11T14:22:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2b1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241a0710 con 0x7f6c240ffad0 2023-08-11T14:22:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2979c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2979c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42650/0 (socket says 172.21.15.3:42650) 2023-08-11T14:22:18.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c2979c700 1 -- 172.21.15.3:0/2763218690 learned_addr learned my addr 172.21.15.3:0/2763218690 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c28f9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 0x7f6c241988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.235+0000 7f6c23fff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 0x7f6c24198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:18.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 -- 172.21.15.3:0/2763218690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 msgr2=0x7f6c24198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 0x7f6c24198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 -- 172.21.15.3:0/2763218690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 msgr2=0x7f6c241988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 0x7f6c241988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 -- 172.21.15.3:0/2763218690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009d00 con 0x7f6c24100490 2023-08-11T14:22:18.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2979c700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b070 tx=0x7f6c1c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e040 con 0x7f6c24100490 2023-08-11T14:22:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c1c004dd0 con 0x7f6c24100490 2023-08-11T14:22:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c241a0990 con 0x7f6c24100490 2023-08-11T14:22:18.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01b6b0 con 0x7f6c24100490 2023-08-11T14:22:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.236+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c241a0e60 con 0x7f6c24100490 2023-08-11T14:22:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.237+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24109f50 con 0x7f6c24100490 2023-08-11T14:22:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.239+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f6c1c003a00 con 0x7f6c24100490 2023-08-11T14:22:18.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.241+0000 7f6c21ffb700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 0x7f6c140a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.241+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f6c1c0cc9c0 con 0x7f6c24100490 2023-08-11T14:22:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.241+0000 7f6c28f9b700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 0x7f6c140a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.241+0000 7f6c28f9b700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 0x7f6c140a24e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c18007c60 tx=0x7f6c180073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:18.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.243+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f6c1c096460 con 0x7f6c24100490 2023-08-11T14:22:18.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:17 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3286828672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:17 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3454842556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:18.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.523+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f6c2410e540 con 0x7f6c24100490 2023-08-11T14:22:18.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.534+0000 7f6c21ffb700 1 -- 172.21.15.3:0/2763218690 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+727847 (secure 0 0 0) 0x7f6c1c099a80 con 0x7f6c24100490 2023-08-11T14:22:18.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:18.544 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:22:18.544 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 18, 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 14150, 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-08-11T14:22:18.545 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2363526909 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2363526909 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.546 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/2363526909", 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2023-08-11T13:55:02.079157+0000", 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540138322906710015, 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.547 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24109, 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540138322906710015, 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.548 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:18.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.550 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.552 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.553 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.555 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.556 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.567 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2023-08-11T14:22:18.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.577 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2023-08-11T14:22:18.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.578 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.579 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2023-08-11T14:22:18.582 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.583 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:18.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2023-08-11T14:22:18.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T14:22:18.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2023-08-11T14:22:18.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.592 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:18.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.595 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.600 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2023-08-11T14:22:18.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap" 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2023-08-11T14:22:18.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:18.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.613 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.615 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2023-08-11T14:22:18.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.618 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2023-08-11T14:22:18.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.621 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2023-08-11T14:22:18.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2023-08-11T14:22:18.624 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T14:22:18.625 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:18.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2023-08-11T14:22:18.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2023-08-11T14:22:18.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2023-08-11T14:22:18.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2023-08-11T14:22:18.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.641 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T14:22:18.648 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.649 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2023-08-11T14:22:18.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2023-08-11T14:22:18.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.657 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2023-08-11T14:22:18.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2023-08-11T14:22:18.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.657 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.657 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T14:22:18.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.660 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2023-08-11T14:22:18.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T14:22:18.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2023-08-11T14:22:18.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.671 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2023-08-11T14:22:18.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2023-08-11T14:22:18.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.677 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.678 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.681 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.684 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.689 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2023-08-11T14:22:18.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T14:22:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.699 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.702 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.703 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.704 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2023-08-11T14:22:18.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.708 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T14:22:18.709 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.715 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2023-08-11T14:22:18.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.722 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.723 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.725 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.729 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2023-08-11T14:22:18.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T14:22:18.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.738 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.739 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T14:22:18.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.747 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T14:22:18.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T14:22:18.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:18.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.764 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2023-08-11T14:22:18.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.773 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T14:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.782 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T14:22:18.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T14:22:18.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T14:22:18.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T14:22:18.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.802 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T14:22:18.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T14:22:18.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T14:22:18.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.810 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.811 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2023-08-11T14:22:18.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2023-08-11T14:22:18.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.826 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.830 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.837 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:18.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.844 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2023-08-11T14:22:18.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2023-08-11T14:22:18.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.854 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.859 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.860 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2023-08-11T14:22:18.862 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T14:22:18.863 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.869 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:18.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2023-08-11T14:22:18.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.877 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.882 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.883 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2023-08-11T14:22:18.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2023-08-11T14:22:18.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:18.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2023-08-11T14:22:18.891 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2023-08-11T14:22:18.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.894 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2023-08-11T14:22:18.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:18.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T14:22:18.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.901 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.905 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2023-08-11T14:22:18.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2023-08-11T14:22:18.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2023-08-11T14:22:18.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2023-08-11T14:22:18.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.922 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.932 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.936 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.940 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2023-08-11T14:22:18.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T14:22:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.945 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.950 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2023-08-11T14:22:18.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.968 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2023-08-11T14:22:18.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T14:22:18.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.974 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.976 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.979 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.981 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.982 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.986 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.991 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:18.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:18.995 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.996 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:18.997 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:18.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:18.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.002 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.006 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.010 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.014 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2023-08-11T14:22:19.020 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.022 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.025 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.031 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.032 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.033 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.038 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2023-08-11T14:22:19.042 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.044 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.049 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "noautoscale": { 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "noautoscale", 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "global autoscale flag", 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2023-08-11T14:22:19.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.058 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T14:22:19.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.064 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.069 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.070 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2023-08-11T14:22:19.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2023-08-11T14:22:19.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2023-08-11T14:22:19.076 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.082 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.082 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.087 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:19.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:19.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T14:22:19.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.106 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.113 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.114 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.119 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2023-08-11T14:22:19.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.127 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.128 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.131 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.136 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:19.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:19.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.138 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.139 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.141 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.145 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.149 INFO:teuthology.orchestra.run.smithi003.stdout: "drive_group_interval": { 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "drive_group_interval", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300.0", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.152 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.157 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.164 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.169 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.185 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2023-08-11T14:22:19.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.195 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.197 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.200 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.201 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.204 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.205 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.206 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.210 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.215 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.219 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.220 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.221 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.226 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.227 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.229 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.230 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:19.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.235 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.240 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.243 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.244 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.246 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.250 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2023-08-11T14:22:19.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2023-08-11T14:22:19.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T14:22:19.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:19.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.271 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.276 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.278 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2023-08-11T14:22:19.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.286 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.288 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.289 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.292 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2023-08-11T14:22:19.296 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.298 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.304 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.306 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.307 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.310 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.312 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:19.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.317 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.328 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.330 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.332 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24115, 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540138322906710015, 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2023-08-11T14:22:19.333 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.337 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.342 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2023-08-11T14:22:19.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2023-08-11T14:22:19.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.358 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2023-08-11T14:22:19.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T14:22:19.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:19.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2023-08-11T14:22:19.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.368 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2023-08-11T14:22:19.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2023-08-11T14:22:19.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2023-08-11T14:22:19.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T14:22:19.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:19.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T14:22:19.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.378 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.382 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.383 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap" 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T14:22:19.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:19.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:19.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T14:22:19.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2023-08-11T14:22:19.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:19.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T14:22:19.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2023-08-11T14:22:19.399 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.402 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2023-08-11T14:22:19.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2023-08-11T14:22:19.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:19.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.409 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2023-08-11T14:22:19.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2023-08-11T14:22:19.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T14:22:19.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.424 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2023-08-11T14:22:19.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2023-08-11T14:22:19.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T14:22:19.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.433 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2023-08-11T14:22:19.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2023-08-11T14:22:19.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T14:22:19.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.444 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2023-08-11T14:22:19.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T14:22:19.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.453 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2023-08-11T14:22:19.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2023-08-11T14:22:19.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.460 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.467 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2023-08-11T14:22:19.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:19.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.480 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.481 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2023-08-11T14:22:19.483 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.484 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T14:22:19.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.492 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2023-08-11T14:22:19.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T14:22:19.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.497 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.498 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.501 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T14:22:19.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.506 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.512 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.517 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.519 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.521 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.525 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T14:22:19.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.532 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.534 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T14:22:19.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.542 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.545 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T14:22:19.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.554 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T14:22:19.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.563 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T14:22:19.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T14:22:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.580 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.581 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.582 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T14:22:19.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.584 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.585 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.587 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.592 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:19.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.598 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2023-08-11T14:22:19.600 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.609 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.610 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.613 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2023-08-11T14:22:19.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:19.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.621 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.622 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.624 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:19.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.628 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:19.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.631 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2023-08-11T14:22:19.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2023-08-11T14:22:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.637 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.640 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.642 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2023-08-11T14:22:19.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2023-08-11T14:22:19.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T14:22:19.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T14:22:19.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.658 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.660 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.664 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2023-08-11T14:22:19.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2023-08-11T14:22:19.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.684 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.685 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.687 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.688 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.695 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2023-08-11T14:22:19.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.698 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.701 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.703 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.704 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.705 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.710 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.711 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.713 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.715 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.720 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.726 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.729 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.730 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.733 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.737 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2023-08-11T14:22:19.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.739 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.741 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.745 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.752 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2023-08-11T14:22:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2023-08-11T14:22:19.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.763 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.765 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2023-08-11T14:22:19.768 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.770 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2023-08-11T14:22:19.778 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.780 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.780 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.785 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.786 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2023-08-11T14:22:19.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T14:22:19.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.792 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.797 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2023-08-11T14:22:19.801 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.802 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.804 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.805 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.809 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.812 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.814 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.819 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.824 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.826 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.829 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.830 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "noautoscale": { 2023-08-11T14:22:19.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "noautoscale", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "global autoscale flag", 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T14:22:19.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.838 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.843 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.844 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.848 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2023-08-11T14:22:19.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:19.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2023-08-11T14:22:19.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.860 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.861 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.862 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.865 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.870 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2023-08-11T14:22:19.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2023-08-11T14:22:19.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2023-08-11T14:22:19.881 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.882 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:19.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.884 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.885 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.887 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.892 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.900 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.901 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:19.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.906 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.911 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.912 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:19.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.918 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.920 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.921 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.924 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.928 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "drive_group_interval": { 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "drive_group_interval", 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300.0", 2023-08-11T14:22:19.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.932 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.936 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.941 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.943 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.948 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2023-08-11T14:22:19.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.957 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2023-08-11T14:22:19.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2023-08-11T14:22:19.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2023-08-11T14:22:19.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2023-08-11T14:22:19.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2023-08-11T14:22:19.973 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.977 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.979 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.982 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.983 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.985 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.986 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.987 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:19.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.992 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.993 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:19.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:19.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:19.998 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:19.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.000 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.003 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.004 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2023-08-11T14:22:20.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T14:22:20.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2023-08-11T14:22:20.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2023-08-11T14:22:20.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2023-08-11T14:22:20.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2023-08-11T14:22:20.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.041 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2023-08-11T14:22:20.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.046 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.051 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2023-08-11T14:22:20.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.057 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2023-08-11T14:22:20.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2023-08-11T14:22:20.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.061 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.063 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.064 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.067 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.071 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.073 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.077 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2023-08-11T14:22:20.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.082 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2023-08-11T14:22:20.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2023-08-11T14:22:20.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.085 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2023-08-11T14:22:20.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.092 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.097 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2023-08-11T14:22:20.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2023-08-11T14:22:20.106 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "restful" 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.111 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.116 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.124 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2023-08-11T14:22:20.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2023-08-11T14:22:20.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.133 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2023-08-11T14:22:20.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:20.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2023-08-11T14:22:20.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.143 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2023-08-11T14:22:20.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2023-08-11T14:22:20.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-08-11T14:22:20.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.150 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.153 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.160 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap" 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-08-11T14:22:20.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.169 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.170 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2023-08-11T14:22:20.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-08-11T14:22:20.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-08-11T14:22:20.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2023-08-11T14:22:20.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.181 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2023-08-11T14:22:20.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-08-11T14:22:20.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-08-11T14:22:20.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.194 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2023-08-11T14:22:20.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2023-08-11T14:22:20.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.201 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.202 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.204 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.207 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.1", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2023-08-11T14:22:20.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.216 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2023-08-11T14:22:20.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.219 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2023-08-11T14:22:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-08-11T14:22:20.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.228 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2023-08-11T14:22:20.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2023-08-11T14:22:20.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2023-08-11T14:22:20.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.232 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.232 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.235 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.235 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.236 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.236 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.236 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.238 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2023-08-11T14:22:20.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2023-08-11T14:22:20.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-08-11T14:22:20.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.243 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-08-11T14:22:20.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.250 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.250 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.250 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.251 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.251 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.251 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.251 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.254 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.259 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2023-08-11T14:22:20.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-08-11T14:22:20.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.262 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-08-11T14:22:20.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-08-11T14:22:20.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2023-08-11T14:22:20.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-08-11T14:22:20.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2023-08-11T14:22:20.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2023-08-11T14:22:20.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2023-08-11T14:22:20.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2023-08-11T14:22:20.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2023-08-11T14:22:20.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2023-08-11T14:22:20.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.339 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2023-08-11T14:22:20.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2023-08-11T14:22:20.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2023-08-11T14:22:20.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-08-11T14:22:20.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.347 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2023-08-11T14:22:20.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2023-08-11T14:22:20.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-08-11T14:22:20.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2023-08-11T14:22:20.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.356 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.358 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2023-08-11T14:22:20.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2023-08-11T14:22:20.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2023-08-11T14:22:20.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.364 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.368 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.370 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.371 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.372 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.374 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2023-08-11T14:22:20.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.382 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.382 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.382 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2023-08-11T14:22:20.383 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.383 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.385 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-08-11T14:22:20.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.388 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2023-08-11T14:22:20.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2023-08-11T14:22:20.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-08-11T14:22:20.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2023-08-11T14:22:20.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-08-11T14:22:20.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:20.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.417 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-08-11T14:22:20.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.430 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-08-11T14:22:20.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2023-08-11T14:22:20.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-08-11T14:22:20.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2023-08-11T14:22:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.450 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-08-11T14:22:20.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-08-11T14:22:20.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.465 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2023-08-11T14:22:20.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.474 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.475 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.478 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.482 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2023-08-11T14:22:20.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2023-08-11T14:22:20.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2023-08-11T14:22:20.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.491 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:20.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.494 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.495 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:20.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2023-08-11T14:22:20.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.499 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.503 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.505 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.506 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.508 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2023-08-11T14:22:20.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2023-08-11T14:22:20.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2023-08-11T14:22:20.514 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.515 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:20.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.524 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.527 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.528 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.531 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.532 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2023-08-11T14:22:20.535 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2023-08-11T14:22:20.541 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2023-08-11T14:22:20.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.543 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2023-08-11T14:22:20.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-08-11T14:22:20.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.550 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.551 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.552 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.555 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.556 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.557 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2023-08-11T14:22:20.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2023-08-11T14:22:20.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2023-08-11T14:22:20.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.569 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.571 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.573 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.576 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.581 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.582 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.582 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.582 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.582 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.582 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.586 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.590 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2023-08-11T14:22:20.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2023-08-11T14:22:20.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.600 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.604 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.607 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.609 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.612 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2023-08-11T14:22:20.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2023-08-11T14:22:20.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2023-08-11T14:22:20.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.625 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.626 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.627 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.628 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2023-08-11T14:22:20.635 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.637 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.640 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.647 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.650 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.652 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:20.655 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.659 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.660 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.664 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2023-08-11T14:22:20.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2023-08-11T14:22:20.670 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.672 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.675 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.677 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.680 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.682 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.687 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.688 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.689 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.691 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.692 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.692 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.694 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.698 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.699 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.702 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "noautoscale": { 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "noautoscale", 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.703 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "global autoscale flag", 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.704 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2023-08-11T14:22:20.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.714 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.715 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.718 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2023-08-11T14:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.723 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2023-08-11T14:22:20.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-08-11T14:22:20.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.731 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.734 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.738 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.739 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.744 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:20.746 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-08-11T14:22:20.747 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2023-08-11T14:22:20.754 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.756 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.765 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.769 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2023-08-11T14:22:20.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2023-08-11T14:22:20.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.775 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.780 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.781 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2023-08-11T14:22:20.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.787 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.789 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.790 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.793 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.794 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.796 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "drive_group_interval": { 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "drive_group_interval", 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300.0", 2023-08-11T14:22:20.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.799 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.801 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.805 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.810 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.811 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.813 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.817 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.822 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2023-08-11T14:22:20.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.831 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2023-08-11T14:22:20.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2023-08-11T14:22:20.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.843 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2023-08-11T14:22:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.853 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.854 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.858 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.862 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.863 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.866 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.867 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.869 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.873 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.877 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-08-11T14:22:20.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.879 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.882 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.887 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.890 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2023-08-11T14:22:20.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2023-08-11T14:22:20.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-08-11T14:22:20.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.904 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2023-08-11T14:22:20.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2023-08-11T14:22:20.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.917 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.918 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.921 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.931 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.937 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.940 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.942 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.947 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2023-08-11T14:22:20.951 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.952 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2023-08-11T14:22:20.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.962 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.963 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2023-08-11T14:22:20.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2023-08-11T14:22:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2023-08-11T14:22:20.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-08-11T14:22:20.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.975 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2023-08-11T14:22:20.976 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2023-08-11T14:22:20.977 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2023-08-11T14:22:20.978 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2023-08-11T14:22:20.979 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2023-08-11T14:22:20.980 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2023-08-11T14:22:20.981 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2023-08-11T14:22:20.982 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 3, 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2023-08-11T14:22:20.983 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 485728070 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.984 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1234370696 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.985 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-08-11T14:22:20.986 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3364078261 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2023-08-11T14:22:20.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2459012490 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:20.988 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-08-11T14:22:20.989 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:22:20.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:18 smithi187 ceph-mon[130973]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 725 op/s 2023-08-11T14:22:20.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:18 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3306442452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:20.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:18 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2763218690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T14:22:20.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:20 smithi187 ceph-mon[130973]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.0 MiB/s wr, 497 op/s 2023-08-11T14:22:20.991 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:18 smithi103 ceph-mon[130434]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 725 op/s 2023-08-11T14:22:20.991 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:18 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3306442452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:20.991 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:18 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2763218690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T14:22:20.991 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:20 smithi103 ceph-mon[130434]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.0 MiB/s wr, 497 op/s 2023-08-11T14:22:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 msgr2=0x7f6c140a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 0x7f6c140a24e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c18007c60 tx=0x7f6c180073d0 comp rx=0 tx=0).stop 2023-08-11T14:22:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 msgr2=0x7f6c2419fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b070 tx=0x7f6c1c004a50 comp rx=0 tx=0).stop 2023-08-11T14:22:20.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 shutdown_connections 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c24100490 0x7f6c2419fe70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f6c140a0020 0x7f6c140a24e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c240ff170 0x7f6c241988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 --2- 172.21.15.3:0/2763218690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c240ffad0 0x7f6c24198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 >> 172.21.15.3:0/2763218690 conn(0x7f6c240fad50 msgr2=0x7f6c240fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 shutdown_connections 2023-08-11T14:22:20.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:18.542+0000 7f6c2b1ff700 1 -- 172.21.15.3:0/2763218690 wait complete. 2023-08-11T14:22:21.000 DEBUG:tasks.check_counter:Getting stats from a 2023-08-11T14:22:21.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.a perf dump 2023-08-11T14:22:21.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:18 smithi003 ceph-mon[97195]: pgmap v898: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.9 MiB/s wr, 725 op/s 2023-08-11T14:22:21.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:18 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3306442452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:21.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:18 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2763218690' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-08-11T14:22:21.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:20 smithi003 ceph-mon[97195]: pgmap v899: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.0 MiB/s wr, 497 op/s 2023-08-11T14:22:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:22:22.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:22 smithi003 ceph-mon[97195]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 391 op/s 2023-08-11T14:22:23.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:22 smithi187 ceph-mon[130973]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 391 op/s 2023-08-11T14:22:23.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:22 smithi103 ceph-mon[130434]: pgmap v900: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 391 op/s 2023-08-11T14:22:23.866 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:22:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2612, 2023-08-11T14:22:23.867 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2614, 2023-08-11T14:22:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2647638, 2023-08-11T14:22:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 812333, 2023-08-11T14:22:23.868 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 6, 2023-08-11T14:22:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.320152773, 2023-08-11T14:22:23.869 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.092312790, 2023-08-11T14:22:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.092721921, 2023-08-11T14:22:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.102668756, 2023-08-11T14:22:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:23.871 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2612, 2023-08-11T14:22:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 1.181452264, 2023-08-11T14:22:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000452317 2023-08-11T14:22:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:23.872 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1062, 2023-08-11T14:22:23.874 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 1314, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 263826, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 841739, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 6, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 4, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.159910365, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.056392562, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.033799656, 2023-08-11T14:22:23.875 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.056096880, 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1313, 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.610349608, 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000464851 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:23.876 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 512944 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:23.877 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 378, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 344, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 718748, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 203431, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.073676913, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.028916236, 2023-08-11T14:22:23.878 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.029158501, 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.015212385, 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 343, 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.120328642, 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000350812 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.879 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 718480, 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 203120 2023-08-11T14:22:23.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2023-08-11T14:22:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2023-08-11T14:22:23.885 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 6135, 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.204003736, 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000033252 2023-08-11T14:22:23.886 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.887 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.888 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 0, 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 0, 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.889 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 0, 2023-08-11T14:22:23.890 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 389, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 1, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 20, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 368, 2023-08-11T14:22:23.891 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 387, 2023-08-11T14:22:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 4, 2023-08-11T14:22:23.892 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 14, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 0, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 0, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2023-08-11T14:22:23.893 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2023-08-11T14:22:23.894 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 0, 2023-08-11T14:22:23.895 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:23.896 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:23.897 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 0, 2023-08-11T14:22:23.898 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2023-08-11T14:22:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 0, 2023-08-11T14:22:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2023-08-11T14:22:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.899 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2023-08-11T14:22:23.900 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:23.901 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:23.902 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 0, 2023-08-11T14:22:23.903 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2023-08-11T14:22:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2023-08-11T14:22:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 1374, 2023-08-11T14:22:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2023-08-11T14:22:23.904 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2023-08-11T14:22:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 0, 2023-08-11T14:22:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2023-08-11T14:22:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2023-08-11T14:22:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 3, 2023-08-11T14:22:23.905 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2023-08-11T14:22:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2023-08-11T14:22:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2023-08-11T14:22:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2023-08-11T14:22:23.906 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 6075123, 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 6074524, 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.907 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 1374 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 377, 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 389, 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 12, 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 25, 2023-08-11T14:22:23.908 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 26, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 1, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 389, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 400, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 11, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 0, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 0, 2023-08-11T14:22:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 50188, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 207152 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 0, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 0, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 0, 2023-08-11T14:22:23.910 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.911 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.912 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2023-08-11T14:22:23.913 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.914 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2023-08-11T14:22:23.915 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.917 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.919 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.920 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.921 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:23.922 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:23.924 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.926 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2023-08-11T14:22:23.928 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2023-08-11T14:22:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 1, 2023-08-11T14:22:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 1, 2023-08-11T14:22:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 0, 2023-08-11T14:22:23.931 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 1, 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 0, 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 3221 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.932 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:23.933 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:23.934 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:23.935 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:23.936 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 203783, 2023-08-11T14:22:23.937 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 420, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:23.938 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 167, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2023-08-11T14:22:23.939 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 1156301, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 10262, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.940 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 3670, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 21, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 3, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 3670, 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2023-08-11T14:22:23.941 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3670, 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 51.075354019, 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.013916990 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3670, 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 3671 2023-08-11T14:22:23.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 3670, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 3670, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 0, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 2226, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 0, 2023-08-11T14:22:23.943 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 1442, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2023-08-11T14:22:23.944 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 1, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:23.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:23.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2023-08-11T14:22:23.947 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2023-08-11T14:22:23.948 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 91, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 4, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 5, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 7, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 2, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2023-08-11T14:22:23.949 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 0, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 2, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 0 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 0, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2023-08-11T14:22:23.950 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "osd-slow-ops": { 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 0 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:23.951 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:23.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 4048, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 3243666, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 4048, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:23.953 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 4048, 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 3243666, 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:23.955 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 3670, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 1880824, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 3670, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1880824, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.956 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:23.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 3670, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 3670, 2023-08-11T14:22:23.958 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 3670, 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 3670, 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.959 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:23.960 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.961 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x55c1ec562500": { 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:23.962 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:23.963 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.964 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:23.964 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:22:24.348 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-08-11T14:22:24.348 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-08-11T14:22:24.349 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-08-11T14:22:24.349 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-08-11T14:22:24.349 DEBUG:tasks.check_counter:Getting stats from d 2023-08-11T14:22:24.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.d perf dump 2023-08-11T14:22:25.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:24 smithi003 ceph-mon[97195]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-08-11T14:22:25.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:24 smithi187 ceph-mon[130973]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-08-11T14:22:25.114 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:22:25.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:24 smithi103 ceph-mon[130434]: pgmap v901: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 296 op/s 2023-08-11T14:22:26.740 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2992, 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:26.741 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.004092904, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000081514, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001261837, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.001272917, 2023-08-11T14:22:26.742 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000424305 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-11T14:22:26.743 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T14:22:26.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.744 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:26.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 343, 2023-08-11T14:22:26.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 340, 2023-08-11T14:22:26.744 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 504464, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 205040, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.064141100, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.028242821, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.024121997, 2023-08-11T14:22:26.745 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.010129165, 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 339, 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.076628972, 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000226044 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:26.746 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 504464, 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 205040 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2023-08-11T14:22:26.747 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 260, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 198368, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.026144338, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.020613922, 2023-08-11T14:22:26.748 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.002249342, 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 258, 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.058712737, 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000227568 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:26.749 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 198368 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2023-08-11T14:22:26.750 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:26.751 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:26.752 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:26.753 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:26.754 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:26.755 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148704, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 29, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:26.756 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:26.757 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2023-08-11T14:22:26.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 345, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 455978, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 345, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 345, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 455978, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:26.760 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:26.761 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:26.761 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:26.761 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:22:27.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:26 smithi003 ceph-mon[97195]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2023-08-11T14:22:27.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:26 smithi187 ceph-mon[130973]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2023-08-11T14:22:27.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:26 smithi103 ceph-mon[130434]: pgmap v902: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2023-08-11T14:22:27.342 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-08-11T14:22:27.342 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-08-11T14:22:27.343 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-08-11T14:22:27.343 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-08-11T14:22:27.343 DEBUG:tasks.check_counter:Getting stats from g 2023-08-11T14:22:27.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.g perf dump 2023-08-11T14:22:28.085 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:22:28.991 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:28 smithi003 ceph-mon[97195]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 972 KiB/s wr, 140 op/s 2023-08-11T14:22:29.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:28 smithi187 ceph-mon[130973]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 972 KiB/s wr, 140 op/s 2023-08-11T14:22:29.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:28 smithi103 ceph-mon[130434]: pgmap v903: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 972 KiB/s wr, 140 op/s 2023-08-11T14:22:29.802 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:22:29.803 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:29.803 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1047, 2023-08-11T14:22:29.803 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 1047, 2023-08-11T14:22:29.803 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 353385, 2023-08-11T14:22:29.804 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 324822, 2023-08-11T14:22:29.804 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 7, 2023-08-11T14:22:29.804 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:29.804 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.164048208, 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.064128812, 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.037202932, 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.046563437, 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1046, 2023-08-11T14:22:29.805 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.654617795, 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000625829 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2211, 2023-08-11T14:22:29.807 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2467, 2023-08-11T14:22:29.808 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2870090, 2023-08-11T14:22:29.808 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 1203351, 2023-08-11T14:22:29.808 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2023-08-11T14:22:29.808 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 4, 2023-08-11T14:22:29.808 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.252552605, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.086057750, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.067180269, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.081358306, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2465, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.743764180, 2023-08-11T14:22:29.809 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000301729 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 515984 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 383, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 349, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 721268, 2023-08-11T14:22:29.811 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 205648, 2023-08-11T14:22:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2023-08-11T14:22:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.069649481, 2023-08-11T14:22:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.029137068, 2023-08-11T14:22:29.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.030046165, 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.010292473, 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 348, 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.122705885, 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000352603 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:29.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 719920, 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 204704 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2023-08-11T14:22:29.814 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 5217, 2023-08-11T14:22:29.815 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.175702173, 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000033678 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2023-08-11T14:22:29.816 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2023-08-11T14:22:29.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.817 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.817 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 0, 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 0, 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2023-08-11T14:22:29.818 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:29.819 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 0, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 477, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 0, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 0, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 477, 2023-08-11T14:22:29.820 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 477, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 9, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 14, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 0, 2023-08-11T14:22:29.821 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 0, 2023-08-11T14:22:29.822 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2023-08-11T14:22:29.822 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 0, 2023-08-11T14:22:29.822 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:29.822 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:29.823 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2023-08-11T14:22:29.823 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2023-08-11T14:22:29.823 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2023-08-11T14:22:29.823 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2023-08-11T14:22:29.824 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2023-08-11T14:22:29.824 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 0, 2023-08-11T14:22:29.824 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 0, 2023-08-11T14:22:29.825 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:29.825 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:29.825 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:29.825 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 0, 2023-08-11T14:22:29.826 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 0, 2023-08-11T14:22:29.826 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2023-08-11T14:22:29.826 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:29.827 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:29.827 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:29.827 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:29.827 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:29.828 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:29.828 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:29.828 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:29.829 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:29.829 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T14:22:29.829 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2023-08-11T14:22:29.831 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:29.831 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 0, 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 0, 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2023-08-11T14:22:29.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2023-08-11T14:22:29.833 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:29.834 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:29.835 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 0, 2023-08-11T14:22:29.836 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 1161, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2023-08-11T14:22:29.837 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 3, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 6503838, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 6503264, 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2023-08-11T14:22:29.838 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 1161 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 478, 2023-08-11T14:22:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 478, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 0, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 34, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 34, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 0, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 477, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 486, 2023-08-11T14:22:29.840 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 9, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 0, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 0, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 51576, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 207152 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2023-08-11T14:22:29.841 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.842 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:29.843 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.845 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.846 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.847 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.848 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.849 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2023-08-11T14:22:29.850 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.852 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.853 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.854 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.855 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.856 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:29.857 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.858 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.858 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.858 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2023-08-11T14:22:29.858 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.860 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.861 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 1, 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 1, 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 0, 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 1, 2023-08-11T14:22:29.862 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 0, 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 3228 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:29.863 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:29.864 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:29.866 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:29.867 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 215674, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 527, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2023-08-11T14:22:29.868 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 167, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:29.869 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 1458463, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 13000, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:29.870 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 3256, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 21, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 3, 2023-08-11T14:22:29.871 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 3256, 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3256, 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 189.836599644, 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.058303623 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2023-08-11T14:22:29.872 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3256, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 3257 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 3256, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 3256, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 0, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2023-08-11T14:22:29.873 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 1928, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 1326, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2023-08-11T14:22:29.874 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 1, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 0, 2023-08-11T14:22:29.875 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2023-08-11T14:22:29.876 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2023-08-11T14:22:29.877 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2023-08-11T14:22:29.878 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 91, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 4, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 6, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 8, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 2, 2023-08-11T14:22:29.879 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 0, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 2, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 0 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 0, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: "osd-slow-ops": { 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 0 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2023-08-11T14:22:29.881 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:29.882 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 3637, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 3595044, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 3637, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:29.883 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 3637, 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 3595044, 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:29.885 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 3256, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 2309539, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 3256, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 2309539, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.886 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:29.887 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 3256, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 3256, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 3256, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 3256, 2023-08-11T14:22:29.888 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:29.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:29.890 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:29.891 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x557046152500": { 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:29.892 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:29.893 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:29.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:29.894 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:22:30.481 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-08-11T14:22:30.481 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-08-11T14:22:30.481 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-08-11T14:22:30.481 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-08-11T14:22:30.482 DEBUG:tasks.check_counter:Getting stats from j 2023-08-11T14:22:30.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.j perf dump 2023-08-11T14:22:30.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:30 smithi003 ceph-mon[97195]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-08-11T14:22:30.544 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:30 smithi187 ceph-mon[130973]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-08-11T14:22:30.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:30 smithi103 ceph-mon[130434]: pgmap v904: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-08-11T14:22:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.a/config 2023-08-11T14:22:32.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:32 smithi003 ceph-mon[97195]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2023-08-11T14:22:32.808 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-08-11T14:22:32.809 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:32.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2023-08-11T14:22:32.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2023-08-11T14:22:32.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2992, 2023-08-11T14:22:32.809 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2023-08-11T14:22:32.810 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:32.810 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2023-08-11T14:22:32.810 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.004030494, 2023-08-11T14:22:32.810 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000060029, 2023-08-11T14:22:32.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001231609, 2023-08-11T14:22:32.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T14:22:32.812 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:32.812 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.001247940, 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000415980 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:32.813 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:32.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.814 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-11T14:22:32.814 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T14:22:32.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.814 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 262, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 198720, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:32.815 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.026907406, 2023-08-11T14:22:32.816 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.021275983, 2023-08-11T14:22:32.816 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.002666696, 2023-08-11T14:22:32.816 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-08-11T14:22:32.816 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:32.816 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 260, 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.044519797, 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000171229 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:32.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 198720 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 346, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 343, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 505280, 2023-08-11T14:22:32.818 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 206912, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.065698750, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.029316209, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.024140543, 2023-08-11T14:22:32.819 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.010534735, 2023-08-11T14:22:32.820 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:32.820 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 342, 2023-08-11T14:22:32.820 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.102972439, 2023-08-11T14:22:32.820 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000301089 2023-08-11T14:22:32.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 505280, 2023-08-11T14:22:32.821 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 206912 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2023-08-11T14:22:32.822 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:32.823 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:32.824 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:32.824 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:32.824 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:32.824 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:32.824 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:32.825 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:32.825 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:32.825 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:32.825 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:32.825 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:32.826 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:32.827 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148704, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 29, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:32.828 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:32.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2023-08-11T14:22:32.830 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2023-08-11T14:22:32.830 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:32.830 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:32.830 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:32.831 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2023-08-11T14:22:32.831 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2023-08-11T14:22:32.831 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2023-08-11T14:22:32.831 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:32.832 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2023-08-11T14:22:32.832 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:32.832 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2023-08-11T14:22:32.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-08-11T14:22:32.833 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:32.833 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2023-08-11T14:22:32.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2023-08-11T14:22:32.834 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2023-08-11T14:22:32.834 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 348, 2023-08-11T14:22:32.834 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 456368, 2023-08-11T14:22:32.834 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:32.835 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 348, 2023-08-11T14:22:32.835 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2023-08-11T14:22:32.835 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2023-08-11T14:22:32.835 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 348, 2023-08-11T14:22:32.836 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 456368, 2023-08-11T14:22:32.836 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2023-08-11T14:22:32.836 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2023-08-11T14:22:32.836 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2023-08-11T14:22:32.837 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2023-08-11T14:22:32.837 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:32.837 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-08-11T14:22:32.837 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-08-11T14:22:33.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:32 smithi187 ceph-mon[130973]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2023-08-11T14:22:33.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:32 smithi103 ceph-mon[130434]: pgmap v905: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.1 KiB/s wr, 15 op/s 2023-08-11T14:22:33.316 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-08-11T14:22:33.317 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-08-11T14:22:33.317 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-08-11T14:22:33.317 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-08-11T14:22:33.317 DEBUG:tasks.check_counter:Getting stats from b 2023-08-11T14:22:33.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.b perf dump 2023-08-11T14:22:33.967 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:22:34.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:34 smithi103 ceph-mon[130434]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-08-11T14:22:35.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:34 smithi003 ceph-mon[97195]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-08-11T14:22:35.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:34 smithi187 ceph-mon[130973]: pgmap v906: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-08-11T14:22:35.666 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2023-08-11T14:22:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5959, 2023-08-11T14:22:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 7627, 2023-08-11T14:22:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2665297, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 7805309, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 12, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.470541476, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.177868041, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.176751801, 2023-08-11T14:22:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.100594915, 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 7624, 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3.148133559, 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000412924 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5904, 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001282674, 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000217 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 757568, 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 215520 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 20718, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 27201, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 6623862, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 20852392, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 16, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2023-08-11T14:22:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 1.341413963, 2023-08-11T14:22:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.514038833, 2023-08-11T14:22:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.867099329, 2023-08-11T14:22:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.169572436, 2023-08-11T14:22:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 27201, 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3.677217650, 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000135186 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 35059, 2023-08-11T14:22:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.006579488, 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000187 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 45360, 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 649120 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 14033, 2023-08-11T14:22:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 15179, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 4469253, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 20248822, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 179, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 9, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.937563790, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.293675589, 2023-08-11T14:22:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.516734425, 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.176056055, 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 15178, 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3.510246403, 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000231271 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 18528, 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003332328, 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000179 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 9451, 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 7.030876969, 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000743929 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 738, 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.031261738, 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000042360 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 9488, 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 9188, 2023-08-11T14:22:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 9188, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 61.789123164, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.006724980 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 300, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 4, 2023-08-11T14:22:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 766, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 3, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 134, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 11, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 113, 2023-08-11T14:22:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 10, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 117, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 1158, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 98, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 98, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 9, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 36109, 2023-08-11T14:22:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 29228, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 293, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 37, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 4275, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 9475, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 4, 2023-08-11T14:22:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 49, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 1079, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 20, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 83, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 937, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 1045, 2023-08-11T14:22:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 9043647727, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 937, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 37, 2023-08-11T14:22:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 2, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 35, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 900, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 10, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 2092, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 4012, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 4643, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 2297, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 116, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 3061 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 179, 2023-08-11T14:22:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 407, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 205, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 429, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 754, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 869, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 155, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 2852, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2023-08-11T14:22:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 2, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 49, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 3, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 7597, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 7556, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 7556, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 41, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 10, 2023-08-11T14:22:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 83, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 82, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 82, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 1, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 1, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 1, 2023-08-11T14:22:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 26205740, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 26343041, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4693, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 160.970368926, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.034300099 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 129, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 1333, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 1204, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 58, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 286, 2023-08-11T14:22:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 228, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 134, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 3114, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 2980, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 98, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 1374, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 1276, 2023-08-11T14:22:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 70004, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 207152 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 21235, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 146, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 9488, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 1646, 2023-08-11T14:22:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 249, 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1187, 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 7.782681944, 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.006556598 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 70, 2023-08-11T14:22:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.605135264, 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.008644789 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 714, 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.040724783, 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002858157 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2706, 2023-08-11T14:22:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 8.519817037, 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003148491 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000190029, 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000190029 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2023-08-11T14:22:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 34, 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.600206439, 2023-08-11T14:22:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.017653130 2023-08-11T14:22:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2023-08-11T14:22:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2023-08-11T14:22:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 654, 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.393640650, 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000601897 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 80, 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.233749545, 2023-08-11T14:22:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002921869 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2973, 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 40.309631804, 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.013558571 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001560781, 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000780390 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2023-08-11T14:22:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 7, 2023-08-11T14:22:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.022422284, 2023-08-11T14:22:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003203183 2023-08-11T14:22:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2023-08-11T14:22:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 760, 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.279362604, 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001683371 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 2, 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2023-08-11T14:22:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 80, 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 80, 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1124 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 360, 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 360, 2023-08-11T14:22:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 311718, 2023-08-11T14:22:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 236, 2023-08-11T14:22:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 704, 2023-08-11T14:22:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 8, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 518016, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 5886, 2023-08-11T14:22:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2023-08-11T14:22:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 11031, 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 26141986, 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 32, 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 11031, 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2023-08-11T14:22:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 11031, 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 596.633798244, 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.054087009 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 11031, 2023-08-11T14:22:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 17516 2023-08-11T14:22:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 11031, 2023-08-11T14:22:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 1037, 2023-08-11T14:22:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 9994, 2023-08-11T14:22:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2023-08-11T14:22:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2023-08-11T14:22:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 696, 2023-08-11T14:22:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 2945, 2023-08-11T14:22:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 83, 2023-08-11T14:22:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 4774, 2023-08-11T14:22:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 347, 2023-08-11T14:22:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2023-08-11T14:22:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2023-08-11T14:22:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2023-08-11T14:22:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 3, 2023-08-11T14:22:35.740 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 909, 2023-08-11T14:22:35.740 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2023-08-11T14:22:35.740 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:35.740 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 943, 2023-08-11T14:22:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 4765, 2023-08-11T14:22:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2023-08-11T14:22:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2023-08-11T14:22:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2023-08-11T14:22:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2023-08-11T14:22:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2023-08-11T14:22:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2023-08-11T14:22:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2023-08-11T14:22:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2023-08-11T14:22:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2023-08-11T14:22:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2023-08-11T14:22:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2023-08-11T14:22:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2023-08-11T14:22:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2023-08-11T14:22:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2023-08-11T14:22:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2023-08-11T14:22:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2023-08-11T14:22:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2023-08-11T14:22:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2023-08-11T14:22:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2023-08-11T14:22:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2023-08-11T14:22:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2023-08-11T14:22:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 91, 2023-08-11T14:22:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2023-08-11T14:22:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 9, 2023-08-11T14:22:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 12, 2023-08-11T14:22:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 12, 2023-08-11T14:22:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2023-08-11T14:22:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2023-08-11T14:22:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 1822, 2023-08-11T14:22:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 19, 2023-08-11T14:22:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 210 2023-08-11T14:22:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2023-08-11T14:22:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2023-08-11T14:22:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2023-08-11T14:22:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2023-08-11T14:22:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2023-08-11T14:22:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2023-08-11T14:22:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2023-08-11T14:22:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 10, 2023-08-11T14:22:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2023-08-11T14:22:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 5, 2023-08-11T14:22:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 155, 2023-08-11T14:22:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 40706, 2023-08-11T14:22:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 10065005, 2023-08-11T14:22:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 40706, 2023-08-11T14:22:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 40706, 2023-08-11T14:22:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 10065005, 2023-08-11T14:22:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 11031, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 26158416, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 11031, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 26158416, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 11031, 2023-08-11T14:22:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 11031, 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 11031, 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 11031, 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 155, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 16895, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 155, 2023-08-11T14:22:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 81, 2023-08-11T14:22:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 16895, 2023-08-11T14:22:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55b2897420a0": { 2023-08-11T14:22:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 7597, 2023-08-11T14:22:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 22148737, 2023-08-11T14:22:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 7597, 2023-08-11T14:22:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4693, 2023-08-11T14:22:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 22148737, 2023-08-11T14:22:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:35.775 INFO:teuthology.orchestra.run.smithi103.stdout:} 2023-08-11T14:22:36.129 INFO:tasks.check_counter:Daemon mds.b mds.exported=49 2023-08-11T14:22:36.129 INFO:tasks.check_counter:Daemon mds.b mds.imported=20 2023-08-11T14:22:36.129 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=179 2023-08-11T14:22:36.130 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=407 2023-08-11T14:22:36.130 DEBUG:tasks.check_counter:Getting stats from e 2023-08-11T14:22:36.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.e perf dump 2023-08-11T14:22:36.717 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:22:36.965 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:36 smithi103 ceph-mon[130434]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-11T14:22:37.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:36 smithi003 ceph-mon[97195]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-11T14:22:37.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:36 smithi187 ceph-mon[130973]: pgmap v907: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-08-11T14:22:38.205 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4896, 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 3493, 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1852614, 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 3745731, 2023-08-11T14:22:38.206 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 11, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.326010010, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.137348293, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.208577176, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.026326124, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3491, 2023-08-11T14:22:38.207 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.811456077, 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000232442 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6986, 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001179877, 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000168 2023-08-11T14:22:38.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2181, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2462, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1704378, 2023-08-11T14:22:38.209 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1519517, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 18, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.192685292, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.066940473, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.143648312, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.010026068, 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:38.210 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2462, 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.511910927, 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000207924 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3104, 2023-08-11T14:22:38.211 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000699711, 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000225 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 735920, 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 212880 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:38.212 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 305, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 666, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 253750, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 979644, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 12, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 6, 2023-08-11T14:22:38.213 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.043092339, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.021321223, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.021397317, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.001323184, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 664, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.213331540, 2023-08-11T14:22:38.214 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000321282 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 583, 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000115419, 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000197 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T14:22:38.215 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 603808 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:38.216 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 909, 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.266836280, 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000293549 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:38.217 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000010863, 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000005431 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 238, 2023-08-11T14:22:38.218 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 226, 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 226, 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.153783920, 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000680459 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2023-08-11T14:22:38.219 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 12, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2023-08-11T14:22:38.220 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 499, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 7, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 492, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 488, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 469, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 469, 2023-08-11T14:22:38.221 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 14, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 666, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 589, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 6, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 19, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2023-08-11T14:22:38.222 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 2324, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 10, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 478, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:38.223 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 1046, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 9043388938, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:38.224 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 13, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 77, 2023-08-11T14:22:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 638, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 606, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 8, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 21 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.226 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 45, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 445, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 232, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 412, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 197, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:38.227 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:38.228 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:38.229 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 1161, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1161, 2023-08-11T14:22:38.230 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 3, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 3, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2023-08-11T14:22:38.231 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 6503838, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 431, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 121.446452898, 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.281778312 2023-08-11T14:22:38.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 500, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 500, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 52, 2023-08-11T14:22:38.233 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 239, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 187, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 499, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 510, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 11, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 469, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 776, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 307, 2023-08-11T14:22:38.234 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 58316, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 198960 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 243, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 1, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 238, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 15, 2023-08-11T14:22:38.235 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 3, 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004382523, 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004382523 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2023-08-11T14:22:38.236 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001970004, 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000985002 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 191, 2023-08-11T14:22:38.238 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.124045142, 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000649451 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.239 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.240 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:38.241 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.242 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.244 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 17, 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.010489516, 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000617030 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.245 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 15, 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012896735, 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000859782 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.246 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.247 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.248 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:38.249 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.251 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.252 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.253 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.254 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.255 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 40, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 40, 2023-08-11T14:22:38.256 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 3535 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:38.257 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:38.258 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:38.259 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:38.260 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:38.261 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:38.261 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:38.261 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:38.261 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:38.261 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 309374, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 603, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:38.262 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:38.263 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 1748539, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 17712, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:38.264 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 551, 2023-08-11T14:22:38.265 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 2351482, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 27, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 551, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 551, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 153.011168834, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.277697221 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.266 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 551, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 593 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 551, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 0, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 551, 2023-08-11T14:22:38.267 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 0, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 431, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 70, 2023-08-11T14:22:38.268 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 6, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2023-08-11T14:22:38.269 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2023-08-11T14:22:38.270 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2023-08-11T14:22:38.271 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2023-08-11T14:22:38.272 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2023-08-11T14:22:38.273 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2023-08-11T14:22:38.273 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2023-08-11T14:22:38.273 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2023-08-11T14:22:38.273 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2023-08-11T14:22:38.273 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 91, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 4, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 9, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 14, 2023-08-11T14:22:38.274 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 5, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 46, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 10 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2023-08-11T14:22:38.275 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2023-08-11T14:22:38.276 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:38.277 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 7378, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 3122758, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 7378, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:38.278 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 7378, 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3122758, 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.280 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:38.280 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:38.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:38.280 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:38.280 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 551, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2351482, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 551, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2351482, 2023-08-11T14:22:38.281 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:38.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 551, 2023-08-11T14:22:38.283 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 551, 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 551, 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 551, 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.284 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:38.285 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:38.286 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x5598be5460a0": { 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:38.287 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1161, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2309534, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1161, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:38.288 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 431, 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2309534, 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:38.289 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.290 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:38.290 INFO:teuthology.orchestra.run.smithi103.stdout:} 2023-08-11T14:22:38.671 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-08-11T14:22:38.671 INFO:tasks.check_counter:Daemon mds.e mds.imported=10 2023-08-11T14:22:38.671 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=45 2023-08-11T14:22:38.672 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=445 2023-08-11T14:22:38.672 DEBUG:tasks.check_counter:Getting stats from h 2023-08-11T14:22:38.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.h perf dump 2023-08-11T14:22:38.930 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:38 smithi103 ceph-mon[130434]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-08-11T14:22:39.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:38 smithi003 ceph-mon[97195]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-08-11T14:22:39.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:38 smithi187 ceph-mon[130973]: pgmap v908: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-08-11T14:22:39.232 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:22:40.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:40 smithi003 ceph-mon[97195]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-08-11T14:22:40.542 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2023-08-11T14:22:40.542 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:40.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4741, 2023-08-11T14:22:40.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 3042, 2023-08-11T14:22:40.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1579342, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 4091131, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 14, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.319401643, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.132983341, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.193433490, 2023-08-11T14:22:40.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.032001413, 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3040, 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.629609328, 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000207108 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:40.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6063, 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001087149, 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000179 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.545 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1515, 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1267, 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1505291, 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 903051, 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2023-08-11T14:22:40.546 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2023-08-11T14:22:40.547 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.120206252, 2023-08-11T14:22:40.547 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.046043306, 2023-08-11T14:22:40.547 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.077306901, 2023-08-11T14:22:40.547 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.010002972, 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1269, 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.339601014, 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000267613 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:40.548 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1821, 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000333581, 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000183 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 746592, 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 213472 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:40.549 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2906, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 3576, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1109342, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 2059085, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 20, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 6, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.250023691, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.079654243, 2023-08-11T14:22:40.550 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.239686395, 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000301398, 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3572, 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.794495397, 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000222423 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.551 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4768, 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001055018, 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000221 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1232, 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 605888 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.552 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1249, 2023-08-11T14:22:40.553 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.370889628, 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000296949 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:40.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 23, 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000520200, 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000022617 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 717, 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 276, 2023-08-11T14:22:40.555 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 276, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.222071158, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000804605 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 441, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:40.556 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 119, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 10, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 98, 2023-08-11T14:22:40.557 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 11, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 103, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 170, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 84, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 84, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 8, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 1202, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 670, 2023-08-11T14:22:40.558 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 441, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 29, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 446, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 4, 2023-08-11T14:22:40.559 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 165, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 6, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 235, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 1046, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 9043388938, 2023-08-11T14:22:40.560 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:40.561 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 9, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 241, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:40.562 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 1665, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 1586, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 11, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 17 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2023-08-11T14:22:40.563 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 100, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 471, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 248, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 420, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 108, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:40.564 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 2, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 2, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:40.565 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 4, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:40.566 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 2370, 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2023-08-11T14:22:40.567 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 2370, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 4, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 4, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2023-08-11T14:22:40.568 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 7149915, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 573, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 20.716041096, 2023-08-11T14:22:40.569 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.036153649 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 121, 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 286, 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 165, 2023-08-11T14:22:40.570 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 45, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 244, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 199, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 119, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 292, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 173, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 84, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 350, 2023-08-11T14:22:40.571 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 266, 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 54408, 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 198960 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 763, 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:40.572 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 717, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 22, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 17, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 11, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.038226423, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003475129 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.573 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001840909, 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001840909 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:40.574 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.576 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2023-08-11T14:22:40.576 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 193, 2023-08-11T14:22:40.576 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.131969892, 2023-08-11T14:22:40.576 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000683781 2023-08-11T14:22:40.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.577 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.578 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.580 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2023-08-11T14:22:40.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 67, 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.036522655, 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000545114 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002043029, 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001021514 2023-08-11T14:22:40.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:40.584 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.586 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.588 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.589 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.590 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2023-08-11T14:22:40.591 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011468250, 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005734125 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:40.592 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2023-08-11T14:22:40.593 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 21, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 21, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1112 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2023-08-11T14:22:40.594 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:40.595 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:40.596 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:40.597 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2023-08-11T14:22:40.598 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 287882, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 220, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:40.599 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:40.600 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 455693, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 4293, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:40.601 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 724, 2023-08-11T14:22:40.602 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 2997068, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 27, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 724, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 724, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 33.132386084, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.045762964 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.603 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 724, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 780 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 724, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 722, 2023-08-11T14:22:40.604 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 2, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 575, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 77, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2023-08-11T14:22:40.605 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 16, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2023-08-11T14:22:40.606 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2023-08-11T14:22:40.607 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:40.607 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:40.607 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:40.607 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2023-08-11T14:22:40.607 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2023-08-11T14:22:40.608 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2023-08-11T14:22:40.609 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2023-08-11T14:22:40.610 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 91, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 4, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 10, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 13, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 3, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2023-08-11T14:22:40.611 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 64, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 16 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2023-08-11T14:22:40.612 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 2, 2023-08-11T14:22:40.613 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 1, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 2, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:40.614 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 9158, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 3345512, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 9158, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:40.615 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 9158, 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3345512, 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.616 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:40.617 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 724, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2997286, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 724, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2997286, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.618 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:40.619 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 724, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 724, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 724, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 724, 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:40.620 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:40.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 218, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:40.622 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2, 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 218, 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:40.623 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x562c41a860a0": { 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2370, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2955611, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2370, 2023-08-11T14:22:40.624 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 573, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2955611, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:40.625 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:40.626 INFO:teuthology.orchestra.run.smithi103.stdout:} 2023-08-11T14:22:40.626 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:40 smithi187 ceph-mon[130973]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-08-11T14:22:40.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:40 smithi103 ceph-mon[130434]: pgmap v909: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-08-11T14:22:40.977 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2023-08-11T14:22:40.977 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2023-08-11T14:22:40.978 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=100 2023-08-11T14:22:40.978 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=471 2023-08-11T14:22:40.978 DEBUG:tasks.check_counter:Getting stats from k 2023-08-11T14:22:40.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.k perf dump 2023-08-11T14:22:41.537 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.b/config 2023-08-11T14:22:42.964 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:42 smithi103 ceph-mon[130434]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-08-11T14:22:43.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:42 smithi003 ceph-mon[97195]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-08-11T14:22:43.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:42 smithi187 ceph-mon[130973]: pgmap v910: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2023-08-11T14:22:43.178 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2023-08-11T14:22:43.178 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:43.178 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3677, 2023-08-11T14:22:43.178 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 3679, 2023-08-11T14:22:43.178 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 6773490, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1143554, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.277244495, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.073439886, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.075174604, 2023-08-11T14:22:43.179 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.109092652, 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3677, 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.812031927, 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000220840 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.180 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2023-08-11T14:22:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 267, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 876, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 606588, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.013706610, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.009798939, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001550090, 2023-08-11T14:22:43.182 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000069758, 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 265, 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.081787170, 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000308630 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:43.183 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 605904 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:43.184 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 395, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 359, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 725964, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 211223, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.051590033, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.018818758, 2023-08-11T14:22:43.185 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.021919356, 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.009637963, 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 359, 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.108279796, 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000301615 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:43.186 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 725696, 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 210912 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2023-08-11T14:22:43.187 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6151, 2023-08-11T14:22:43.188 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.143760905, 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000023371 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.189 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2023-08-11T14:22:43.190 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:43.191 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 682, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 18, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 489, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 175, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 656, 2023-08-11T14:22:43.192 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 3, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2023-08-11T14:22:43.193 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:43.194 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:43.194 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2023-08-11T14:22:43.194 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2023-08-11T14:22:43.194 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2023-08-11T14:22:43.194 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2023-08-11T14:22:43.195 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:43.196 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:43.197 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2023-08-11T14:22:43.198 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:43.199 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2023-08-11T14:22:43.200 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2023-08-11T14:22:43.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 3100, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2023-08-11T14:22:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 5, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2023-08-11T14:22:43.203 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 9941349, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 9940211, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.204 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 3100 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 79, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 499, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 420, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 14, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 66, 2023-08-11T14:22:43.205 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 52, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 682, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 968, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 286, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 50640, 2023-08-11T14:22:43.206 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 207152 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2023-08-11T14:22:43.207 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.208 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2023-08-11T14:22:43.209 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.210 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2023-08-11T14:22:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.212 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.212 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.212 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2023-08-11T14:22:43.212 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2023-08-11T14:22:43.214 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.216 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.217 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.218 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.219 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.220 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:43.221 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.223 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.226 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.227 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2023-08-11T14:22:43.228 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 3541 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:43.229 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:43.230 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:43.231 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:43.232 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:43.233 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 184643, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 117, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:43.234 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 167, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2023-08-11T14:22:43.235 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 694645, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 20369, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.236 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3680, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3680, 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2023-08-11T14:22:43.237 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3680, 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 159.119064897, 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.043238876 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3680, 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3681 2023-08-11T14:22:43.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3680, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3680, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2165, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2023-08-11T14:22:43.239 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1513, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2023-08-11T14:22:43.240 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:43.241 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2023-08-11T14:22:43.242 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2023-08-11T14:22:43.243 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2023-08-11T14:22:43.244 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 91, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 4, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 4, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 5, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2023-08-11T14:22:43.245 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2023-08-11T14:22:43.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2023-08-11T14:22:43.247 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:43.248 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4072, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 7110918, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4072, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4072, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 7110918, 2023-08-11T14:22:43.249 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:43.250 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3680, 2023-08-11T14:22:43.251 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 5747050, 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3680, 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 5747050, 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:43.253 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3680, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3680, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3680, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3680, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.254 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:43.255 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:43.256 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x556138378460": { 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2023-08-11T14:22:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2023-08-11T14:22:43.258 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout: } 2023-08-11T14:22:43.259 INFO:teuthology.orchestra.run.smithi103.stdout:} 2023-08-11T14:22:43.595 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-08-11T14:22:43.595 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-08-11T14:22:43.596 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-08-11T14:22:43.596 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-08-11T14:22:43.596 DEBUG:tasks.check_counter:Getting stats from c 2023-08-11T14:22:43.597 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.c perf dump 2023-08-11T14:22:44.240 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:22:45.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:44 smithi003 ceph-mon[97195]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2023-08-11T14:22:45.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:44 smithi187 ceph-mon[130973]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2023-08-11T14:22:45.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:44 smithi103 ceph-mon[130434]: pgmap v911: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2023-08-11T14:22:46.006 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-08-11T14:22:46.006 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:46.006 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2758, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 3039, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1049332, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 3373695, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 17, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 7, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.234321176, 2023-08-11T14:22:46.007 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.083577679, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.161135300, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.021016891, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3033, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.703780491, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000232041 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.008 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4013, 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000994260, 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000247 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:46.009 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 880, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 738, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1071621, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 525618, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 12, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2023-08-11T14:22:46.010 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.081898282, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.028744986, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.046770154, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.011340665, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 737, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.202437718, 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000274678 2023-08-11T14:22:46.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 673, 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000140800, 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000209 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 739408, 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 213904 2023-08-11T14:22:46.012 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 4338, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 3041, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1749511, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1758278, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 10, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:46.013 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.246152331, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.089957186, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.194007451, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.007767841, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3044, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.804394373, 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000264255 2023-08-11T14:22:46.014 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6666, 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.001399212, 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000209 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-08-11T14:22:46.015 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 607568 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:46.016 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 871, 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.200574679, 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000230280 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:46.017 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 85, 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.003348364, 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000039392 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-08-11T14:22:46.018 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 765, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 302, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 302, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.326942074, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001082589 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 463, 2023-08-11T14:22:46.019 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 11, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 395, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 5, 2023-08-11T14:22:46.020 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 380, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 10, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 387, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 2, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 366, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 366, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 17, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 1282, 2023-08-11T14:22:46.021 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 708, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 463, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 29, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 7, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 1814, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2023-08-11T14:22:46.022 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 1, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 1, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 13, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 368, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 1046, 2023-08-11T14:22:46.023 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 9043388938, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:46.024 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 18, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 117, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:46.025 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 582, 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 512, 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 21, 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 63 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 160, 2023-08-11T14:22:46.026 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 393, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 208, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 347, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 189, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 11, 2023-08-11T14:22:46.027 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 11, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-08-11T14:22:46.028 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 1, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.029 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 1375, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 1375, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-08-11T14:22:46.030 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 3, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 3, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2023-08-11T14:22:46.031 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 6075697, 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 371, 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 29.927345758, 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.080666700 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2023-08-11T14:22:46.032 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 394, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 406, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 12, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 56, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 210, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 154, 2023-08-11T14:22:46.033 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 395, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 419, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 24, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 366, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 703, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 337, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 57064, 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 198960 2023-08-11T14:22:46.034 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 872, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 765, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 24, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 44, 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-08-11T14:22:46.035 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 12, 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.063804335, 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005317027 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.036 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-08-11T14:22:46.037 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 229, 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.164818735, 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000719732 2023-08-11T14:22:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1, 2023-08-11T14:22:46.039 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000135720, 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000135720 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.040 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.041 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.042 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.043 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.044 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 31, 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.021193662, 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000683666 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 18, 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.015560173, 2023-08-11T14:22:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000864454 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 7, 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.032316642, 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.004616663 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.047 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:46.048 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.049 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.050 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.051 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.052 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.053 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-08-11T14:22:46.054 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4, 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.029112807, 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.007278201 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:46.055 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-08-11T14:22:46.056 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 1, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 0, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 50, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 50, 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1125 2023-08-11T14:22:46.057 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:46.058 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:46.059 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:46.060 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:46.061 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:46.062 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 300521, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 502, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 792, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 9, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:46.063 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:46.064 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:46.064 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:46.064 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:46.065 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 167, 2023-08-11T14:22:46.065 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:46.065 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:46.065 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 1401798, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 14516, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-08-11T14:22:46.066 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 585, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 1932608, 2023-08-11T14:22:46.067 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 28, 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 585, 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 585, 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 60.201115882, 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.102907890 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.068 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 585, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 665 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 585, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 11, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 574, 2023-08-11T14:22:46.069 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 10, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 11, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 382, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 83, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-08-11T14:22:46.070 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 34, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 0, 2023-08-11T14:22:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 20, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:46.072 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-08-11T14:22:46.073 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-08-11T14:22:46.074 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2023-08-11T14:22:46.075 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 1, 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 99, 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 26 2023-08-11T14:22:46.076 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-08-11T14:22:46.077 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:46.078 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 2, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 1, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 11, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:46.079 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:46.080 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:46.080 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 7972, 2023-08-11T14:22:46.080 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 3128840, 2023-08-11T14:22:46.080 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 7972, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 7972, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 3128840, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.081 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:46.082 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 585, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 1933807, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 585, 2023-08-11T14:22:46.083 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1933807, 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.084 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:46.085 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 585, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 585, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 585, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 585, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.086 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:46.087 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 11, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 1199, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 11, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 11, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1199, 2023-08-11T14:22:46.088 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x555c33f4a0a0": { 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:46.089 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1375, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 1881393, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1375, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:46.090 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 371, 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1881393, 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:46.091 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-08-11T14:22:46.385 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2023-08-11T14:22:46.385 INFO:tasks.check_counter:Daemon mds.c mds.imported=13 2023-08-11T14:22:46.385 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=160 2023-08-11T14:22:46.386 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=393 2023-08-11T14:22:46.386 DEBUG:tasks.check_counter:Getting stats from f 2023-08-11T14:22:46.386 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.f perf dump 2023-08-11T14:22:46.938 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:22:47.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:46 smithi003 ceph-mon[97195]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-08-11T14:22:47.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:46 smithi187 ceph-mon[130973]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-08-11T14:22:47.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:46 smithi103 ceph-mon[130434]: pgmap v912: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-08-11T14:22:48.165 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-08-11T14:22:48.165 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:48.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2188, 2023-08-11T14:22:48.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2190, 2023-08-11T14:22:48.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 9043325, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 680478, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.206775631, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.051265311, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.059859833, 2023-08-11T14:22:48.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.078037312, 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2188, 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.722689689, 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000330296 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.167 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:48.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1275, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1241, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 9334718, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 485301, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 4, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 3, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.129742008, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.038864787, 2023-08-11T14:22:48.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.045052707, 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.039081584, 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1240, 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.250074058, 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000201672 2023-08-11T14:22:48.170 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 722592, 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 210000 2023-08-11T14:22:48.171 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1008, 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1268, 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 5650434, 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 835694, 2023-08-11T14:22:48.172 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.112008115, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.032430127, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.029617059, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.040547216, 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:48.173 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1267, 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.400754068, 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000316301 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.174 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 523760 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.175 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:48.176 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6936, 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.219410307, 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000031633 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:48.177 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.178 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.179 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-08-11T14:22:48.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 616, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 399, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 196, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 21, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 45, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 161, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2023-08-11T14:22:48.181 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 7, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:48.182 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2023-08-11T14:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 1045, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 9043649591, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:48.184 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:48.185 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2023-08-11T14:22:48.186 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2023-08-11T14:22:48.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:48.188 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:48.189 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-08-11T14:22:48.190 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 7603, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2023-08-11T14:22:48.191 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 2, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 78, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 26205740, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 26367465, 2023-08-11T14:22:48.192 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 26361002, 2023-08-11T14:22:48.193 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-08-11T14:22:48.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.193 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 7603 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 116, 2023-08-11T14:22:48.194 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 1306, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 1190, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 39, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 74, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 35, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 616, 2023-08-11T14:22:48.195 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 2496, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 1880, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 52360, 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 207152 2023-08-11T14:22:48.196 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2023-08-11T14:22:48.197 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.198 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.200 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-08-11T14:22:48.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.203 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.204 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.205 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.206 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.207 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-08-11T14:22:48.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.209 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.210 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.212 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.212 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-08-11T14:22:48.212 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.212 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.213 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-08-11T14:22:48.214 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.215 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.216 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.218 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:48.219 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-08-11T14:22:48.220 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 2, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 3586 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.221 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:48.222 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:48.223 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:48.224 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:48.225 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:48.225 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:48.225 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:48.225 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:48.225 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:48.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:48.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:48.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:48.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:48.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 202810, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 153, 2023-08-11T14:22:48.227 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:48.228 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:48.228 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-08-11T14:22:48.228 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-08-11T14:22:48.228 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:48.228 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 167, 2023-08-11T14:22:48.229 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-08-11T14:22:48.230 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 746227, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 5146, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.231 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 4076, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 26, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 4, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 4076, 2023-08-11T14:22:48.232 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4076, 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 67.904977389, 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.016659709 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4076, 2023-08-11T14:22:48.233 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4077 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 4076, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 4076, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-08-11T14:22:48.234 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 2360, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1713, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-08-11T14:22:48.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 2, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2023-08-11T14:22:48.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-08-11T14:22:48.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-08-11T14:22:48.238 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-08-11T14:22:48.239 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 6, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 6, 2023-08-11T14:22:48.240 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-08-11T14:22:48.241 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-08-11T14:22:48.242 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:48.243 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 4467, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 23603564, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:48.244 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 4467, 2023-08-11T14:22:48.245 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:48.245 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:48.249 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4467, 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 23603564, 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.250 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:48.251 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4076, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 22173171, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4076, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 22173171, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:48.253 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 4076, 2023-08-11T14:22:48.254 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 4076, 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4076, 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 4076, 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.255 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.256 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.256 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:48.256 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:48.256 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:48.256 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:48.257 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.258 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x563008982460": { 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:48.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-08-11T14:22:48.260 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.261 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:48.262 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-08-11T14:22:48.584 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-08-11T14:22:48.584 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-08-11T14:22:48.585 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-08-11T14:22:48.585 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-08-11T14:22:48.585 DEBUG:tasks.check_counter:Getting stats from i 2023-08-11T14:22:48.585 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.i perf dump 2023-08-11T14:22:49.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:48 smithi003 ceph-mon[97195]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-08-11T14:22:49.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:48 smithi187 ceph-mon[130973]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-08-11T14:22:49.142 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:22:49.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:48 smithi103 ceph-mon[130434]: pgmap v913: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-08-11T14:22:50.363 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-08-11T14:22:50.363 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:50.364 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 8300, 2023-08-11T14:22:50.364 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 9466, 2023-08-11T14:22:50.364 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 3204829, 2023-08-11T14:22:50.364 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 10715566, 2023-08-11T14:22:50.364 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 16, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 7, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.587822903, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.216831382, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.374557889, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.068160435, 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:50.365 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 9462, 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.588988164, 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000167933 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 10986, 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.002265863, 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000206 2023-08-11T14:22:50.366 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 751360, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 265504 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 6449, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 7113, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1874427, 2023-08-11T14:22:50.367 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 2744038, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 12, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 8, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.386436361, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.149653398, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.270209254, 2023-08-11T14:22:50.368 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.014012657, 2023-08-11T14:22:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 7111, 2023-08-11T14:22:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.246012029, 2023-08-11T14:22:50.369 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000175223 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 12202, 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.002209993, 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000181 2023-08-11T14:22:50.370 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 524752 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1881, 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2097, 2023-08-11T14:22:50.371 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 974860, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1232802, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 10, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.135392344, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.049015383, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.077203449, 2023-08-11T14:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.019911092, 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2099, 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.414806938, 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000197621 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2579, 2023-08-11T14:22:50.373 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000437401, 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000169 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-08-11T14:22:50.374 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3258, 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.970394132, 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000604786 2023-08-11T14:22:50.375 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 842, 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.035892002, 2023-08-11T14:22:50.376 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000042627 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 3795, 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 3455, 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-08-11T14:22:50.377 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3455, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 25.571293615, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.007401242 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 340, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:50.378 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 11, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 545, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 16, 2023-08-11T14:22:50.379 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 513, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 16, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 520, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 284, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 73, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 73, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 14, 2023-08-11T14:22:50.380 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 16232, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 11848, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 339, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 47, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:50.381 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 3135, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 2489, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 2, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 20, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 262, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 25, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 343, 2023-08-11T14:22:50.382 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 120, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 1046, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 9043388938, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 120, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:50.383 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 5, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 5, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 115, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 19, 2023-08-11T14:22:50.384 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 339, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 1033, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 1412, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 783, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 137, 2023-08-11T14:22:50.385 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 1757 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 389, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 126, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 76, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 147, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 507, 2023-08-11T14:22:50.386 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 426, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 151, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 1155, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-08-11T14:22:50.387 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 20, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-08-11T14:22:50.388 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 3100, 2023-08-11T14:22:50.389 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 3100, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 5, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-08-11T14:22:50.390 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 5, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2023-08-11T14:22:50.391 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 9941349, 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1540, 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 41.726348374, 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.027095031 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.392 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 120, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 526, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 406, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 58, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 113, 2023-08-11T14:22:50.393 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 55, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 545, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 1005, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 460, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 73, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 562, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 489, 2023-08-11T14:22:50.394 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 58032, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 198960 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 10222, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 52, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 3795, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 541, 2023-08-11T14:22:50.395 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 202, 2023-08-11T14:22:50.396 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-08-11T14:22:50.396 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 154, 2023-08-11T14:22:50.396 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.334940763, 2023-08-11T14:22:50.396 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002174940 2023-08-11T14:22:50.396 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 13, 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.119514233, 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.009193402 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:50.397 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 275, 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.827589974, 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003009418 2023-08-11T14:22:50.398 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1275, 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 3.509755780, 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002752749 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:50.399 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.400 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:50.402 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.402 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.402 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.402 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.403 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:50.403 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.403 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.403 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.404 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.404 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-08-11T14:22:50.404 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.404 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.405 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.405 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.405 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-08-11T14:22:50.405 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1, 2023-08-11T14:22:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000639606, 2023-08-11T14:22:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000639606 2023-08-11T14:22:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-08-11T14:22:50.406 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.407 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.407 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.407 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.407 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-08-11T14:22:50.408 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.408 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.408 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.409 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.409 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-08-11T14:22:50.409 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 133, 2023-08-11T14:22:50.409 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.099655085, 2023-08-11T14:22:50.409 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000749286 2023-08-11T14:22:50.410 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.410 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-08-11T14:22:50.410 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 21, 2023-08-11T14:22:50.410 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.062338743, 2023-08-11T14:22:50.410 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002968511 2023-08-11T14:22:50.411 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.411 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-08-11T14:22:50.411 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1302, 2023-08-11T14:22:50.411 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 19.724941644, 2023-08-11T14:22:50.412 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.015149724 2023-08-11T14:22:50.412 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.412 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:50.412 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.413 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.413 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.413 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.413 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:50.413 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.414 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.414 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.414 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.414 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-08-11T14:22:50.415 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.415 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.415 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.415 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.415 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:50.416 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.416 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.416 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.416 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.416 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.417 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:50.418 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.419 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-08-11T14:22:50.420 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 281, 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.891917787, 2023-08-11T14:22:50.421 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003174084 2023-08-11T14:22:50.422 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.422 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:50.422 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:50.422 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:50.423 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.423 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.423 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.423 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.424 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.424 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-08-11T14:22:50.424 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-08-11T14:22:50.424 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 1, 2023-08-11T14:22:50.425 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 0, 2023-08-11T14:22:50.425 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-08-11T14:22:50.425 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-08-11T14:22:50.425 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 380, 2023-08-11T14:22:50.426 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 380, 2023-08-11T14:22:50.426 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1124 2023-08-11T14:22:50.426 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.427 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-08-11T14:22:50.427 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:50.427 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:50.427 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:50.428 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:50.428 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:50.428 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:50.429 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:50.429 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:50.429 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:50.430 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:50.430 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:50.430 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:50.430 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:50.430 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:50.431 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:50.431 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:50.431 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:50.431 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:50.431 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:50.432 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:50.432 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:50.432 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:50.433 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:50.433 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:50.433 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:50.433 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-08-11T14:22:50.434 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 283943, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 202, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 176, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 2, 2023-08-11T14:22:50.435 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:50.436 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 167, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 812591, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 24808, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:50.437 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 2670, 2023-08-11T14:22:50.438 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 5851766, 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 28, 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 2670, 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2670, 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 102.429457778, 2023-08-11T14:22:50.439 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.038363092 2023-08-11T14:22:50.440 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.440 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-08-11T14:22:50.440 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-08-11T14:22:50.440 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2670, 2023-08-11T14:22:50.441 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2886 2023-08-11T14:22:50.441 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.441 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 2670, 2023-08-11T14:22:50.441 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 199, 2023-08-11T14:22:50.442 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 2471, 2023-08-11T14:22:50.442 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-08-11T14:22:50.442 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-08-11T14:22:50.442 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2023-08-11T14:22:50.443 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 10, 2023-08-11T14:22:50.443 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 79, 2023-08-11T14:22:50.443 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 1619, 2023-08-11T14:22:50.444 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 133, 2023-08-11T14:22:50.444 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-08-11T14:22:50.444 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-08-11T14:22:50.444 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-08-11T14:22:50.445 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-08-11T14:22:50.445 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 553, 2023-08-11T14:22:50.445 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-08-11T14:22:50.445 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 120, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 20, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-08-11T14:22:50.446 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-08-11T14:22:50.447 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-08-11T14:22:50.448 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-08-11T14:22:50.449 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 244, 2023-08-11T14:22:50.450 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2023-08-11T14:22:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 108 2023-08-11T14:22:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-08-11T14:22:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2023-08-11T14:22:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 240, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 5, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 151, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.453 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 16626, 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 4533433, 2023-08-11T14:22:50.454 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 16626, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 16626, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 4533433, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.455 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:50.456 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:50.457 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:50.457 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:50.457 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:50.457 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:50.457 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2670, 2023-08-11T14:22:50.458 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 5868225, 2023-08-11T14:22:50.458 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2670, 2023-08-11T14:22:50.458 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 5868225, 2023-08-11T14:22:50.459 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:50.459 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.459 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.459 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-08-11T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2670, 2023-08-11T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 2670, 2023-08-11T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2670, 2023-08-11T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2670, 2023-08-11T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 151, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 16459, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 151, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 79, 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 16459, 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55efd0ea00a0": { 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 3100, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 5747045, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 3100, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1540, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 5747045, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-08-11T14:22:50.470 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:50 smithi187 ceph-mon[130973]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:50.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:50 smithi003 ceph-mon[97195]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:50.678 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:50 smithi103 ceph-mon[130434]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:50.687 INFO:tasks.check_counter:Daemon mds.i mds.exported=20 2023-08-11T14:22:50.687 INFO:tasks.check_counter:Daemon mds.i mds.imported=25 2023-08-11T14:22:50.688 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=389 2023-08-11T14:22:50.688 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=126 2023-08-11T14:22:50.688 DEBUG:tasks.check_counter:Getting stats from l 2023-08-11T14:22:50.689 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:92684807f016ce045ac66e37fdde6f8c47857bcc shell --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c -- ceph daemon mds.l perf dump 2023-08-11T14:22:51.244 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/mon.c/config 2023-08-11T14:22:52.456 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-08-11T14:22:52.457 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-08-11T14:22:52.457 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1524, 2023-08-11T14:22:52.457 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1526, 2023-08-11T14:22:52.457 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 3350098, 2023-08-11T14:22:52.458 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 474028, 2023-08-11T14:22:52.458 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 7, 2023-08-11T14:22:52.458 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:52.458 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.125545685, 2023-08-11T14:22:52.459 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.031480440, 2023-08-11T14:22:52.459 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.036421492, 2023-08-11T14:22:52.460 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.046031321, 2023-08-11T14:22:52.460 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:52.460 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1524, 2023-08-11T14:22:52.461 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.419653441, 2023-08-11T14:22:52.461 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000275363 2023-08-11T14:22:52.461 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.461 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:52.461 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.462 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.463 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-08-11T14:22:52.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1123, 2023-08-11T14:22:52.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1388, 2023-08-11T14:22:52.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 276194, 2023-08-11T14:22:52.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 872787, 2023-08-11T14:22:52.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.120057247, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.032012991, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.027873631, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.049720782, 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:52.465 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1386, 2023-08-11T14:22:52.466 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.424495411, 2023-08-11T14:22:52.466 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000306273 2023-08-11T14:22:52.466 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.466 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:52.466 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2023-08-11T14:22:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 523840 2023-08-11T14:22:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-08-11T14:22:52.469 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1578, 2023-08-11T14:22:52.469 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1542, 2023-08-11T14:22:52.469 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1123328, 2023-08-11T14:22:52.470 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 580075, 2023-08-11T14:22:52.470 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 3, 2023-08-11T14:22:52.470 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 2, 2023-08-11T14:22:52.470 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.176084292, 2023-08-11T14:22:52.471 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.051661930, 2023-08-11T14:22:52.471 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.053869166, 2023-08-11T14:22:52.471 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.057932405, 2023-08-11T14:22:52.471 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-08-11T14:22:52.472 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1542, 2023-08-11T14:22:52.472 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.536996219, 2023-08-11T14:22:52.472 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000348246 2023-08-11T14:22:52.472 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.472 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-08-11T14:22:52.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.473 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.474 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 726512, 2023-08-11T14:22:52.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 212784 2023-08-11T14:22:52.474 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.474 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-08-11T14:22:52.475 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-08-11T14:22:52.475 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-08-11T14:22:52.475 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.475 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-08-11T14:22:52.476 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:52.476 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:52.476 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6474, 2023-08-11T14:22:52.477 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.178009093, 2023-08-11T14:22:52.477 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000027495 2023-08-11T14:22:52.477 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.477 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.478 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-08-11T14:22:52.478 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-08-11T14:22:52.478 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-08-11T14:22:52.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.479 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.479 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.479 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.479 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.480 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-08-11T14:22:52.480 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2023-08-11T14:22:52.480 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2023-08-11T14:22:52.481 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-08-11T14:22:52.481 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.481 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.481 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.482 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.482 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-08-11T14:22:52.482 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2023-08-11T14:22:52.483 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-08-11T14:22:52.483 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-08-11T14:22:52.483 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2023-08-11T14:22:52.483 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-08-11T14:22:52.484 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-08-11T14:22:52.484 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 95, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 0, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 0, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 95, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 95, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 1, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2023-08-11T14:22:52.485 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 4, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-08-11T14:22:52.486 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2023-08-11T14:22:52.487 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-08-11T14:22:52.488 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-08-11T14:22:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-08-11T14:22:52.490 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.491 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-08-11T14:22:52.492 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2023-08-11T14:22:52.492 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2023-08-11T14:22:52.492 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2023-08-11T14:22:52.493 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2023-08-11T14:22:52.493 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2023-08-11T14:22:52.493 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-08-11T14:22:52.493 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-08-11T14:22:52.494 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-08-11T14:22:52.494 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2023-08-11T14:22:52.494 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2023-08-11T14:22:52.495 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-08-11T14:22:52.495 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-08-11T14:22:52.495 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-08-11T14:22:52.495 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-08-11T14:22:52.496 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-08-11T14:22:52.496 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-08-11T14:22:52.496 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-08-11T14:22:52.497 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-08-11T14:22:52.497 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2023-08-11T14:22:52.497 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-08-11T14:22:52.497 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-08-11T14:22:52.498 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-08-11T14:22:52.498 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-08-11T14:22:52.498 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.499 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-08-11T14:22:52.499 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-08-11T14:22:52.499 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2023-08-11T14:22:52.499 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-08-11T14:22:52.500 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-08-11T14:22:52.500 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 2370, 2023-08-11T14:22:52.500 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-08-11T14:22:52.501 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-08-11T14:22:52.501 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2023-08-11T14:22:52.501 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-08-11T14:22:52.501 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-08-11T14:22:52.502 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 4, 2023-08-11T14:22:52.502 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2023-08-11T14:22:52.502 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-08-11T14:22:52.502 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-08-11T14:22:52.503 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2023-08-11T14:22:52.503 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 7149915, 2023-08-11T14:22:52.503 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 7149341, 2023-08-11T14:22:52.503 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-08-11T14:22:52.504 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.504 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.504 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.505 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.505 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 2370 2023-08-11T14:22:52.505 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.505 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-08-11T14:22:52.506 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 95, 2023-08-11T14:22:52.506 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 258, 2023-08-11T14:22:52.506 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 163, 2023-08-11T14:22:52.506 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 12, 2023-08-11T14:22:52.506 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 16, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 4, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 95, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 258, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 163, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2023-08-11T14:22:52.507 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2023-08-11T14:22:52.508 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2023-08-11T14:22:52.508 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 47480, 2023-08-11T14:22:52.508 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 207152 2023-08-11T14:22:52.508 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.508 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2023-08-11T14:22:52.509 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.510 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.511 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-08-11T14:22:52.512 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.512 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.512 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.513 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.513 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-08-11T14:22:52.513 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.513 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.514 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.514 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.514 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-08-11T14:22:52.515 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.515 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.515 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.516 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.516 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-08-11T14:22:52.516 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.516 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.517 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.517 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.517 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-08-11T14:22:52.518 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.518 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.518 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.518 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.519 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-08-11T14:22:52.519 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.519 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.519 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.520 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.520 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-08-11T14:22:52.520 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.521 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.521 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.521 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.521 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-08-11T14:22:52.522 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.522 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.522 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.522 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.522 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.523 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.524 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.524 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.524 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-08-11T14:22:52.525 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.525 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.525 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.525 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.526 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-08-11T14:22:52.526 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.526 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.527 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.527 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-08-11T14:22:52.527 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.528 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.528 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.528 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.528 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-08-11T14:22:52.529 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.529 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.529 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.530 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.530 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-08-11T14:22:52.530 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.531 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.531 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.531 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.532 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-08-11T14:22:52.532 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.532 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.533 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.533 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.533 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-08-11T14:22:52.533 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.533 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.534 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.534 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.534 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-08-11T14:22:52.534 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.535 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.535 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.535 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.536 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.537 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.538 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-08-11T14:22:52.539 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.540 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.541 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.542 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-08-11T14:22:52.543 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 3588 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-08-11T14:22:52.544 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-08-11T14:22:52.545 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-08-11T14:22:52.546 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-08-11T14:22:52.547 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-08-11T14:22:52.548 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-08-11T14:22:52.549 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-08-11T14:22:52.550 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 193404, 2023-08-11T14:22:52.550 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 140, 2023-08-11T14:22:52.550 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2023-08-11T14:22:52.550 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-08-11T14:22:52.551 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8704, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 167, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-08-11T14:22:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 297228, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 2513, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-08-11T14:22:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 3827, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2023-08-11T14:22:52.554 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 3827, 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3827, 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 75.383887629, 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.019697906 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-08-11T14:22:52.555 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3827, 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 3828 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 3827, 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 3827, 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2023-08-11T14:22:52.556 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 2301, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1524, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2023-08-11T14:22:52.557 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-08-11T14:22:52.558 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-08-11T14:22:52.559 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-08-11T14:22:52.560 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-08-11T14:22:52.561 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-08-11T14:22:52.561 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-08-11T14:22:52.561 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-08-11T14:22:52.561 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-08-11T14:22:52.561 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-08-11T14:22:52.562 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-08-11T14:22:52.562 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-08-11T14:22:52.562 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-08-11T14:22:52.562 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-08-11T14:22:52.563 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-08-11T14:22:52.563 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-08-11T14:22:52.563 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-08-11T14:22:52.563 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-08-11T14:22:52.563 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 5, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 7, 2023-08-11T14:22:52.564 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 2, 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-08-11T14:22:52.565 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-08-11T14:22:52.566 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2023-08-11T14:22:52.567 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 4221, 2023-08-11T14:22:52.568 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 4346685, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 4221, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 4221, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 4346685, 2023-08-11T14:22:52.569 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-08-11T14:22:52.570 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:52.571 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 3827, 2023-08-11T14:22:52.572 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 2955616, 2023-08-11T14:22:52.572 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 3827, 2023-08-11T14:22:52.572 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2955616, 2023-08-11T14:22:52.572 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:52.573 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.573 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.573 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.574 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.574 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.574 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-08-11T14:22:52.574 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:52.575 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-08-11T14:22:52.575 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:52.575 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:52.576 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:52.576 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:52.576 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:52.576 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 3827, 2023-08-11T14:22:52.577 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 3827, 2023-08-11T14:22:52.577 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 3827, 2023-08-11T14:22:52.577 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 3827, 2023-08-11T14:22:52.578 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:52.578 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.578 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.578 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.578 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.579 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.579 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-08-11T14:22:52.579 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:52.579 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:52.580 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:52.580 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:52.580 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:52.580 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:52.581 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:52.581 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:52.581 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:52.581 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-08-11T14:22:52.582 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-08-11T14:22:52.582 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:52.582 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.582 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.582 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.583 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.583 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-08-11T14:22:52.583 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55a99703a460": { 2023-08-11T14:22:52.583 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-08-11T14:22:52.583 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-08-11T14:22:52.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-08-11T14:22:52.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-08-11T14:22:52.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-08-11T14:22:52.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-08-11T14:22:52.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-08-11T14:22:52.585 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-08-11T14:22:52.585 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-08-11T14:22:52.585 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-08-11T14:22:52.586 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-08-11T14:22:52.586 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-08-11T14:22:52.586 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-08-11T14:22:52.587 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-08-11T14:22:52.587 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-08-11T14:22:52.587 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.588 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-08-11T14:22:52.588 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-08-11T14:22:52.847 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:52 smithi187 ceph-mon[130973]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:52.901 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-08-11T14:22:52.901 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-08-11T14:22:52.901 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-08-11T14:22:52.902 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-08-11T14:22:52.902 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-08-11T14:22:52.914 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-11T14:22:52.914 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:22:52.915 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:22:52.983 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:22:52.984 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:22:53.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:22:53.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:52 smithi003 ceph-mon[97195]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:53.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:52 smithi103 ceph-mon[130434]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-08-11T14:22:53.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/2643509375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78100920 msgr2=0x7fbf78100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/2643509375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78100920 0x7fbf78100d40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009920 tx=0x7fbf68009c30 comp rx=0 tx=0).stop 2023-08-11T14:22:53.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/2643509375 shutdown_connections 2023-08-11T14:22:53.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/2643509375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf78108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/2643509375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78100920 0x7fbf78100d40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/2643509375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78101b10 0x7fbf78101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/2643509375 >> 172.21.15.3:0/2643509375 conn(0x7fbf780fbe50 msgr2=0x7fbf780fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/2643509375 shutdown_connections 2023-08-11T14:22:53.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.249+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/2643509375 wait complete. 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.250+0000 7fbf7e7ff700 1 Processor -- start 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 -- start start 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78100920 0x7fbf78198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 0x7fbf78198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf781a03d0 con 0x7fbf78101b10 2023-08-11T14:22:53.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf781a0510 con 0x7fbf78100920 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7e7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf781a06a0 con 0x7fbf781024b0 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7cd9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 0x7fbf78198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7cd9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.3:37952/0 (socket says 172.21.15.3:37952) 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7cd9c700 1 -- 172.21.15.3:0/1478297862 learned_addr learned my addr 172.21.15.3:0/1478297862 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf77fff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78100920 0x7fbf78198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7cd9c700 1 -- 172.21.15.3:0/1478297862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78100920 msgr2=0x7fbf78198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.251+0000 7fbf7cd9c700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78100920 0x7fbf78198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7cd9c700 1 -- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 msgr2=0x7fbf78198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7cd9c700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 0x7fbf78198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7cd9c700 1 -- 172.21.15.3:0/1478297862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68009520 con 0x7fbf781024b0 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf777fe700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 0x7fbf78198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7cd9c700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ef30 tx=0x7fbf7000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70018070 con 0x7fbf781024b0 2023-08-11T14:22:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf781a0920 con 0x7fbf781024b0 2023-08-11T14:22:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7000f040 con 0x7fbf781024b0 2023-08-11T14:22:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000ccf0 con 0x7fbf781024b0 2023-08-11T14:22:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.252+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf781a0e30 con 0x7fbf781024b0 2023-08-11T14:22:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.253+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf78192f20 con 0x7fbf781024b0 2023-08-11T14:22:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.255+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fbf70003a00 con 0x7fbf781024b0 2023-08-11T14:22:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.256+0000 7fbf757fa700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 0x7fbf600a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.257+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fbf700cb510 con 0x7fbf781024b0 2023-08-11T14:22:53.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.257+0000 7fbf77fff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 0x7fbf600a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.257+0000 7fbf77fff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 0x7fbf600a2460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbf680098f0 tx=0x7fbf68019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.258+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fbf70094fb0 con 0x7fbf781024b0 2023-08-11T14:22:53.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.471+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf7810c330 con 0x7fbf781024b0 2023-08-11T14:22:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.473+0000 7fbf757fa700 1 -- 172.21.15.3:0/1478297862 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7fbf700133e0 con 0x7fbf781024b0 2023-08-11T14:22:53.474 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:53.475 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:22:53.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 msgr2=0x7fbf600a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 0x7fbf600a2460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbf680098f0 tx=0x7fbf68019040 comp rx=0 tx=0).stop 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 msgr2=0x7fbf7819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000ef30 tx=0x7fbf7000c3a0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 shutdown_connections 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf781024b0 0x7fbf7819fe00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fbf6009ffa0 0x7fbf600a2460 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf78100920 0x7fbf78198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 --2- 172.21.15.3:0/1478297862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf78101b10 0x7fbf78198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 >> 172.21.15.3:0/1478297862 conn(0x7fbf780fbe50 msgr2=0x7fbf781074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 shutdown_connections 2023-08-11T14:22:53.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.475+0000 7fbf7e7ff700 1 -- 172.21.15.3:0/1478297862 wait complete. 2023-08-11T14:22:53.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-11T13:57:20.695647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-11T14:01:30.942315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-08-11T14:01:32.272156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-08-11T14:01:35.852024+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-08-11T14:22:53.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-11T14:22:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 -- 172.21.15.3:0/1970900961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 msgr2=0x7fb01c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 --2- 172.21.15.3:0/1970900961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c1037e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009990 tx=0x7fb00c009ca0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 -- 172.21.15.3:0/1970900961 shutdown_connections 2023-08-11T14:22:53.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 --2- 172.21.15.3:0/1970900961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 --2- 172.21.15.3:0/1970900961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c1037e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 --2- 172.21.15.3:0/1970900961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 -- 172.21.15.3:0/1970900961 >> 172.21.15.3:0/1970900961 conn(0x7fb01c0fad80 msgr2=0x7fb01c0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:53.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 -- 172.21.15.3:0/1970900961 shutdown_connections 2023-08-11T14:22:53.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.707+0000 7fb0225ff700 1 -- 172.21.15.3:0/1970900961 wait complete. 2023-08-11T14:22:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 Processor -- start 2023-08-11T14:22:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 -- start start 2023-08-11T14:22:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e1b0 con 0x7fb01c1013f0 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e2f0 con 0x7fb01c103d20 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.709+0000 7fb0225ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c19e480 con 0x7fb01c106650 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb01b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb01b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43726/0 (socket says 172.21.15.3:43726) 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb01b7fe700 1 -- 172.21.15.3:0/2827861847 learned_addr learned my addr 172.21.15.3:0/2827861847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:53.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb01bfff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 -- 172.21.15.3:0/2827861847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 msgr2=0x7fb01c196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 -- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 msgr2=0x7fb01c196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 -- 172.21.15.3:0/2827861847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0095f0 con 0x7fb01c106650 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb01bfff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:53.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.710+0000 7fb020b9c700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c19dbe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb01400efc0 tx=0x7fb01400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.711+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014018070 con 0x7fb01c106650 2023-08-11T14:22:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.711+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c19e700 con 0x7fb01c106650 2023-08-11T14:22:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.711+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01400f040 con 0x7fb01c106650 2023-08-11T14:22:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.711+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400cd40 con 0x7fb01c106650 2023-08-11T14:22:53.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.711+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c19ec10 con 0x7fb01c106650 2023-08-11T14:22:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.712+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0fc900 con 0x7fb01c106650 2023-08-11T14:22:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.714+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb014003a00 con 0x7fb01c106650 2023-08-11T14:22:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.715+0000 7fb0197fa700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 0x7fb0040a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.715+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb0140cb0e0 con 0x7fb01c106650 2023-08-11T14:22:53.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.716+0000 7fb01bfff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 0x7fb0040a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:53.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.716+0000 7fb01bfff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 0x7fb0040a2490 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b2e0 tx=0x7fb00c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:53.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.719+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb014094b80 con 0x7fb01c106650 2023-08-11T14:22:53.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:53 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1478297862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:53.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.928+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb01c02cd30 con 0x7fb01c106650 2023-08-11T14:22:53.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.929+0000 7fb0197fa700 1 -- 172.21.15.3:0/2827861847 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb01401b320 con 0x7fb01c106650 2023-08-11T14:22:53.930 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-08-11T14:22:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 msgr2=0x7fb0040a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 0x7fb0040a2490 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00b2e0 tx=0x7fb00c005fb0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 msgr2=0x7fb01c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c19dbe0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb01400efc0 tx=0x7fb01400c3a0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 shutdown_connections 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb01c106650 0x7fb01c19dbe0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb01c1013f0 0x7fb01c196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb00409ffd0 0x7fb0040a2490 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 --2- 172.21.15.3:0/2827861847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01c103d20 0x7fb01c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.931+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 >> 172.21.15.3:0/2827861847 conn(0x7fb01c0fad80 msgr2=0x7fb01c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.932+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 shutdown_connections 2023-08-11T14:22:53.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:53.932+0000 7fb0225ff700 1 -- 172.21.15.3:0/2827861847 wait complete. 2023-08-11T14:22:53.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-11T14:22:54.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:53 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1478297862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:54.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 -- 172.21.15.3:0/2104273162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44100910 msgr2=0x7ffa44100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2104273162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44100910 0x7ffa44100d30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b0d0 tx=0x7ffa3800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 -- 172.21.15.3:0/2104273162 shutdown_connections 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2104273162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa441024a0 0x7ffa44108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2104273162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44100910 0x7ffa44100d30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2104273162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa44101b00 0x7ffa44101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 -- 172.21.15.3:0/2104273162 >> 172.21.15.3:0/2104273162 conn(0x7ffa440fbe60 msgr2=0x7ffa440fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:54.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 -- 172.21.15.3:0/2104273162 shutdown_connections 2023-08-11T14:22:54.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.158+0000 7ffa4afff700 1 -- 172.21.15.3:0/2104273162 wait complete. 2023-08-11T14:22:54.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.159+0000 7ffa4afff700 1 Processor -- start 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.159+0000 7ffa4afff700 1 -- start start 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa44100910 0x7ffa440746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 0x7ffa44073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419a650 con 0x7ffa44101b00 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44073790 con 0x7ffa441024a0 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4afff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44073950 con 0x7ffa44100910 2023-08-11T14:22:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55570/0 (socket says 172.21.15.3:55570) 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 -- 172.21.15.3:0/2202758634 learned_addr learned my addr 172.21.15.3:0/2202758634 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa48d9b700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa44100910 0x7ffa440746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4959c700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 0x7ffa44073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 -- 172.21.15.3:0/2202758634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa44100910 msgr2=0x7ffa440746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa44100910 0x7ffa440746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 -- 172.21.15.3:0/2202758634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 msgr2=0x7ffa44073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 0x7ffa44073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa43fff700 1 -- 172.21.15.3:0/2202758634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa38009d00 con 0x7ffa44101b00 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.160+0000 7ffa4959c700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 0x7ffa44073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa43fff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e9a0 tx=0x7ffa3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30009790 con 0x7ffa44101b00 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa3000cbd0 con 0x7ffa44101b00 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa441a5000 con 0x7ffa44101b00 2023-08-11T14:22:54.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30018680 con 0x7ffa44101b00 2023-08-11T14:22:54.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.161+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa441a5480 con 0x7ffa44101b00 2023-08-11T14:22:54.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.162+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4410c170 con 0x7ffa44101b00 2023-08-11T14:22:54.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.164+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7ffa3000cd40 con 0x7ffa44101b00 2023-08-11T14:22:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.165+0000 7ffa41ffb700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 0x7ffa340a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.165+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7ffa30014070 con 0x7ffa44101b00 2023-08-11T14:22:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.166+0000 7ffa48d9b700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 0x7ffa340a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.166+0000 7ffa48d9b700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 0x7ffa340a2490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009fd0 tx=0x7ffa3800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:54.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.167+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7ffa300954e0 con 0x7ffa44101b00 2023-08-11T14:22:54.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:53 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1478297862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:22:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.381+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa44107840 con 0x7ffa44101b00 2023-08-11T14:22:54.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.382+0000 7ffa41ffb700 1 -- 172.21.15.3:0/2202758634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa30098b00 con 0x7ffa44101b00 2023-08-11T14:22:54.384 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.384+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 msgr2=0x7ffa340a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.384+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 0x7ffa340a2490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009fd0 tx=0x7ffa3800b040 comp rx=0 tx=0).stop 2023-08-11T14:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.384+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 msgr2=0x7ffa44072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.384+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000e9a0 tx=0x7ffa3000ecb0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 shutdown_connections 2023-08-11T14:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7ffa3409ffd0 0x7ffa340a2490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa441024a0 0x7ffa44073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa44100910 0x7ffa440746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 --2- 172.21.15.3:0/2202758634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa44101b00 0x7ffa44072d10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 >> 172.21.15.3:0/2202758634 conn(0x7ffa440fbe60 msgr2=0x7ffa440fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 shutdown_connections 2023-08-11T14:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.385+0000 7ffa4afff700 1 -- 172.21.15.3:0/2202758634 wait complete. 2023-08-11T14:22:54.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-11T14:22:54.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 -- 172.21.15.3:0/460850066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 msgr2=0x7f57b8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/460850066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8101f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f57a00099c0 tx=0x7f57a0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 -- 172.21.15.3:0/460850066 shutdown_connections 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/460850066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b81024a0 0x7f57b8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/460850066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b8100910 0x7f57b8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/460850066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8101f60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 -- 172.21.15.3:0/460850066 >> 172.21.15.3:0/460850066 conn(0x7f57b80fbe60 msgr2=0x7f57b80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 -- 172.21.15.3:0/460850066 shutdown_connections 2023-08-11T14:22:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.612+0000 7f57bc9ff700 1 -- 172.21.15.3:0/460850066 wait complete. 2023-08-11T14:22:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.613+0000 7f57bc9ff700 1 Processor -- start 2023-08-11T14:22:54.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 -- start start 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b81024a0 0x7f57b819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819e1e0 con 0x7f57b8100910 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819e320 con 0x7f57b8101b00 2023-08-11T14:22:54.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57bc9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819e4b0 con 0x7f57b81024a0 2023-08-11T14:22:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.614+0000 7f57b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55580/0 (socket says 172.21.15.3:55580) 2023-08-11T14:22:54.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 -- 172.21.15.3:0/1772760038 learned_addr learned my addr 172.21.15.3:0/1772760038 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57affff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b6d9d700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b81024a0 0x7f57b819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 -- 172.21.15.3:0/1772760038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b81024a0 msgr2=0x7f57b819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b81024a0 0x7f57b819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 -- 172.21.15.3:0/1772760038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 msgr2=0x7f57b8196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 -- 172.21.15.3:0/1772760038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8009520 con 0x7f57b8100910 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57affff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57b659c700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ea50 tx=0x7f57a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a800cbd0 con 0x7f57b8100910 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a800cd30 con 0x7f57b8100910 2023-08-11T14:22:54.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b0009c40 con 0x7f57b8100910 2023-08-11T14:22:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.615+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a80186c0 con 0x7f57b8100910 2023-08-11T14:22:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.616+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00095f0 con 0x7f57b8100910 2023-08-11T14:22:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.617+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b810c170 con 0x7f57b8100910 2023-08-11T14:22:54.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.619+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f57a8018820 con 0x7f57b8100910 2023-08-11T14:22:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.621+0000 7f57af7fe700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 0x7f57a40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:54.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.621+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f57a8014070 con 0x7f57b8100910 2023-08-11T14:22:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.621+0000 7f57affff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 0x7f57a40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:54.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.622+0000 7f57affff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 0x7f57a40a2410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b580 tx=0x7f57a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.625+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f57a8095540 con 0x7f57b8100910 2023-08-11T14:22:54.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.838+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57b8107860 con 0x7f57b8100910 2023-08-11T14:22:54.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.839+0000 7f57af7fe700 1 -- 172.21.15.3:0/1772760038 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57a8098b60 con 0x7f57b8100910 2023-08-11T14:22:54.841 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-08-11T14:22:54.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 msgr2=0x7f57a40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 0x7f57a40a2410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b580 tx=0x7f57a0005fb0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 msgr2=0x7f57b8196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:54.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57a800ea50 tx=0x7f57a800ed60 comp rx=0 tx=0).stop 2023-08-11T14:22:54.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 shutdown_connections 2023-08-11T14:22:54.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b81024a0 0x7f57b819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57b8100910 0x7f57b8196660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f57a409ff50 0x7f57a40a2410 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 --2- 172.21.15.3:0/1772760038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57b8101b00 0x7f57b8196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:54.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 >> 172.21.15.3:0/1772760038 conn(0x7f57b80fbe60 msgr2=0x7f57b80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:54.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.841+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 shutdown_connections 2023-08-11T14:22:54.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:54.842+0000 7f57bc9ff700 1 -- 172.21.15.3:0/1772760038 wait complete. 2023-08-11T14:22:54.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-08-11T14:22:55.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:54 smithi003 ceph-mon[97195]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-11T14:22:55.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2827861847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:54 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2202758634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:54 smithi187 ceph-mon[130973]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-11T14:22:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2827861847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:54 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2202758634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 -- 172.21.15.3:0/1458009187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41841024a0 msgr2=0x7f4184108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1458009187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41841024a0 0x7f4184108d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f417c009fc0 tx=0x7f417c00b330 comp rx=0 tx=0).stop 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 -- 172.21.15.3:0/1458009187 shutdown_connections 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1458009187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41841024a0 0x7f4184108d50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1458009187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 0x7f4184100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1458009187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4184101b00 0x7f4184101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 -- 172.21.15.3:0/1458009187 >> 172.21.15.3:0/1458009187 conn(0x7f41840fbe60 msgr2=0x7f41840fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 -- 172.21.15.3:0/1458009187 shutdown_connections 2023-08-11T14:22:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.069+0000 7f418b7ff700 1 -- 172.21.15.3:0/1458009187 wait complete. 2023-08-11T14:22:55.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.070+0000 7f418b7ff700 1 Processor -- start 2023-08-11T14:22:55.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.070+0000 7f418b7ff700 1 -- start start 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.070+0000 7f418b7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 0x7f41841966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.070+0000 7f418b7ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 0x7f4184196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f418b7ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f418b7ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419e240 con 0x7f41841024a0 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f418b7ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419e380 con 0x7f4184101b00 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f418b7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418419e510 con 0x7f4184100910 2023-08-11T14:22:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4189d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4189d9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55594/0 (socket says 172.21.15.3:55594) 2023-08-11T14:22:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 0x7f4184196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4189d9c700 1 -- 172.21.15.3:0/1564284738 learned_addr learned my addr 172.21.15.3:0/1564284738 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f418959b700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 0x7f41841966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 -- 172.21.15.3:0/1564284738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 msgr2=0x7f41841966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 0x7f41841966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 -- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 msgr2=0x7f418419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4188d9a700 1 -- 172.21.15.3:0/1564284738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c009c40 con 0x7f4184101b00 2023-08-11T14:22:55.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.071+0000 7f4189d9c700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f4188d9a700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 0x7f4184196c00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f417400e9f0 tx=0x7f417400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400c9c0 con 0x7f4184101b00 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4174010430 con 0x7f4184101b00 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418419e7f0 con 0x7f4184101b00 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174018680 con 0x7f4184101b00 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.072+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418419ed00 con 0x7f4184101b00 2023-08-11T14:22:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.073+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418410c180 con 0x7f4184101b00 2023-08-11T14:22:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.075+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f4174010860 con 0x7f4184101b00 2023-08-11T14:22:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.076+0000 7f417a7fc700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 0x7f41700a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.077+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f4174014070 con 0x7f4184101b00 2023-08-11T14:22:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.077+0000 7f418959b700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 0x7f41700a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.077+0000 7f418959b700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 0x7f41700a2410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4184101960 tx=0x7f4180009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.078+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f4174095280 con 0x7f4184101b00 2023-08-11T14:22:55.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:54 smithi103 ceph-mon[130434]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2023-08-11T14:22:55.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2827861847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:54 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2202758634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-08-11T14:22:55.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.292+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f41841077f0 con 0x7f4184101b00 2023-08-11T14:22:55.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.293+0000 7f417a7fc700 1 -- 172.21.15.3:0/1564284738 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f41740988a0 con 0x7f4184101b00 2023-08-11T14:22:55.294 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 msgr2=0x7f41700a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 0x7f41700a2410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4184101960 tx=0x7f4180009310 comp rx=0 tx=0).stop 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 msgr2=0x7f4184196c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 0x7f4184196c00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f417400e9f0 tx=0x7f417400ed00 comp rx=0 tx=0).stop 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 shutdown_connections 2023-08-11T14:22:55.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41841024a0 0x7f418419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4184100910 0x7f41841966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f417009ff50 0x7f41700a2410 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 --2- 172.21.15.3:0/1564284738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4184101b00 0x7f4184196c00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.295+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 >> 172.21.15.3:0/1564284738 conn(0x7f41840fbe60 msgr2=0x7f41840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.296+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 shutdown_connections 2023-08-11T14:22:55.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.296+0000 7f418b7ff700 1 -- 172.21.15.3:0/1564284738 wait complete. 2023-08-11T14:22:55.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-11T14:22:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 -- 172.21.15.3:0/3172050052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8101b00 msgr2=0x7f3ac8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/3172050052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8101b00 0x7f3ac8101f60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009920 tx=0x7f3abc009c30 comp rx=0 tx=0).stop 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 -- 172.21.15.3:0/3172050052 shutdown_connections 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/3172050052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 0x7f3ac8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/3172050052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8100910 0x7f3ac8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/3172050052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8101b00 0x7f3ac8101f60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 -- 172.21.15.3:0/3172050052 >> 172.21.15.3:0/3172050052 conn(0x7f3ac80fbe60 msgr2=0x7f3ac80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 -- 172.21.15.3:0/3172050052 shutdown_connections 2023-08-11T14:22:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.520+0000 7f3acc9ff700 1 -- 172.21.15.3:0/3172050052 wait complete. 2023-08-11T14:22:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.521+0000 7f3acc9ff700 1 Processor -- start 2023-08-11T14:22:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 -- start start 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 0x7f3ac8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 0x7f3ac819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81a03d0 con 0x7f3ac8101b00 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81a0510 con 0x7f3ac8100910 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.522+0000 7f3acc9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81a06a0 con 0x7f3ac81024a0 2023-08-11T14:22:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac5d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 0x7f3ac8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43780/0 (socket says 172.21.15.3:43780) 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 -- 172.21.15.3:0/2204606657 learned_addr learned my addr 172.21.15.3:0/2204606657 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac6d9d700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 0x7f3ac819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 -- 172.21.15.3:0/2204606657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 msgr2=0x7f3ac819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 0x7f3ac819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 -- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 msgr2=0x7f3ac8198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 0x7f3ac8198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 -- 172.21.15.3:0/2204606657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8009d00 con 0x7f3ac8100910 2023-08-11T14:22:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac5d9b700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 0x7f3ac8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.523+0000 7f3ac659c700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e3f0 tx=0x7f3ab800e700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.524+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80092d0 con 0x7f3ac8100910 2023-08-11T14:22:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.524+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc009520 con 0x7f3ac8100910 2023-08-11T14:22:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.524+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab800f040 con 0x7f3ac8100910 2023-08-11T14:22:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.524+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80146e0 con 0x7f3ac8100910 2023-08-11T14:22:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.524+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81a0cf0 con 0x7f3ac8100910 2023-08-11T14:22:55.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.525+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8192f10 con 0x7f3ac8100910 2023-08-11T14:22:55.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.527+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f3ab80075c0 con 0x7f3ac8100910 2023-08-11T14:22:55.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.528+0000 7f3ab77fe700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 0x7f3ab00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:55.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.528+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f3ab80cb970 con 0x7f3ac8100910 2023-08-11T14:22:55.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.528+0000 7f3ac5d9b700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 0x7f3ab00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:55.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.529+0000 7f3ac5d9b700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 0x7f3ab00a2410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00fa40 tx=0x7f3abc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.530+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f3ab8095380 con 0x7f3ac8100910 2023-08-11T14:22:55.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.758+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3ac810c120 con 0x7f3ac8100910 2023-08-11T14:22:55.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.759+0000 7f3ab77fe700 1 -- 172.21.15.3:0/2204606657 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+580 (secure 0 0 0) 0x7f3ab80989a0 con 0x7f3ac8100910 2023-08-11T14:22:55.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2366653083 2023-08-12T13:55:02.078980+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1168721553 2023-08-12T13:55:02.078980+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1070970995 2023-08-12T13:55:02.078980+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1776709742 2023-08-12T13:55:02.078980+0000 2023-08-11T14:22:55.761 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/586553398 2023-08-12T13:54:34.603107+0000 2023-08-11T14:22:55.762 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3848927136 2023-08-12T13:54:34.603107+0000 2023-08-11T14:22:55.762 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3010670797 2023-08-12T13:54:34.603107+0000 2023-08-11T14:22:55.762 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1680525538 2023-08-12T13:54:34.603107+0000 2023-08-11T14:22:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 msgr2=0x7f3ab00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 0x7f3ab00a2410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00fa40 tx=0x7f3abc019040 comp rx=0 tx=0).stop 2023-08-11T14:22:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 msgr2=0x7f3ac8198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:55.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e3f0 tx=0x7f3ab800e700 comp rx=0 tx=0).stop 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 shutdown_connections 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ac81024a0 0x7f3ac819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f3ab009ff50 0x7f3ab00a2410 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ac8100910 0x7f3ac8198850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 --2- 172.21.15.3:0/2204606657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ac8101b00 0x7f3ac8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:55.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 >> 172.21.15.3:0/2204606657 conn(0x7f3ac80fbe60 msgr2=0x7f3ac81074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 shutdown_connections 2023-08-11T14:22:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:55.761+0000 7f3acc9ff700 1 -- 172.21.15.3:0/2204606657 wait complete. 2023-08-11T14:22:55.765 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2023-08-11T14:22:55.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-11T14:22:55.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:55.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /tmp/cephfs 2023-08-11T14:22:56.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:55 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1772760038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:56.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:55 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1564284738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:56.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:55 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1772760038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:56.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:55 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1564284738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:56.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:55 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1772760038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-08-11T14:22:56.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:55 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1564284738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-08-11T14:22:57.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:56 smithi003 ceph-mon[97195]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-08-11T14:22:57.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:56 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2204606657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:22:57.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:56 smithi187 ceph-mon[130973]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-08-11T14:22:57.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:56 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2204606657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:22:57.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:56 smithi103 ceph-mon[130434]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-08-11T14:22:57.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:56 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2204606657' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-08-11T14:22:57.381 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:22:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:57.382 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-08-11T14:22:57.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:57.633 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-08-11T14:22:57.719 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-08-11T14:22:57.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:57.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-08-11T14:22:57.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:57.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-08-11T14:22:57.945 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-11T14:22:57.946 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-08-11T14:22:57.946 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-11T14:22:57.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348696k,nr_inodes=4087174,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=38,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/0f5df4c607208318e5ff3878d3008c6ac34cdf12aed86edb60cea37d9015d656/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/0f5df4c607208318e5ff3878d3008c6ac34cdf12aed86edb60cea37d9015d656/diff,workdir=/var/lib/containers/storage/overlay/0f5df4c607208318e5ff3878d3008c6ac34cdf12aed86edb60cea37d9015d656/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/623034d3caba850bec9d3de06b86b293480674b52e49dffb19eb1a2fe9124693/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/623034d3caba850bec9d3de06b86b293480674b52e49dffb19eb1a2fe9124693/diff,workdir=/var/lib/containers/storage/overlay/623034d3caba850bec9d3de06b86b293480674b52e49dffb19eb1a2fe9124693/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/ed97bb901d790646f24dc52e941a34470d20b0a25f60cabc2bba0b78ed05d040/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/ed97bb901d790646f24dc52e941a34470d20b0a25f60cabc2bba0b78ed05d040/diff,workdir=/var/lib/containers/storage/overlay/ed97bb901d790646f24dc52e941a34470d20b0a25f60cabc2bba0b78ed05d040/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7135eadf55d07633638dbeb06029b543ddca08b6b0b84cf7d0c97592d1ff64a1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/7135eadf55d07633638dbeb06029b543ddca08b6b0b84cf7d0c97592d1ff64a1/diff,workdir=/var/lib/containers/storage/overlay/7135eadf55d07633638dbeb06029b543ddca08b6b0b84cf7d0c97592d1ff64a1/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2c31ecf1021d9f56af979949994aadde606a21598c2fb7f583d1f81f7c922f6e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/2c31ecf1021d9f56af979949994aadde606a21598c2fb7f583d1f81f7c922f6e/diff,workdir=/var/lib/containers/storage/overlay/2c31ecf1021d9f56af979949994aadde606a21598c2fb7f583d1f81f7c922f6e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c3b701efead5e1b85c2da1a303987f5e01a218cd770b0f66101ab241a3f97c11/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/c3b701efead5e1b85c2da1a303987f5e01a218cd770b0f66101ab241a3f97c11/diff,workdir=/var/lib/containers/storage/overlay/c3b701efead5e1b85c2da1a303987f5e01a218cd770b0f66101ab241a3f97c11/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3b8c61268494de795c6835de0202e20861a0a449a11e58a7daaf98310247d4ad/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/3b8c61268494de795c6835de0202e20861a0a449a11e58a7daaf98310247d4ad/diff,workdir=/var/lib/containers/storage/overlay/3b8c61268494de795c6835de0202e20861a0a449a11e58a7daaf98310247d4ad/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f17283c84d4d6d614a5346c686d3c1ce84f239f9d4fde7471113e822bc62f747/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/f17283c84d4d6d614a5346c686d3c1ce84f239f9d4fde7471113e822bc62f747/diff,workdir=/var/lib/containers/storage/overlay/f17283c84d4d6d614a5346c686d3c1ce84f239f9d4fde7471113e822bc62f747/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7d9bd72f82ac5c37a69ad903e5e939f28a20b7b3755312bc0f735b9a8593bb5f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/7d9bd72f82ac5c37a69ad903e5e939f28a20b7b3755312bc0f735b9a8593bb5f/diff,workdir=/var/lib/containers/storage/overlay/7d9bd72f82ac5c37a69ad903e5e939f28a20b7b3755312bc0f735b9a8593bb5f/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1bda6235d38e32f81df7bd46a67c02cf930183ee059048f848fea2caa149d93f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/C72U56F4ZYRHVY5WUXN4VP6BFA:/var/lib/containers/storage/overlay/l/A7NL7KYCZEOAI76INBZCMGWMBD,upperdir=/var/lib/containers/storage/overlay/1bda6235d38e32f81df7bd46a67c02cf930183ee059048f848fea2caa149d93f/diff,workdir=/var/lib/containers/storage/overlay/1bda6235d38e32f81df7bd46a67c02cf930183ee059048f848fea2caa149d93f/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-08-11T14:22:57.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:57.994 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-08-11T14:22:58.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:57 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:22:58.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:57 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:22:58.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-11T14:22:58.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-08-11T14:22:58.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:57 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-08-11T14:22:58.325 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2023-08-11T14:22:58.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:22:58.327 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-08-11T14:22:58.339 INFO:tasks.cephadm:Teardown begin 2023-08-11T14:22:58.339 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-11T14:22:58.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:22:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 -- 172.21.15.3:0/3170148718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 msgr2=0x7f0590108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 --2- 172.21.15.3:0/3170148718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f0590108d70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f058800b0d0 tx=0x7f058800b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 -- 172.21.15.3:0/3170148718 shutdown_connections 2023-08-11T14:22:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 --2- 172.21.15.3:0/3170148718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f0590108d70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 --2- 172.21.15.3:0/3170148718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f0590100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 --2- 172.21.15.3:0/3170148718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 -- 172.21.15.3:0/3170148718 >> 172.21.15.3:0/3170148718 conn(0x7f05900fbe60 msgr2=0x7f05900fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:58.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.544+0000 7f0597fff700 1 -- 172.21.15.3:0/3170148718 shutdown_connections 2023-08-11T14:22:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.545+0000 7f0597fff700 1 -- 172.21.15.3:0/3170148718 wait complete. 2023-08-11T14:22:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.545+0000 7f0597fff700 1 Processor -- start 2023-08-11T14:22:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 -- start start 2023-08-11T14:22:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f05901966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059019e240 con 0x7f05901024c0 2023-08-11T14:22:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059019e380 con 0x7f0590101b20 2023-08-11T14:22:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0597fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059019e510 con 0x7f0590100930 2023-08-11T14:22:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f059659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f0595d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f05901966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f059659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55628/0 (socket says 172.21.15.3:55628) 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f059659c700 1 -- 172.21.15.3:0/2488725228 learned_addr learned my addr 172.21.15.3:0/2488725228 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.546+0000 7f059559a700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 -- 172.21.15.3:0/2488725228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 msgr2=0x7f05901966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f05901966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 -- 172.21.15.3:0/2488725228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 msgr2=0x7f0590196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 -- 172.21.15.3:0/2488725228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588009d00 con 0x7f05901024c0 2023-08-11T14:22:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f0595d9b700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f05901966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059559a700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f059659c700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0588008eb0 tx=0x7f0588008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058800e040 con 0x7f05901024c0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f058800bdb0 con 0x7f05901024c0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059019e790 con 0x7f05901024c0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.547+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058801d470 con 0x7f05901024c0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.548+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059019ec10 con 0x7f05901024c0 2023-08-11T14:22:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.548+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059010c1a0 con 0x7f05901024c0 2023-08-11T14:22:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.551+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f0588005000 con 0x7f05901024c0 2023-08-11T14:22:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.552+0000 7f0586ffd700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 0x7f057c0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:58.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.552+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f05880ccae0 con 0x7f05901024c0 2023-08-11T14:22:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.552+0000 7f0595d9b700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 0x7f057c0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.553+0000 7f0595d9b700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 0x7f057c0a2460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0590101980 tx=0x7f058c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:58.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.554+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f0588096580 con 0x7f05901024c0 2023-08-11T14:22:58.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.810+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0590107730 con 0x7f05901024c0 2023-08-11T14:22:58.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.811+0000 7f0586ffd700 1 -- 172.21.15.3:0/2488725228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f0588099ba0 con 0x7f05901024c0 2023-08-11T14:22:58.812 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:58.813 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:18:52.434922+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:22:58.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 msgr2=0x7f057c0a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:58.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 0x7f057c0a2460 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0590101980 tx=0x7f058c009260 comp rx=0 tx=0).stop 2023-08-11T14:22:58.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 msgr2=0x7f059019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:58.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0588008eb0 tx=0x7f0588008ee0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 shutdown_connections 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05901024c0 0x7f059019dc70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0590100930 0x7f05901966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f057c09ffa0 0x7f057c0a2460 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 --2- 172.21.15.3:0/2488725228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0590101b20 0x7f0590196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.813+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 >> 172.21.15.3:0/2488725228 conn(0x7f05900fbe60 msgr2=0x7f05900fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.814+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 shutdown_connections 2023-08-11T14:22:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:58.814+0000 7f0597fff700 1 -- 172.21.15.3:0/2488725228 wait complete. 2023-08-11T14:22:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 58 2023-08-11T14:22:58.835 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-11T14:22:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 -- 172.21.15.3:0/2915552441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 msgr2=0x7f33e810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 --2- 172.21.15.3:0/2915552441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e810ac70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f33e000b0a0 tx=0x7f33e000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 -- 172.21.15.3:0/2915552441 shutdown_connections 2023-08-11T14:22:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 --2- 172.21.15.3:0/2915552441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e810ac70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 --2- 172.21.15.3:0/2915552441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8074960 0x7f33e8072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 --2- 172.21.15.3:0/2915552441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8073380 0x7f33e8073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 -- 172.21.15.3:0/2915552441 >> 172.21.15.3:0/2915552441 conn(0x7f33e80fbe50 msgr2=0x7f33e80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 -- 172.21.15.3:0/2915552441 shutdown_connections 2023-08-11T14:22:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.039+0000 7f33efdff700 1 -- 172.21.15.3:0/2915552441 wait complete. 2023-08-11T14:22:59.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:58 smithi003 ceph-mon[97195]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2023-08-11T14:22:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:22:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:22:59.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:58 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:22:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.040+0000 7f33efdff700 1 Processor -- start 2023-08-11T14:22:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 -- start start 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 0x7f33e81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 0x7f33e8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81a0440 con 0x7f33e81087e0 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81a0580 con 0x7f33e8073380 2023-08-11T14:22:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33efdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81a0710 con 0x7f33e8074960 2023-08-11T14:22:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33ee39c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33ed39a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 0x7f33e8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33edb9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 0x7f33e81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33ee39c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55644/0 (socket says 172.21.15.3:55644) 2023-08-11T14:22:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.041+0000 7f33ee39c700 1 -- 172.21.15.3:0/2104203476 learned_addr learned my addr 172.21.15.3:0/2104203476 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 -- 172.21.15.3:0/2104203476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 msgr2=0x7f33e8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 0x7f33e8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 -- 172.21.15.3:0/2104203476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 msgr2=0x7f33e81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 0x7f33e81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 -- 172.21.15.3:0/2104203476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0009d00 con 0x7f33e81087e0 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ed39a700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 0x7f33e8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33edb9b700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 0x7f33e81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33ee39c700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f33e000b780 tx=0x7f33e000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e000e040 con 0x7f33e81087e0 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e81a0990 con 0x7f33e81087e0 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33e0004c50 con 0x7f33e81087e0 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.042+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0005280 con 0x7f33e81087e0 2023-08-11T14:22:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.043+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e81a0e60 con 0x7f33e81087e0 2023-08-11T14:22:59.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:58 smithi187 ceph-mon[130973]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2023-08-11T14:22:59.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:22:59.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:22:59.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:58 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:22:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.044+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8106e90 con 0x7f33e81087e0 2023-08-11T14:22:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.046+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f33e001d470 con 0x7f33e81087e0 2023-08-11T14:22:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.047+0000 7f33deffd700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 0x7f33d40a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.047+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7f33e00cca40 con 0x7f33e81087e0 2023-08-11T14:22:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.047+0000 7f33edb9b700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 0x7f33d40a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.048+0000 7f33edb9b700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 0x7f33d40a24e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33e4011490 tx=0x7f33e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.051+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f33e00964e0 con 0x7f33e81087e0 2023-08-11T14:22:59.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:58 smithi103 ceph-mon[130434]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 11 op/s 2023-08-11T14:22:59.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-08-11T14:22:59.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-08-11T14:22:59.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:58 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' 2023-08-11T14:22:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.310+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e810e310 con 0x7f33e81087e0 2023-08-11T14:22:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.311+0000 7f33deffd700 1 -- 172.21.15.3:0/2104203476 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+11366 (secure 0 0 0) 0x7f33e0099b00 con 0x7f33e81087e0 2023-08-11T14:22:59.312 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:59.312 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/113676080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":113676080},{"type":"v1","addr":"172.21.15.3:6835","nonce":113676080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/3880902111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":3880902111},{"type":"v1","addr":"172.21.15.3:6837","nonce":3880902111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-08-11T14:01:32.831322+0000","modified":"2023-08-11T14:18:52.434922+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24532,"mds_2":24527,"mds_3":24511,"mds_4":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14502":{"gid":14502,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/3729253320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3729253320},{"type":"v1","addr":"172.21.15.3:6839","nonce":3729253320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14514":{"gid":14514,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6841/1195165407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1195165407},{"type":"v1","addr":"172.21.15.3:6841","nonce":1195165407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1102333992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1102333992},{"type":"v1","addr":"172.21.15.187:6833","nonce":1102333992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/618362106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":618362106},{"type":"v1","addr":"172.21.15.103:6833","nonce":618362106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/3236312344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":3236312344},{"type":"v1","addr":"172.21.15.187:6835","nonce":3236312344}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/1763153778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1763153778},{"type":"v1","addr":"172.21.15.103:6835","nonce":1763153778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24511":{"gid":24511,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6837/1876247524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":1876247524},{"type":"v1","addr":"172.21.15.187:6837","nonce":1876247524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/4115864783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":4115864783},{"type":"v1","addr":"172.21.15.103:6837","nonce":4115864783}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.187:6839/1685844974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":1685844974},{"type":"v1","addr":"172.21.15.187:6839","nonce":1685844974}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6839/2494449018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2494449018},{"type":"v1","addr":"172.21.15.103:6839","nonce":2494449018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-11T14:22:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 msgr2=0x7f33d40a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 0x7f33d40a24e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33e4011490 tx=0x7f33e4009260 comp rx=0 tx=0).stop 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 msgr2=0x7f33e819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f33e000b780 tx=0x7f33e000bab0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 shutdown_connections 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33e81087e0 0x7f33e819fe70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f33d40a0020 0x7f33d40a24e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e8073380 0x7f33e81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 --2- 172.21.15.3:0/2104203476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33e8074960 0x7f33e8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 >> 172.21.15.3:0/2104203476 conn(0x7f33e80fbe50 msgr2=0x7f33e80fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 shutdown_connections 2023-08-11T14:22:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.313+0000 7f33efdff700 1 -- 172.21.15.3:0/2104203476 wait complete. 2023-08-11T14:22:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 58 2023-08-11T14:22:59.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-11T14:22:59.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 -- 172.21.15.3:0/1101181255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab78101b20 msgr2=0x7fab78101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/1101181255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab78101b20 0x7fab78101f80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009920 tx=0x7fab6c009c30 comp rx=0 tx=0).stop 2023-08-11T14:22:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 -- 172.21.15.3:0/1101181255 shutdown_connections 2023-08-11T14:22:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/1101181255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab781024c0 0x7fab78108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/1101181255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78100930 0x7fab78100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/1101181255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab78101b20 0x7fab78101f80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 -- 172.21.15.3:0/1101181255 >> 172.21.15.3:0/1101181255 conn(0x7fab780fbe60 msgr2=0x7fab780fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 -- 172.21.15.3:0/1101181255 shutdown_connections 2023-08-11T14:22:59.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.538+0000 7fab7e5ff700 1 -- 172.21.15.3:0/1101181255 wait complete. 2023-08-11T14:22:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.539+0000 7fab7e5ff700 1 Processor -- start 2023-08-11T14:22:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.539+0000 7fab7e5ff700 1 -- start start 2023-08-11T14:22:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.539+0000 7fab7e5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 0x7fab78198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7e5ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7e5ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab781024c0 0x7fab7819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7e5ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781a03d0 con 0x7fab78101b20 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7e5ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781a0510 con 0x7fab781024c0 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7e5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781a06a0 con 0x7fab78100930 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55654/0 (socket says 172.21.15.3:55654) 2023-08-11T14:22:59.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 -- 172.21.15.3:0/860993924 learned_addr learned my addr 172.21.15.3:0/860993924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab77fff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 0x7fab78198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab7cb9c700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab781024c0 0x7fab7819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 -- 172.21.15.3:0/860993924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 msgr2=0x7fab78198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 0x7fab78198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 -- 172.21.15.3:0/860993924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab781024c0 msgr2=0x7fab7819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab781024c0 0x7fab7819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab777fe700 1 -- 172.21.15.3:0/860993924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680095f0 con 0x7fab78101b20 2023-08-11T14:22:59.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.540+0000 7fab77fff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 0x7fab78198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:22:59.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab777fe700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009230 tx=0x7fab6c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:59.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c01c070 con 0x7fab78101b20 2023-08-11T14:22:59.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c009520 con 0x7fab78101b20 2023-08-11T14:22:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab6c005040 con 0x7fab78101b20 2023-08-11T14:22:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c003ae0 con 0x7fab78101b20 2023-08-11T14:22:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.541+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab781a0bb0 con 0x7fab78101b20 2023-08-11T14:22:59.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.542+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7810c190 con 0x7fab78101b20 2023-08-11T14:22:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.544+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fab6c020030 con 0x7fab78101b20 2023-08-11T14:22:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.545+0000 7fab757fa700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 0x7fab600a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.545+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fab6c0cc510 con 0x7fab78101b20 2023-08-11T14:22:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.546+0000 7fab77fff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 0x7fab600a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.546+0000 7fab77fff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 0x7fab600a24e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fab78101980 tx=0x7fab68009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:59.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.549+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fab6c095fb0 con 0x7fab78101b20 2023-08-11T14:22:59.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.762+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab781a0dd0 con 0x7fab78101b20 2023-08-11T14:22:59.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.764+0000 7fab757fa700 1 -- 172.21.15.3:0/860993924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24460 (secure 0 0 0) 0x7fab6c0995d0 con 0x7fab78101b20 2023-08-11T14:22:59.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-08-11T14:22:59.766 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"69ee59aa-384e-11ee-9b37-001a4aab830c","created":"2023-08-11T13:54:00.622002+0000","modified":"2023-08-11T14:03:04.331784+0000","last_up_change":"2023-08-11T14:00:44.254747+0000","last_in_change":"2023-08-11T14:00:27.303576+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-11T13:57:20.695647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-08-11T14:01:30.942315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-08-11T14:01:32.272156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-08-11T14:01:35.852024+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f155a1aa-25d5-4785-9e30-63ab3dc892ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6803","nonce":2824703498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6805","nonce":2824703498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6809","nonce":2824703498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2824703498},{"type":"v1","addr":"172.21.15.3:6807","nonce":2824703498}]},"public_addr":"172.21.15.3:6803/2824703498","cluster_addr":"172.21.15.3:6805/2824703498","heartbeat_back_addr":"172.21.15.3:6809/2824703498","heartbeat_front_addr":"172.21.15.3:6807/2824703498","state":["exists","up"]},{"osd":1,"uuid":"86cb5b3f-ad6a-47d8-8241-18b6a6889961","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6801","nonce":1127248620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6803","nonce":1127248620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6807","nonce":1127248620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1127248620},{"type":"v1","addr":"172.21.15.103:6805","nonce":1127248620}]},"public_addr":"172.21.15.103:6801/1127248620","cluster_addr":"172.21.15.103:6803/1127248620","heartbeat_back_addr":"172.21.15.103:6807/1127248620","heartbeat_front_addr":"172.21.15.103:6805/1127248620","state":["exists","up"]},{"osd":2,"uuid":"3d3838c3-0422-4ead-946c-f9dd572bff9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6801","nonce":2156234450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6803","nonce":2156234450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6807","nonce":2156234450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2156234450},{"type":"v1","addr":"172.21.15.187:6805","nonce":2156234450}]},"public_addr":"172.21.15.187:6801/2156234450","cluster_addr":"172.21.15.187:6803/2156234450","heartbeat_back_addr":"172.21.15.187:6807/2156234450","heartbeat_front_addr":"172.21.15.187:6805/2156234450","state":["exists","up"]},{"osd":3,"uuid":"58d0f37b-9009-4887-b9fb-daa5c9d047e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6811","nonce":2193340429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6813","nonce":2193340429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6817","nonce":2193340429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2193340429},{"type":"v1","addr":"172.21.15.3:6815","nonce":2193340429}]},"public_addr":"172.21.15.3:6811/2193340429","cluster_addr":"172.21.15.3:6813/2193340429","heartbeat_back_addr":"172.21.15.3:6817/2193340429","heartbeat_front_addr":"172.21.15.3:6815/2193340429","state":["exists","up"]},{"osd":4,"uuid":"1c77ed74-b7d5-4c96-8507-8cd0a6a5b379","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6809","nonce":811611617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6811","nonce":811611617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6815","nonce":811611617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":811611617},{"type":"v1","addr":"172.21.15.103:6813","nonce":811611617}]},"public_addr":"172.21.15.103:6809/811611617","cluster_addr":"172.21.15.103:6811/811611617","heartbeat_back_addr":"172.21.15.103:6815/811611617","heartbeat_front_addr":"172.21.15.103:6813/811611617","state":["exists","up"]},{"osd":5,"uuid":"03c1c84c-d569-4383-b6b6-c0a38ea0ac4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6809","nonce":916876636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6811","nonce":916876636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6815","nonce":916876636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":916876636},{"type":"v1","addr":"172.21.15.187:6813","nonce":916876636}]},"public_addr":"172.21.15.187:6809/916876636","cluster_addr":"172.21.15.187:6811/916876636","heartbeat_back_addr":"172.21.15.187:6815/916876636","heartbeat_front_addr":"172.21.15.187:6813/916876636","state":["exists","up"]},{"osd":6,"uuid":"46bc04c3-30e0-461f-a65c-a9acecf575a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6819","nonce":3251538985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6821","nonce":3251538985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6825","nonce":3251538985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3251538985},{"type":"v1","addr":"172.21.15.3:6823","nonce":3251538985}]},"public_addr":"172.21.15.3:6819/3251538985","cluster_addr":"172.21.15.3:6821/3251538985","heartbeat_back_addr":"172.21.15.3:6825/3251538985","heartbeat_front_addr":"172.21.15.3:6823/3251538985","state":["exists","up"]},{"osd":7,"uuid":"1080578d-40d4-418d-9db4-999d831dd105","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6817","nonce":3619083157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6819","nonce":3619083157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6823","nonce":3619083157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3619083157},{"type":"v1","addr":"172.21.15.103:6821","nonce":3619083157}]},"public_addr":"172.21.15.103:6817/3619083157","cluster_addr":"172.21.15.103:6819/3619083157","heartbeat_back_addr":"172.21.15.103:6823/3619083157","heartbeat_front_addr":"172.21.15.103:6821/3619083157","state":["exists","up"]},{"osd":8,"uuid":"295943c9-dfde-47cd-96f0-570e5e989919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6817","nonce":4219051644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6819","nonce":4219051644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6823","nonce":4219051644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":4219051644},{"type":"v1","addr":"172.21.15.187:6821","nonce":4219051644}]},"public_addr":"172.21.15.187:6817/4219051644","cluster_addr":"172.21.15.187:6819/4219051644","heartbeat_back_addr":"172.21.15.187:6823/4219051644","heartbeat_front_addr":"172.21.15.187:6821/4219051644","state":["exists","up"]},{"osd":9,"uuid":"9cd878b1-5843-4397-a987-2437787cd241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6827","nonce":3936829119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6829","nonce":3936829119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6833","nonce":3936829119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3936829119},{"type":"v1","addr":"172.21.15.3:6831","nonce":3936829119}]},"public_addr":"172.21.15.3:6827/3936829119","cluster_addr":"172.21.15.3:6829/3936829119","heartbeat_back_addr":"172.21.15.3:6833/3936829119","heartbeat_front_addr":"172.21.15.3:6831/3936829119","state":["exists","up"]},{"osd":10,"uuid":"51595dfa-2865-4bde-985a-b538b1d68644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6825","nonce":1382085877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6827","nonce":1382085877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6831","nonce":1382085877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1382085877},{"type":"v1","addr":"172.21.15.103:6829","nonce":1382085877}]},"public_addr":"172.21.15.103:6825/1382085877","cluster_addr":"172.21.15.103:6827/1382085877","heartbeat_back_addr":"172.21.15.103:6831/1382085877","heartbeat_front_addr":"172.21.15.103:6829/1382085877","state":["exists","up"]},{"osd":11,"uuid":"b3738e6b-7935-4522-89c7-91cd190ab5e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6825","nonce":3190542621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6827","nonce":3190542621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6831","nonce":3190542621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3190542621},{"type":"v1","addr":"172.21.15.187:6829","nonce":3190542621}]},"public_addr":"172.21.15.187:6825/3190542621","cluster_addr":"172.21.15.187:6827/3190542621","heartbeat_back_addr":"172.21.15.187:6831/3190542621","heartbeat_front_addr":"172.21.15.187:6829/3190542621","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:34.551829+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:56:56.339536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:17.385288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:57:43.054726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:04.539528+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:25.363558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:58:51.693550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:13.076638+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T13:59:34.000403+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:00.342102+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:22.405749+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-11T14:00:43.016017+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2366653083":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/1168721553":"2023-08-12T13:55:02.078980+0000","172.21.15.3:0/1070970995":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6801/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1776709742":"2023-08-12T13:55:02.078980+0000","172.21.15.3:6800/586553398":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3848927136":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/3010670797":"2023-08-12T13:54:34.603107+0000","172.21.15.3:0/1680525538":"2023-08-12T13:54:34.603107+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-11T14:22:59.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 msgr2=0x7fab600a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 0x7fab600a24e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fab78101980 tx=0x7fab68009310 comp rx=0 tx=0).stop 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 msgr2=0x7fab78198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009230 tx=0x7fab6c00bf80 comp rx=0 tx=0).stop 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 shutdown_connections 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fab600a0020 0x7fab600a24e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab781024c0 0x7fab7819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab78100930 0x7fab78198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 --2- 172.21.15.3:0/860993924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab78101b20 0x7fab78198d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 >> 172.21.15.3:0/860993924 conn(0x7fab780fbe60 msgr2=0x7fab780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 shutdown_connections 2023-08-11T14:22:59.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.766+0000 7fab7e5ff700 1 -- 172.21.15.3:0/860993924 wait complete. 2023-08-11T14:22:59.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-11T14:22:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 -- 172.21.15.3:0/2454062960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 msgr2=0x7fb800100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 --2- 172.21.15.3:0/2454062960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb800100d30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00b0d0 tx=0x7fb7fc00b3e0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 -- 172.21.15.3:0/2454062960 shutdown_connections 2023-08-11T14:22:59.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 --2- 172.21.15.3:0/2454062960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8001024a0 0x7fb800108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 --2- 172.21.15.3:0/2454062960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb800100d30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 --2- 172.21.15.3:0/2454062960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb800101b00 0x7fb800101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.990+0000 7fb8081ff700 1 -- 172.21.15.3:0/2454062960 >> 172.21.15.3:0/2454062960 conn(0x7fb8000fbe60 msgr2=0x7fb8000fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:22:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.991+0000 7fb8081ff700 1 -- 172.21.15.3:0/2454062960 shutdown_connections 2023-08-11T14:22:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.991+0000 7fb8081ff700 1 -- 172.21.15.3:0/2454062960 wait complete. 2023-08-11T14:22:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 Processor -- start 2023-08-11T14:22:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 -- start start 2023-08-11T14:22:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 0x7fb80019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001a0530 con 0x7fb800100910 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001a0670 con 0x7fb800101b00 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb8081ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001a0800 con 0x7fb8001024a0 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb805f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb80579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb805f9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55670/0 (socket says 172.21.15.3:55670) 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb80579a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.3:43852/0 (socket says 172.21.15.3:43852) 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.992+0000 7fb805f9b700 1 -- 172.21.15.3:0/142778244 learned_addr learned my addr 172.21.15.3:0/142778244 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80679c700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 0x7fb80019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 -- 172.21.15.3:0/142778244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 msgr2=0x7fb80019ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 0x7fb80019ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 -- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 msgr2=0x7fb8001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:22:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 -- 172.21.15.3:0/142778244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc009d00 con 0x7fb800101b00 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb805f9b700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80679c700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 0x7fb80019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb80579a700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9f0 tx=0x7fb7f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00096d0 con 0x7fb800101b00 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8001a0ae0 con 0x7fb800101b00 2023-08-11T14:22:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f000cbd0 con 0x7fb800101b00 2023-08-11T14:22:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.993+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0018680 con 0x7fb800101b00 2023-08-11T14:22:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.994+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8001a1040 con 0x7fb800101b00 2023-08-11T14:22:59.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.995+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80010c170 con 0x7fb800101b00 2023-08-11T14:23:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.996+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fb7f000cd40 con 0x7fb800101b00 2023-08-11T14:23:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.998+0000 7fb7f6ffd700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 0x7fb7ec0a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.998+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9028+0+0 (secure 0 0 0) 0x7fb7f0014070 con 0x7fb800101b00 2023-08-11T14:23:00.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.998+0000 7fb805f9b700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 0x7fb7ec0a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:00.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:22:59.999+0000 7fb805f9b700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 0x7fb7ec0a2490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009fd0 tx=0x7fb7fc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:00.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.000+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fb7f0095180 con 0x7fb800101b00 2023-08-11T14:23:00.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2488725228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:22:59 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/2104203476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.043 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2488725228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:22:59 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/2104203476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2488725228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:22:59 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/2104203476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-08-11T14:23:00.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.261+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb80010c330 con 0x7fb800101b00 2023-08-11T14:23:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.274+0000 7fb7f6ffd700 1 -- 172.21.15.3:0/142778244 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb7f00987a0 con 0x7fb800101b00 2023-08-11T14:23:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.276+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 msgr2=0x7fb7ec0a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.276+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 0x7fb7ec0a2490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009fd0 tx=0x7fb7fc00b040 comp rx=0 tx=0).stop 2023-08-11T14:23:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.276+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 msgr2=0x7fb800198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.276+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9f0 tx=0x7fb7f000ed00 comp rx=0 tx=0).stop 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 shutdown_connections 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8001024a0 0x7fb80019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb800100910 0x7fb8001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fb7ec09ffd0 0x7fb7ec0a2490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 --2- 172.21.15.3:0/142778244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb800101b00 0x7fb800198ef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 >> 172.21.15.3:0/142778244 conn(0x7fb8000fbe60 msgr2=0x7fb8000fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 shutdown_connections 2023-08-11T14:23:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.277+0000 7fb8081ff700 1 -- 172.21.15.3:0/142778244 wait complete. 2023-08-11T14:23:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-11T14:23:00.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-11T14:23:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 -- 172.21.15.3:0/4280180020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 msgr2=0x7f1a58107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/4280180020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a58107c50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b0a0 tx=0x7f1a5000b3b0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 -- 172.21.15.3:0/4280180020 shutdown_connections 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/4280180020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a58107c50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/4280180020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a580ff810 0x7f1a580ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/4280180020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a58100a00 0x7f1a58100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 -- 172.21.15.3:0/4280180020 >> 172.21.15.3:0/4280180020 conn(0x7f1a580fad80 msgr2=0x7f1a580fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 -- 172.21.15.3:0/4280180020 shutdown_connections 2023-08-11T14:23:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.517+0000 7f1a5ffff700 1 -- 172.21.15.3:0/4280180020 wait complete. 2023-08-11T14:23:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 Processor -- start 2023-08-11T14:23:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 -- start start 2023-08-11T14:23:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a580ff810 0x7f1a58198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a5819ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581a04e0 con 0x7f1a58100a00 2023-08-11T14:23:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581a0620 con 0x7f1a580ff810 2023-08-11T14:23:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.518+0000 7f1a5ffff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a581a07b0 con 0x7f1a581013a0 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55722/0 (socket says 172.21.15.3:55722) 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 -- 172.21.15.3:0/1729317386 learned_addr learned my addr 172.21.15.3:0/1729317386 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5dd9b700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a580ff810 0x7f1a58198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5e59c700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a5819ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 -- 172.21.15.3:0/1729317386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 msgr2=0x7f1a5819ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a5819ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 -- 172.21.15.3:0/1729317386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a580ff810 msgr2=0x7f1a58198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a580ff810 0x7f1a58198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 -- 172.21.15.3:0/1729317386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48009520 con 0x7f1a58100a00 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.519+0000 7f1a5d59a700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800eb30 tx=0x7f1a4800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.520+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48009880 con 0x7f1a58100a00 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.520+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a50009d00 con 0x7f1a58100a00 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.520+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a480044e0 con 0x7f1a58100a00 2023-08-11T14:23:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.520+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003d80 con 0x7f1a58100a00 2023-08-11T14:23:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.520+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a581a0e60 con 0x7f1a58100a00 2023-08-11T14:23:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.521+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5810b080 con 0x7f1a58100a00 2023-08-11T14:23:00.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.523+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f1a48003f20 con 0x7f1a58100a00 2023-08-11T14:23:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.524+0000 7f1a4effd700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 0x7f1a440a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.524+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9458+0+0 (secure 0 0 0) 0x7f1a48014070 con 0x7f1a58100a00 2023-08-11T14:23:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.525+0000 7f1a5dd9b700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 0x7f1a440a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:00.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.525+0000 7f1a5dd9b700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 0x7f1a440a2410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1a58100860 tx=0x7f1a54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:00.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.528+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f1a48095500 con 0x7f1a58100a00 2023-08-11T14:23:00.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:23:00.263+0000 7f7a3f2b6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T14:23:00.541 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.541 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mds[131991]: -- 172.21.15.3:0/1579826766 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4e082b1e0 con 0x55b4e0801000 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: starting mds.a at 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mds[131604]: -- 172.21.15.3:0/2354454853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa86daf1e0 con 0x55aa86d84800 2023-08-11T14:23:00.542 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: starting mds.g at 2023-08-11T14:23:00.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.679 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mds[155288]: -- 172.21.15.103:0/3603235002 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1837fb1e0 con 0x55f1837d0800 2023-08-11T14:23:00.679 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: starting mds.h at 2023-08-11T14:23:00.679 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.679 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.680 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mds[155627]: -- 172.21.15.103:0/89446061 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563897ae71e0 con 0x563897abc800 2023-08-11T14:23:00.680 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: starting mds.b at 2023-08-11T14:23:00.680 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.680 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.680 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mds[154945]: -- 172.21.15.103:0/3703660505 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56334f2011e0 con 0x56334f1d7000 2023-08-11T14:23:00.681 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: starting mds.e at 2023-08-11T14:23:00.681 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.681 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.681 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mds[154591]: -- 172.21.15.103:0/2202321298 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fff7edb1e0 con 0x55fff7eb1000 2023-08-11T14:23:00.681 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: starting mds.k at 2023-08-11T14:23:00.729 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.730 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.730 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mds[156582]: -- 172.21.15.187:0/2115455444 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56537c5691e0 con 0x56537c53f000 2023-08-11T14:23:00.730 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: starting mds.c at 2023-08-11T14:23:00.731 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.731 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.731 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mds[156244]: -- 172.21.15.187:0/922937435 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55761f3591e0 con 0x55761f32ec00 2023-08-11T14:23:00.731 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: starting mds.f at 2023-08-11T14:23:00.732 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.732 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.732 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mds[156925]: -- 172.21.15.187:0/2399624686 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5645a03191e0 con 0x5645a02eec00 2023-08-11T14:23:00.733 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: starting mds.i at 2023-08-11T14:23:00.733 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: ignoring --setuser ceph since I am not root 2023-08-11T14:23:00.734 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: ignoring --setgroup ceph since I am not root 2023-08-11T14:23:00.734 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mds[155884]: -- 172.21.15.187:0/3078376112 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5599b49f71e0 con 0x5599b49cc800 2023-08-11T14:23:00.734 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: starting mds.l at 2023-08-11T14:23:00.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:00.788+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1a581a1110 con 0x7f1a58100a00 2023-08-11T14:23:01.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-08-11T14:23:01.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/860993924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:23:01.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/142778244' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T14:23:01.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T14:23:01.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:00 smithi003 ceph-mon[97195]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-08-11T14:23:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-08-11T14:23:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/860993924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:23:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/142778244' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T14:23:01.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T14:23:01.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-08-11T14:23:01.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T14:23:01.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T14:23:01.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:00 smithi187 ceph-mon[130973]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-08-11T14:23:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-08-11T14:23:01.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/860993924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/142778244' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: osdmap e92: 12 total, 12 up, 12 in 2023-08-11T14:23:01.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-08-11T14:23:01.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:00 smithi103 ceph-mon[130434]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-08-11T14:23:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.288+0000 7f1a4effd700 1 -- 172.21.15.3:0/1729317386 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1a48098b20 con 0x7f1a58100a00 2023-08-11T14:23:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 msgr2=0x7f1a440a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 0x7f1a440a2410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1a58100860 tx=0x7f1a54009260 comp rx=0 tx=0).stop 2023-08-11T14:23:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 msgr2=0x7f1a58198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800eb30 tx=0x7f1a4800ee40 comp rx=0 tx=0).stop 2023-08-11T14:23:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 shutdown_connections 2023-08-11T14:23:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a581013a0 0x7f1a5819ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f1a4409ff50 0x7f1a440a2410 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a580ff810 0x7f1a58198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 --2- 172.21.15.3:0/1729317386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a58100a00 0x7f1a58198ea0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 >> 172.21.15.3:0/1729317386 conn(0x7f1a580fad80 msgr2=0x7f1a58067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.292+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 shutdown_connections 2023-08-11T14:23:01.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.293+0000 7f1a5ffff700 1 -- 172.21.15.3:0/1729317386 wait complete. 2023-08-11T14:23:01.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-11T14:23:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 -- 172.21.15.3:0/859783306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 msgr2=0x7f72c4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/859783306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4100d30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f72b40099c0 tx=0x7f72b4009cd0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 -- 172.21.15.3:0/859783306 shutdown_connections 2023-08-11T14:23:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/859783306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c41024a0 0x7f72c4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/859783306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4100d30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/859783306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c4101b00 0x7f72c4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 -- 172.21.15.3:0/859783306 >> 172.21.15.3:0/859783306 conn(0x7f72c40fbe60 msgr2=0x7f72c40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 -- 172.21.15.3:0/859783306 shutdown_connections 2023-08-11T14:23:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.529+0000 7f72ca1ff700 1 -- 172.21.15.3:0/859783306 wait complete. 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 Processor -- start 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 -- start start 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c4101b00 0x7f72c4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 0x7f72c419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:01.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c41a0400 con 0x7f72c4100910 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c41a0540 con 0x7f72c41024a0 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.530+0000 7f72ca1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c41a06d0 con 0x7f72c4101b00 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44872/0 (socket says 172.21.15.3:44872) 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 -- 172.21.15.3:0/3085569361 learned_addr learned my addr 172.21.15.3:0/3085569361 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72bbfff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c4101b00 0x7f72c4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:01.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c3fff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 0x7f72c419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 -- 172.21.15.3:0/3085569361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c4101b00 msgr2=0x7f72c4198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c4101b00 0x7f72c4198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 -- 172.21.15.3:0/3085569361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 msgr2=0x7f72c419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 0x7f72c419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 -- 172.21.15.3:0/3085569361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40095f0 con 0x7f72c4100910 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c3fff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 0x7f72c419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-11T14:23:01.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.531+0000 7f72c37fe700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f72b40098e0 tx=0x7f72b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.532+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b401d070 con 0x7f72c4100910 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.532+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c41a0950 con 0x7f72c4100910 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.532+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72b4005510 con 0x7f72c4100910 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.532+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4004c00 con 0x7f72c4100910 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.532+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c41a0dd0 con 0x7f72c4100910 2023-08-11T14:23:01.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.533+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c410c170 con 0x7f72c4100910 2023-08-11T14:23:01.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.535+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f72b400fb80 con 0x7f72c4100910 2023-08-11T14:23:01.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.536+0000 7f72c17fa700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 0x7f72b00a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:01.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.536+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9458+0+0 (secure 0 0 0) 0x7f72b40cc650 con 0x7f72c4100910 2023-08-11T14:23:01.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.537+0000 7f72bbfff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 0x7f72b00a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.537+0000 7f72bbfff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 0x7f72b00a2410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009c60 tx=0x7f72ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.540+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f72b4095f40 con 0x7f72c4100910 2023-08-11T14:23:01.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:01.756+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72c41a1320 con 0x7f72c4100910 2023-08-11T14:23:02.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: Cluster is now healthy 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6833/3478423035,v1:172.21.15.103:6837/3478423035] up:boot 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6835/2489198081,v1:172.21.15.103:6839/2489198081] up:boot 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6832/733137907,v1:172.21.15.103:6836/733137907] up:boot 2023-08-11T14:23:02.042 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.103:6834/2950211492,v1:172.21.15.103:6838/2950211492] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6832/2445091904,v1:172.21.15.187:6834/2445091904] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6836/3223318828,v1:172.21.15.187:6837/3223318828] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6838/3954905595,v1:172.21.15.187:6839/3954905595] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.187:6833/835801654,v1:172.21.15.187:6835/835801654] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6838/2055770764,v1:172.21.15.3:6840/2055770764] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: mds.? [v2:172.21.15.3:6839/3993525271,v1:172.21.15.3:6841/3993525271] up:boot 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T14:23:02.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: fsmap 12 up:standby 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:23:02.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:23:02.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:23:02.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:23:02.045 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:01 smithi003 ceph-mon[97195]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:23:02.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T14:23:02.045 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: Cluster is now healthy 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6833/3478423035,v1:172.21.15.103:6837/3478423035] up:boot 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6835/2489198081,v1:172.21.15.103:6839/2489198081] up:boot 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6832/733137907,v1:172.21.15.103:6836/733137907] up:boot 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.103:6834/2950211492,v1:172.21.15.103:6838/2950211492] up:boot 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6832/2445091904,v1:172.21.15.187:6834/2445091904] up:boot 2023-08-11T14:23:02.046 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6836/3223318828,v1:172.21.15.187:6837/3223318828] up:boot 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6838/3954905595,v1:172.21.15.187:6839/3954905595] up:boot 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.187:6833/835801654,v1:172.21.15.187:6835/835801654] up:boot 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6838/2055770764,v1:172.21.15.3:6840/2055770764] up:boot 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: mds.? [v2:172.21.15.3:6839/3993525271,v1:172.21.15.3:6841/3993525271] up:boot 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: fsmap 12 up:standby 2023-08-11T14:23:02.047 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:23:02.048 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:23:02.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:01 smithi187 ceph-mon[130973]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:23:02.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-08-11T14:23:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-08-11T14:23:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-08-11T14:23:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: Cluster is now healthy 2023-08-11T14:23:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6833/3478423035,v1:172.21.15.103:6837/3478423035] up:boot 2023-08-11T14:23:02.179 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6835/2489198081,v1:172.21.15.103:6839/2489198081] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6832/733137907,v1:172.21.15.103:6836/733137907] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.103:6834/2950211492,v1:172.21.15.103:6838/2950211492] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6832/2445091904,v1:172.21.15.187:6834/2445091904] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6836/3223318828,v1:172.21.15.187:6837/3223318828] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6838/3954905595,v1:172.21.15.187:6839/3954905595] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.187:6833/835801654,v1:172.21.15.187:6835/835801654] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6838/2055770764,v1:172.21.15.3:6840/2055770764] up:boot 2023-08-11T14:23:02.180 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: mds.? [v2:172.21.15.3:6839/3993525271,v1:172.21.15.3:6841/3993525271] up:boot 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/1729317386' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: fsmap 12 up:standby 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-08-11T14:23:02.181 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-08-11T14:23:02.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-08-11T14:23:02.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-08-11T14:23:02.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-08-11T14:23:02.182 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:01 smithi103 ceph-mon[130434]: from='mgr.14150 172.21.15.3:0/3869922586' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-08-11T14:23:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.746+0000 7f72c17fa700 1 -- 172.21.15.3:0/3085569361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72b4099560 con 0x7f72c4100910 2023-08-11T14:23:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.752+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 msgr2=0x7f72b00a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.752+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 0x7f72b00a2410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009c60 tx=0x7f72ac009260 comp rx=0 tx=0).stop 2023-08-11T14:23:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.752+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 msgr2=0x7f72c4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.752+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f72b40098e0 tx=0x7f72b400f690 comp rx=0 tx=0).stop 2023-08-11T14:23:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 shutdown_connections 2023-08-11T14:23:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f72b009ff50 0x7f72b00a2410 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c41024a0 0x7f72c419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72c4100910 0x7f72c4198880 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:02.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 --2- 172.21.15.3:0/3085569361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72c4101b00 0x7f72c4198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 >> 172.21.15.3:0/3085569361 conn(0x7f72c40fbe60 msgr2=0x7f72c40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 shutdown_connections 2023-08-11T14:23:02.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.753+0000 7f72ca1ff700 1 -- 172.21.15.3:0/3085569361 wait complete. 2023-08-11T14:23:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-08-11T14:23:02.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-11T14:23:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 -- 172.21.15.3:0/48808206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 msgr2=0x7fd178108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 --2- 172.21.15.3:0/48808206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178108d50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b0d0 tx=0x7fd17000b3e0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 -- 172.21.15.3:0/48808206 shutdown_connections 2023-08-11T14:23:03.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 --2- 172.21.15.3:0/48808206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178108d50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 --2- 172.21.15.3:0/48808206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178100910 0x7fd178100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 --2- 172.21.15.3:0/48808206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178101b00 0x7fd178101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 -- 172.21.15.3:0/48808206 >> 172.21.15.3:0/48808206 conn(0x7fd1780fbe60 msgr2=0x7fd1780fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 -- 172.21.15.3:0/48808206 shutdown_connections 2023-08-11T14:23:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.998+0000 7fd17cbff700 1 -- 172.21.15.3:0/48808206 wait complete. 2023-08-11T14:23:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.999+0000 7fd17cbff700 1 Processor -- start 2023-08-11T14:23:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:02.999+0000 7fd17cbff700 1 -- start start 2023-08-11T14:23:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178100910 0x7fd1780746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178101b00 0x7fd178072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:03.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780739f0 con 0x7fd1781024a0 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178073b60 con 0x7fd178101b00 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781a07a0 con 0x7fd178100910 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd176d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd176d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44888/0 (socket says 172.21.15.3:44888) 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd176d9d700 1 -- 172.21.15.3:0/3525990006 learned_addr learned my addr 172.21.15.3:0/3525990006 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:23:03.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd17659c700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178100910 0x7fd1780746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd16ffff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178101b00 0x7fd178072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd176d9d700 1 -- 172.21.15.3:0/3525990006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178100910 msgr2=0x7fd1780746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.000+0000 7fd176d9d700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178100910 0x7fd1780746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd176d9d700 1 -- 172.21.15.3:0/3525990006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178101b00 msgr2=0x7fd178072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd176d9d700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178101b00 0x7fd178072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd176d9d700 1 -- 172.21.15.3:0/3525990006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009d00 con 0x7fd1781024a0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd176d9d700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd170008eb0 tx=0x7fd170008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000e040 con 0x7fd1781024a0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1781a0a20 con 0x7fd1781024a0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd17000bdb0 con 0x7fd1781024a0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17001d470 con 0x7fd1781024a0 2023-08-11T14:23:03.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.001+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1781a0ea0 con 0x7fd1781024a0 2023-08-11T14:23:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.002+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17810c180 con 0x7fd1781024a0 2023-08-11T14:23:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.004+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7fd170005020 con 0x7fd1781024a0 2023-08-11T14:23:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.005+0000 7fd16f7fe700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 0x7fd1640a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:03.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.006+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8962+0+0 (secure 0 0 0) 0x7fd1700ccec0 con 0x7fd1781024a0 2023-08-11T14:23:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.006+0000 7fd17659c700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 0x7fd1640a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.006+0000 7fd17659c700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 0x7fd1640a2410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd178101960 tx=0x7fd168009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:03.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.007+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7fd1700969a0 con 0x7fd1781024a0 2023-08-11T14:23:03.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:02 smithi003 ceph-mon[97195]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-08-11T14:23:03.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:02 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:02 smithi187 ceph-mon[130973]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-08-11T14:23:03.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:02 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:03.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:02 smithi103 ceph-mon[130434]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-08-11T14:23:03.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:02 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:03.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.223+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd178107820 con 0x7fd1781024a0 2023-08-11T14:23:03.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.750+0000 7fd16f7fe700 1 -- 172.21.15.3:0/3525990006 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd170099fc0 con 0x7fd1781024a0 2023-08-11T14:23:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 msgr2=0x7fd1640a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 0x7fd1640a2410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd178101960 tx=0x7fd168009260 comp rx=0 tx=0).stop 2023-08-11T14:23:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 msgr2=0x7fd178073250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd170008eb0 tx=0x7fd170008ee0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 shutdown_connections 2023-08-11T14:23:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1781024a0 0x7fd178073250 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd178100910 0x7fd1780746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7fd16409ff50 0x7fd1640a2410 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 --2- 172.21.15.3:0/3525990006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd178101b00 0x7fd178072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.755+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 >> 172.21.15.3:0/3525990006 conn(0x7fd1780fbe60 msgr2=0x7fd1780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.757+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 shutdown_connections 2023-08-11T14:23:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.757+0000 7fd17cbff700 1 -- 172.21.15.3:0/3525990006 wait complete. 2023-08-11T14:23:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-08-11T14:23:03.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-08-11T14:23:04.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 -- 172.21.15.3:0/1141278345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc101b20 msgr2=0x7f75dc101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:04.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 --2- 172.21.15.3:0/1141278345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc101b20 0x7f75dc101f80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f75d00099c0 tx=0x7f75d0009cd0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 -- 172.21.15.3:0/1141278345 shutdown_connections 2023-08-11T14:23:04.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 --2- 172.21.15.3:0/1141278345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 0x7f75dc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 --2- 172.21.15.3:0/1141278345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc100930 0x7f75dc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 --2- 172.21.15.3:0/1141278345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc101b20 0x7f75dc101f80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 -- 172.21.15.3:0/1141278345 >> 172.21.15.3:0/1141278345 conn(0x7f75dc0fbea0 msgr2=0x7f75dc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 -- 172.21.15.3:0/1141278345 shutdown_connections 2023-08-11T14:23:04.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.998+0000 7f75e19ff700 1 -- 172.21.15.3:0/1141278345 wait complete. 2023-08-11T14:23:04.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:03.999+0000 7f75e19ff700 1 Processor -- start 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 -- start start 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc101b20 0x7f75dc198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 0x7f75dc19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a03d0 con 0x7f75dc100930 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a0510 con 0x7f75dc101b20 2023-08-11T14:23:04.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75e19ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc1a06a0 con 0x7f75dc1024c0 2023-08-11T14:23:04.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75daffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:04.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75da7fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc101b20 0x7f75dc198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:04.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75daffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44904/0 (socket says 172.21.15.3:44904) 2023-08-11T14:23:04.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75daffd700 1 -- 172.21.15.3:0/593308384 learned_addr learned my addr 172.21.15.3:0/593308384 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.000+0000 7f75db7fe700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 0x7f75dc19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 -- 172.21.15.3:0/593308384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 msgr2=0x7f75dc19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 0x7f75dc19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 -- 172.21.15.3:0/593308384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc101b20 msgr2=0x7f75dc198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc101b20 0x7f75dc198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 -- 172.21.15.3:0/593308384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d00095f0 con 0x7f75dc100930 2023-08-11T14:23:04.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75daffd700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e970 tx=0x7f75cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:04.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc009790 con 0x7f75dc100930 2023-08-11T14:23:04.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75cc00c7e0 con 0x7f75dc100930 2023-08-11T14:23:04.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc1a0980 con 0x7f75dc100930 2023-08-11T14:23:04.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003c70 con 0x7f75dc100930 2023-08-11T14:23:04.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.001+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1a0e90 con 0x7f75dc100930 2023-08-11T14:23:04.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.002+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc10c190 con 0x7f75dc100930 2023-08-11T14:23:04.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.004+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) v1 ==== 134670+0+0 (secure 0 0 0) 0x7f75cc00cc50 con 0x7f75dc100930 2023-08-11T14:23:04.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.006+0000 7f75e09fd700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 0x7f75c40a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-11T14:23:04.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.006+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f75cc014070 con 0x7f75dc100930 2023-08-11T14:23:04.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.006+0000 7f75da7fc700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 0x7f75c40a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-11T14:23:04.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.007+0000 7f75da7fc700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 0x7f75c40a2410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b580 tx=0x7f75d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-11T14:23:04.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.007+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179220 (secure 0 0 0) 0x7f75cc0952e0 con 0x7f75dc100930 2023-08-11T14:23:04.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:03 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:04.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:03 smithi003 ceph-mon[97195]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T14:23:04.041 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:03 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:04.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:03 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:04.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:03 smithi187 ceph-mon[130973]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T14:23:04.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:03 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:04.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:03 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3085569361' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:04.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:03 smithi103 ceph-mon[130434]: osdmap e93: 12 total, 12 up, 12 in 2023-08-11T14:23:04.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:03 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:04.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.229+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75dc10c370 con 0x7f75dc100930 2023-08-11T14:23:04.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.750+0000 7f75e09fd700 1 -- 172.21.15.3:0/593308384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) v1 ==== 178+0+0 (secure 0 0 0) 0x7f75cc098900 con 0x7f75dc100930 2023-08-11T14:23:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 msgr2=0x7f75c40a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 0x7f75c40a2410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75d000b580 tx=0x7f75d0005fb0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 msgr2=0x7f75dc198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e970 tx=0x7f75cc00ec80 comp rx=0 tx=0).stop 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 shutdown_connections 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75dc1024c0 0x7f75dc19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75dc100930 0x7f75dc198850 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.3:6800/2363526909,v1:172.21.15.3:6801/2363526909] conn(0x7f75c409ff50 0x7f75c40a2410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 --2- 172.21.15.3:0/593308384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75dc101b20 0x7f75dc198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-11T14:23:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 >> 172.21.15.3:0/593308384 conn(0x7f75dc0fbea0 msgr2=0x7f75dc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-11T14:23:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 shutdown_connections 2023-08-11T14:23:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-08-11T14:23:04.755+0000 7f75e19ff700 1 -- 172.21.15.3:0/593308384 wait complete. 2023-08-11T14:23:04.759 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' removed 2023-08-11T14:23:04.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:23:04.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:23:05.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:04 smithi003 ceph-mon[97195]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:23:05.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:04 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:05.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:04 smithi003 ceph-mon[97195]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T14:23:05.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:04 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:05.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:04 smithi187 ceph-mon[130973]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:23:05.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:04 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:05.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:04 smithi187 ceph-mon[130973]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T14:23:05.049 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:04 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:05.050 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:04 smithi103 ceph-mon[130434]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail 2023-08-11T14:23:05.050 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:04 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/3525990006' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:05.050 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:04 smithi103 ceph-mon[130434]: osdmap e94: 12 total, 12 up, 12 in 2023-08-11T14:23:05.050 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:04 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-08-11T14:23:05.077 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:23:05.179 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-08-11T14:23:05.180 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T14:23:05.292 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T14:23:05.360 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-08-11T14:23:05.470 INFO:tasks.cephadm:Stopping all daemons... 2023-08-11T14:23:05.470 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-08-11T14:23:05.471 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a 2023-08-11T14:23:06.043 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:05 smithi003 systemd[1]: Stopping Ceph mon.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:06.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:05 smithi003 ceph-mon[97195]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:06.044 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:05 smithi003 ceph-mon[97195]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T14:23:06.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:05 smithi103 ceph-mon[130434]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:06.178 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:05 smithi103 ceph-mon[130434]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T14:23:06.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:05 smithi187 ceph-mon[130973]: from='client.? 172.21.15.3:0/593308384' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-08-11T14:23:06.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:05 smithi187 ceph-mon[130973]: osdmap e95: 12 total, 12 up, 12 in 2023-08-11T14:23:06.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:06 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:23:06.408+0000 7f7a44ac1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:06.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:06 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a[97171]: 2023-08-11T14:23:06.408+0000 7f7a44ac1700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-08-11T14:23:07.127 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 11 14:23:06 smithi003 bash[160425]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-a 2023-08-11T14:23:07.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.a.service' 2023-08-11T14:23:07.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:07.961 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-08-11T14:23:07.962 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-08-11T14:23:07.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.b 2023-08-11T14:23:08.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:08 smithi103 systemd[1]: Stopping Ceph mon.b for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:08.834 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:08 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-b[130410]: 2023-08-11T14:23:08.541+0000 7f24f585a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:08.834 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:08 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-b[130410]: 2023-08-11T14:23:08.541+0000 7f24f585a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-08-11T14:23:11.488 INFO:journalctl@ceph.mon.b.smithi103.stdout:Aug 11 14:23:11 smithi103 bash[161723]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-b 2023-08-11T14:23:11.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.b.service' 2023-08-11T14:23:11.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:11.560 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-08-11T14:23:11.561 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-08-11T14:23:11.561 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.c 2023-08-11T14:23:12.044 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:11 smithi187 systemd[1]: Stopping Ceph mon.c for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:13.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-c[130949]: 2023-08-11T14:23:13.712+0000 7f56ff160700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:13.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-c[130949]: 2023-08-11T14:23:13.712+0000 7f56ff160700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-08-11T14:23:14.294 INFO:journalctl@ceph.mon.c.smithi187.stdout:Aug 11 14:23:13 smithi187 bash[163639]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mon-c 2023-08-11T14:23:14.355 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mon.c.service' 2023-08-11T14:23:14.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:14.405 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-08-11T14:23:14.405 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-08-11T14:23:14.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x 2023-08-11T14:23:14.791 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:23:14 smithi003 systemd[1]: Stopping Ceph mgr.x for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:16.611 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:23:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:23:16.336+0000 7ff210431700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:16.631 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:23:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x[97441]: 2023-08-11T14:23:16.336+0000 7ff210431700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-08-11T14:23:24.026 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 11 14:23:23 smithi003 bash[160560]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-x 2023-08-11T14:23:24.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.x.service' 2023-08-11T14:23:24.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:24.149 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-08-11T14:23:24.149 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-08-11T14:23:24.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.y 2023-08-11T14:23:24.568 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 14:23:24 smithi103 systemd[1]: Stopping Ceph mgr.y for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:24.861 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 14:23:24 smithi103 bash[161864]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-y 2023-08-11T14:23:25.040 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Aug 11 14:23:24 smithi103 systemd[1]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-08-11T14:23:25.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.y.service' 2023-08-11T14:23:25.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:25.082 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-08-11T14:23:25.082 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-08-11T14:23:25.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.z 2023-08-11T14:23:25.506 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 14:23:25 smithi187 systemd[1]: Stopping Ceph mgr.z for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:25.789 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 14:23:25 smithi187 bash[163798]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mgr-z 2023-08-11T14:23:25.970 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Aug 11 14:23:25 smithi187 systemd[1]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-08-11T14:23:25.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mgr.z.service' 2023-08-11T14:23:26.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:26.012 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-08-11T14:23:26.012 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-08-11T14:23:26.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.0 2023-08-11T14:23:26.537 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 14:23:26 smithi003 systemd[1]: Stopping Ceph osd.0 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:26.791 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 14:23:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T14:23:26.535+0000 7fdf97ff2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:26.791 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 14:23:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T14:23:26.535+0000 7fdf97ff2700 -1 osd.0 95 *** Got signal Terminated *** 2023-08-11T14:23:26.791 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 14:23:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0[110191]: 2023-08-11T14:23:26.535+0000 7fdf97ff2700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:23:32.726 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 11 14:23:32 smithi003 bash[160707]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-0 2023-08-11T14:23:34.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.0.service' 2023-08-11T14:23:34.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:34.862 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-08-11T14:23:34.863 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-08-11T14:23:34.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.1 2023-08-11T14:23:35.178 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 14:23:34 smithi103 systemd[1]: Stopping Ceph osd.1 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:35.678 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 14:23:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T14:23:35.217+0000 7fafea837700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:35.679 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 14:23:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T14:23:35.217+0000 7fafea837700 -1 osd.1 95 *** Got signal Terminated *** 2023-08-11T14:23:35.679 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 14:23:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1[135500]: 2023-08-11T14:23:35.217+0000 7fafea837700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:23:41.179 INFO:journalctl@ceph.osd.1.smithi103.stdout:Aug 11 14:23:40 smithi103 bash[162013]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-1 2023-08-11T14:23:42.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.1.service' 2023-08-11T14:23:42.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:42.873 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-08-11T14:23:42.874 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-08-11T14:23:42.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.2 2023-08-11T14:23:43.294 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 14:23:42 smithi187 systemd[1]: Stopping Ceph osd.2 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:43.794 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 14:23:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T14:23:43.326+0000 7fb6b2f8a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:43.795 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 14:23:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T14:23:43.326+0000 7fb6b2f8a700 -1 osd.2 95 *** Got signal Terminated *** 2023-08-11T14:23:43.795 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 14:23:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2[136750]: 2023-08-11T14:23:43.326+0000 7fb6b2f8a700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:23:49.208 INFO:journalctl@ceph.osd.2.smithi187.stdout:Aug 11 14:23:49 smithi187 bash[163945]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-2 2023-08-11T14:23:50.871 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.2.service' 2023-08-11T14:23:50.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:50.914 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-08-11T14:23:50.914 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-08-11T14:23:50.915 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.3 2023-08-11T14:23:51.291 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:51 smithi003 systemd[1]: Stopping Ceph osd.3 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:23:51.791 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:51 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:51.440+0000 7f4752342700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:23:51.791 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:51 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:51.440+0000 7f4752342700 -1 osd.3 95 *** Got signal Terminated *** 2023-08-11T14:23:51.791 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:51 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:51.440+0000 7f4752342700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:23:53.291 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:52 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:52.833+0000 7f474b131700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.961510+0000 front 2023-08-11T14:23:27.961549+0000 (oldest deadline 2023-08-11T14:23:52.661681+0000) 2023-08-11T14:23:53.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:53.062+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:53.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:53 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:53.220+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:54.291 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:53 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:53.841+0000 7f474b131700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.961510+0000 front 2023-08-11T14:23:27.961549+0000 (oldest deadline 2023-08-11T14:23:52.661681+0000) 2023-08-11T14:23:54.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:54.096+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:54.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:54.139+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:54.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:54 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:54.206+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:55.291 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:54 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:54.864+0000 7f474b131700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.961510+0000 front 2023-08-11T14:23:27.961549+0000 (oldest deadline 2023-08-11T14:23:52.661681+0000) 2023-08-11T14:23:55.294 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:23:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:23:55.034+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:23:55.294 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:55.130+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:55.294 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:55.135+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:55.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:55 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:55.241+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:56.291 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:55 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3[115638]: 2023-08-11T14:23:55.837+0000 7f474b131700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.961510+0000 front 2023-08-11T14:23:27.961549+0000 (oldest deadline 2023-08-11T14:23:52.661681+0000) 2023-08-11T14:23:56.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:23:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:23:56.067+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:23:56.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:56.088+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:56.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:56.161+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:56.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:56 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:56.255+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:57.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:23:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:23:57.082+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:23:57.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:57.076+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:57.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:57.207+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:57.641 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 11 14:23:57 smithi003 bash[160978]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-3 2023-08-11T14:23:57.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:57 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:57.287+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:58.537 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:23:58 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:23:58.189+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:23:58.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:23:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:23:58.112+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:23:58.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:58.079+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:58.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:58.205+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:58.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:58 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:58.323+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:59.291 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:23:59 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:23:59.197+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:23:59.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.3.service' 2023-08-11T14:23:59.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:23:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:23:59.157+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:23:59.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:23:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:23:59.126+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:23:59.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:23:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:23:59.232+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:23:59.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:23:59.585 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-08-11T14:23:59.585 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-08-11T14:23:59.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.4 2023-08-11T14:23:59.616 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:23:59 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:23:59.344+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:23:59.928 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:23:59 smithi103 systemd[1]: Stopping Ceph osd.4 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:00.303 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:23:59 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:23:59.995+0000 7f85a3a25700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:00.303 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:23:59 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:23:59.995+0000 7f85a3a25700 -1 osd.4 95 *** Got signal Terminated *** 2023-08-11T14:24:00.303 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:23:59 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:23:59.995+0000 7f85a3a25700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:00.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:00 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:00.227+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:00.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:00.132+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:00.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:00.163+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:00.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:00.210+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:00.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:00 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:00.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:01.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:01 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:01.270+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:01.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:01 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:01.160+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:01.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:01 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:01.128+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:01.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:01 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:01.227+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:01.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:01 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:01.265+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:02.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:01 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:01.718+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:02.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:02 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:02.245+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:02.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:02 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:02.245+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:02.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:02 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:02.141+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:02.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:02 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:02.212+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:02.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:02 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:02.083+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:02.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:02 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:02.256+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:03.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:02 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:02.691+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:03.178 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:24:02 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:24:02.894+0000 7f859e017700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.324893+0000 front 2023-08-11T14:23:37.324916+0000 (oldest deadline 2023-08-11T14:24:02.025010+0000) 2023-08-11T14:24:03.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:03.248+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:03.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:03.248+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:03.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:03 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:03.099+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:03.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:03 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:03.253+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:03.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:03 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:03.073+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:03.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:03 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:03.073+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:03.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:03 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:03.273+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:04.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:03.656+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:04.178 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:24:03 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:24:03.919+0000 7f859e017700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.324893+0000 front 2023-08-11T14:23:37.324916+0000 (oldest deadline 2023-08-11T14:24:02.025010+0000) 2023-08-11T14:24:04.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:04 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:04.213+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:04.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:04 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:04.213+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:04.543 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:04 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:04.132+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:04.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:04 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:04.067+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:04.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:04 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:04.067+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:04.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:04 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:04.248+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:04.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:04 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:04.280+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:05.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:04 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:04.657+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:05.178 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:24:04 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4[140935]: 2023-08-11T14:24:04.897+0000 7f859e017700 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.324893+0000 front 2023-08-11T14:23:37.324916+0000 (oldest deadline 2023-08-11T14:24:02.025010+0000) 2023-08-11T14:24:05.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:05 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:05.257+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:05.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:05 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:05.257+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:05.543 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:05 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:05.161+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:05.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:05 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:05.224+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:05.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:05 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:05.094+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:05.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:05 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:05.094+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:05.562 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:05 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:05.262+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:05.824 INFO:journalctl@ceph.osd.4.smithi103.stdout:Aug 11 14:24:05 smithi103 bash[162291]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-4 2023-08-11T14:24:06.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:05 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:05.651+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:06.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:06 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:06.210+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:06.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:06 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:06.210+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:06.543 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:06 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:06.114+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:06.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:06 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:06.114+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:06.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:06 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:06.257+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:06.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:06 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:06.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:06.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:06 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:06.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:06.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:06 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:06.215+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:06.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:06 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:06.215+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:07.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:06 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:06.626+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:07.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.4.service' 2023-08-11T14:24:07.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:07.165 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-08-11T14:24:07.165 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-08-11T14:24:07.165 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.5 2023-08-11T14:24:07.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:07 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:07.309+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:07.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:07 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:07.221+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:07.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:07 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:07.221+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:07.517 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:07.267+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:07.517 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:07.123+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:07.517 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:07.123+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:07.518 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:07.130+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:07.518 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:07.130+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:07.518 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 systemd[1]: Stopping Ceph osd.5 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:07.518 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:07.515+0000 7f3100041700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:07.518 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:07.515+0000 7f3100041700 -1 osd.5 95 *** Got signal Terminated *** 2023-08-11T14:24:07.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:07 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:07.173+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:07.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:07 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:07.173+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:07.793 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:07 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:07.515+0000 7f3100041700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:08.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:07 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:07.637+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:08.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:08 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:08.169+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:08.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:08 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:08.169+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:08.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:08.293+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:08.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:08.132+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:08.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:08.132+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:08.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:08.148+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:08.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:08 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:08.148+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:08.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:08 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:08.275+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:08.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:08 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:08.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:08.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:08 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:08.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:09.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:08 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:08.663+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:09.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:09.216+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:09.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:09.216+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:09.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:09.216+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:09.543 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:09.104+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:09.544 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:09.104+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:09.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:09.193+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:09.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:09.193+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:09.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:09.321+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:09.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:09 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:09.321+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:09.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:09.271+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:09.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:09.273+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:09.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:09 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:09.273+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:10.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:09 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:09.624+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:10.370 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:10.144+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:10.370 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:10.144+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:10.370 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:10.057+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:10.371 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:10.057+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:10.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:10.254+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:10.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:10.254+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:10.542 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:10.254+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:10.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:10.321+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:10.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:10.286+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:10.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:10 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:10.286+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:10.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:10.369+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:10.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:10 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:10.369+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:11.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:10 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:10.633+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:11.379 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:11.161+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:11.379 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:11.161+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:11.379 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:11.074+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:11.379 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:11.074+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:11.380 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:11.074+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.428054+0000 front 2023-08-11T14:23:45.428085+0000 (oldest deadline 2023-08-11T14:24:10.728227+0000) 2023-08-11T14:24:11.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:11.237+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:11.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:11.237+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:11.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:11.237+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:11.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:11 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:11.280+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:11.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:11 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:11.296+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:11.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:11 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:11.296+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:11.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:11.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:11.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:11 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:11.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:12.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:11 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:11.661+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:12.425 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:12.138+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:12.425 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:12.138+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:12.425 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:12.070+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:29.526376+0000 front 2023-08-11T14:23:29.526282+0000 (oldest deadline 2023-08-11T14:23:54.826379+0000) 2023-08-11T14:24:12.425 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:12.070+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.527714+0000 front 2023-08-11T14:23:39.527629+0000 (oldest deadline 2023-08-11T14:24:05.427697+0000) 2023-08-11T14:24:12.426 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5[142104]: 2023-08-11T14:24:12.070+0000 7f30f9631700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.428054+0000 front 2023-08-11T14:23:45.428085+0000 (oldest deadline 2023-08-11T14:24:10.728227+0000) 2023-08-11T14:24:12.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:12.279+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:12.541 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:12.279+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:12.542 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:12.279+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:12.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:12.252+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:12.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:12.252+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:12.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:12.328+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:12.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:12.328+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:12.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:12 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:12.328+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:12.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:12.423+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:12.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:12 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:12.423+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:13.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:12 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:12.620+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:13.354 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:13.101+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:13.354 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:13.101+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:13.604 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:13.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:13.604 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:13 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:13.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:13.605 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:13 smithi187 bash[164208]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-5 2023-08-11T14:24:13.614 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:13.309+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:13.615 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:13.309+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:13.615 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:13.309+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:13.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:13.274+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:13.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:13.274+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:13.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:13.292+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:13.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:13.292+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:13.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:13 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:13.292+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:14.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:13.614+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:14.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:13.614+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:14.380 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:14.100+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:14.381 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:14.100+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:14.381 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:14.100+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:14.641 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:14.295+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:14.642 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:14.295+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:14.642 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:14.295+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:14.643 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:14.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:14.643 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:14 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:14.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:14.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:14.257+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:14.679 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:14.257+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:14.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:14.249+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:14.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:14.249+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:14.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:14 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:14.249+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:14.868 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.5.service' 2023-08-11T14:24:14.895 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:14 smithi187 systemd[1]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.5.service: Succeeded. 2023-08-11T14:24:14.895 INFO:journalctl@ceph.osd.5.smithi187.stdout:Aug 11 14:24:14 smithi187 systemd[1]: Stopped Ceph osd.5 for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:24:14.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:14.909 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-08-11T14:24:14.910 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-08-11T14:24:14.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.6 2023-08-11T14:24:14.979 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:14.641+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:14.979 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:14 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:14.641+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:15.291 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 systemd[1]: Stopping Ceph osd.6 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:15.294 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:15 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:15.106+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:15.294 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:15 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:15.106+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:15.294 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:15 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:15.106+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:15.639 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:15.290+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:15.639 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:15.290+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:15.640 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:15.290+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:15.640 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:15.428+0000 7f1dff751700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:15.640 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:15.428+0000 7f1dff751700 -1 osd.6 95 *** Got signal Terminated *** 2023-08-11T14:24:15.640 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:15.428+0000 7f1dff751700 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:15.640 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:15.638+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:15.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:15.225+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:15.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:15.225+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:15.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:15.241+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:15.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:15.241+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:15.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:15 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:15.241+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:15.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:15 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:15.353+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:15.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:15 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:15.353+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:16.402 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:16.108+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:16.402 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:16.108+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:16.402 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:16.108+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:16.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:16.192+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:16.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:16.192+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:16.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:16.223+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:16.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:16.223+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:16.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:16 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:16.223+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:16.688 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:15 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:15.638+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:16.689 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:16.306+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:16.689 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:16.306+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:16.689 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:16.306+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:16.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:16.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:16.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:16 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:16.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:17.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:16.687+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:17.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:16 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:16.687+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:17.407 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:17 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:17.082+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:17.408 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:17 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:17.082+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:17.408 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:17 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:17.082+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:17.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:17 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:17.149+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:17.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:17 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:17.149+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:17.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:17 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:17.270+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:17.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:17 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:17.270+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:17.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:17 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:17.270+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:17.674 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:17.336+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:17.674 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:17.336+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:17.675 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:17.336+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:17.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:17 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:17.407+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:17.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:17 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:17.407+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:18.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:17.673+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:18.042 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:17 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:17.673+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:18.403 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:18.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:18.403 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:18.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:18.403 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:18.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:18.403 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:18.131+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:18.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:18.108+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:18.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:18.108+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:18.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:18.315+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:18.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:18.315+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:18.430 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:18.315+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:18.430 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:18 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:18.315+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:18.633 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:18.368+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:18.633 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:18.368+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:18.634 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:18.368+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:18.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:18.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:18.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:18.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:18.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:18 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:18.401+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:19.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:18.632+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:19.041 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:18.632+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:19.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:19.109+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:19.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:19.109+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:19.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:19.109+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:19.291+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:19.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:19.291+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:19.430 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:19.291+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:19.430 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:19 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:19.291+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:19.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:19.180+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:19.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:19.180+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:19.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:19.181+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:19.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:19.181+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:19.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:19.386+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:19.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:19.386+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:19.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:19 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:19.386+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:19.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:19 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:19.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:19.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:19 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:19.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:19.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:19 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:19.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:19.792 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:19 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:19.638+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.190795+0000 front 2023-08-11T14:23:37.190635+0000 (oldest deadline 2023-08-11T14:24:01.290513+0000) 2023-08-11T14:24:19.792 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:19 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6[121099]: 2023-08-11T14:24:19.638+0000 7f1df9d43700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:47.093077+0000 front 2023-08-11T14:23:47.093002+0000 (oldest deadline 2023-08-11T14:24:12.992973+0000) 2023-08-11T14:24:20.424 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:20.169+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:20.424 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:20.169+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:20.425 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:20.169+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:20.425 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:20.169+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:20.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:20.079+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:20.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:20.079+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:20.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:20.079+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:20.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:20.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:20.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:20.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:20.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:20.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:20.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:20 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:20.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:20.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:20 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:20.371+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:20.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:20 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:20.371+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:20.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:20 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:20.371+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:20.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:20.423+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:20.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:20.423+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:20.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:20 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:20.423+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:21.312 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:21.033+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:21.312 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:21.033+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:21.313 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:21.033+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:21.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:21.443+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:21.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:21.443+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:21.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:21.443+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:21.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:21.197+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:21.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:21.197+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:21.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:21.197+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:21.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:21 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:21.197+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:21.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:21.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:21.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:21.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:21.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:21.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:21.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:21.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:21.791 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 11 14:24:21 smithi003 bash[161247]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-6 2023-08-11T14:24:21.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:21 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:21.362+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:21.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:21 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:21.362+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:21.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:21 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:21.362+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:22.428 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:22.075+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:22.075+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:22.075+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:22.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:22.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:22.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:22.429 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:22 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:22.302+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:22.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:22.427+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:22.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:22.427+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:22.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:22.427+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:22.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:22.185+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:22.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:22.185+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:22.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:22.185+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:22.546 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:22 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:22.186+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:22.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:22 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:22.393+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:22.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:22 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:22.393+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:22.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:22 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:22.393+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:22.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:22 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:22.393+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:23.312 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.026+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:23.313 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.027+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:23.313 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.027+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:23.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.6.service' 2023-08-11T14:24:23.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:23.496 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-08-11T14:24:23.497 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-08-11T14:24:23.497 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.7 2023-08-11T14:24:23.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:23.424+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:23.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:23.424+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:23.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:23.424+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:23.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:23.198+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:23.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:23.198+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:23.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:23.198+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:23.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:23 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:23.198+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:23.678 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 systemd[1]: Stopping Ceph osd.7 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:23.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:23.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:23.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:23.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:23.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:23.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:23.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:23.311+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:23.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:23 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:23.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:23.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:23 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:23.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:23.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:23 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:23.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:23.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:23 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:23.406+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:24.178 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.843+0000 7f561c03f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:24.178 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.843+0000 7f561c03f700 -1 osd.7 95 *** Got signal Terminated *** 2023-08-11T14:24:24.178 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.843+0000 7f561c03f700 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:24.179 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.983+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:24.179 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.983+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:24.179 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:23 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:23.983+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:24.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:24.397+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:24.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:24.397+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:24.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:24.397+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:24.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:24.160+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:24.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:24.160+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:24.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:24.160+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:24.546 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:24 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:24.160+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:24.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:24 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:24.310+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:24.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:24 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:24.310+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:24.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:24 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:24.310+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:24.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:24 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:24.310+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:24.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:24 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:24.359+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:24.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:24 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:24.359+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:24.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:24 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:24.359+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:24.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:24 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:24.359+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:25.298 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:25.027+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:25.299 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:25.027+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:25.299 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:25.027+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:25.386 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:25.110+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:25.387 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:25.110+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:25.387 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:25.110+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:25.388 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:25.110+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:25.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:25.297+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:25.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:25.297+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:25.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:25.297+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:25.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:25.297+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:25.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:25.340+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:25.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:25.340+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:25.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:25.340+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:25.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:25 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:25.340+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:25.793 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:25.385+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:25.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:25.385+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:25.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:25 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:25.385+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:26.345 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:26.059+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:26.345 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:26.059+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:26.346 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:26.059+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:26.374 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:26.064+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:26.374 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:26.064+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:26.375 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:26.064+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:26.375 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:26.064+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:26.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:26.344+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:26.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:26.344+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:26.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:26.344+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:26.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:26 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:26.344+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:26.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:26.356+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:26.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:26.356+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:26.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:26.356+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:26.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:26 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:26.356+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:26.793 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:26.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:26.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:26.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:26.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:26 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:26.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:27.391 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.011+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:27.392 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.011+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:27.392 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.011+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:27.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:27.335+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:27.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:27.335+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:27.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:27.335+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:27.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:27.086+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:27.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:27.086+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:27.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:27.086+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:27.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:27 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:27.086+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:27.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:27.390+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:27.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:27.390+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:27.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:27.390+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:27.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:27.390+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:27.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:27.390+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:27.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:27.350+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:27.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:27.350+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:27.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:27.350+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:27.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:27 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:27.350+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:28.379 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.969+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:39.317231+0000 front 2023-08-11T14:23:39.317435+0000 (oldest deadline 2023-08-11T14:24:06.317972+0000) 2023-08-11T14:24:28.379 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.969+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:46.318698+0000 front 2023-08-11T14:23:46.318572+0000 (oldest deadline 2023-08-11T14:24:12.218618+0000) 2023-08-11T14:24:28.379 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:27 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7[146333]: 2023-08-11T14:24:27.969+0000 7f5618635700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:52.219406+0000 front 2023-08-11T14:23:52.219546+0000 (oldest deadline 2023-08-11T14:24:18.119384+0000) 2023-08-11T14:24:28.383 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:28.112+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:28.383 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:28.112+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:28.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:28.112+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:28.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:28.112+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:28.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:28 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:28.377+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:28.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:28 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:28.377+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:28.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:28 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:28.377+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:28.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:28 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:28.377+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:28.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:28 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:28.377+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:28.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:28.386+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:28.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:28.386+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:28.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:28.386+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:28.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:28.386+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:28.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:28 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:28.386+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:28.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:28.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:28.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:28.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:28.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:28.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:28.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:28 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:28.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:29.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:29.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:29.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:29.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:29.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:29.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:29.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:29.378+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:29.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:29.132+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:29.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:29.132+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:29.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:29.132+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:29.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:29 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:29.132+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:29.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:29.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:29.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:29.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:29.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:29.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:29.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:29.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:29.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:29.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:29.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:29.352+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:29.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:29.352+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:29.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:29.352+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:29.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:29.352+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:29.793 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:29 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:29.352+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:30.178 INFO:journalctl@ceph.osd.7.smithi103.stdout:Aug 11 14:24:29 smithi103 bash[162553]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-7 2023-08-11T14:24:30.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:30.135+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:30.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:30.135+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:30.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:30.135+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:30.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:30.135+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:30.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:30.363+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:30.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:30.363+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:30.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:30.363+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:30.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:30 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:30.363+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:30.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:30 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:30.381+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:30.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:30 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:30.381+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:30.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:30 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:30.381+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:30.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:30 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:30.381+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:30.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:30 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:30.381+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:30.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:30.365+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:30.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:30.365+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:30.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:30.365+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:30.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:30.365+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:30.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:30 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:30.365+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:31.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.7.service' 2023-08-11T14:24:31.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:31.233 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-08-11T14:24:31.233 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-08-11T14:24:31.234 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.8 2023-08-11T14:24:31.383 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.098+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:31.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.098+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:31.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.098+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:31.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.098+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:31.384 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 systemd[1]: Stopping Ceph osd.8 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:31.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:31 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:31.399+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:31.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:31 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:31.399+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:31.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:31 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:31.399+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:31.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:31 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:31.399+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:31.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:31 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:31.399+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:31.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:31.405+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:31.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:31.405+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:31.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:31.405+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:31.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:31.405+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:31.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:31 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:31.405+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:31.794 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.587+0000 7f06404ba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:31.794 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.587+0000 7f06404ba700 -1 osd.8 95 *** Got signal Terminated *** 2023-08-11T14:24:31.794 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:31.587+0000 7f06404ba700 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:31.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:31.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:31.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:31.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:31.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:31.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:31.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:31 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:31.382+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:32.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:32.146+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:32.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:32.146+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:32.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:32.146+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:32.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:32.146+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:32.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:32.376+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:32.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:32.376+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:32.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:32.376+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:32.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:32 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:32.376+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:32.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:32.432+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:32.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:32.433+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:32.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:32.433+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:32.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:32.433+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:32.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:32 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:32.433+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:32.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:32 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:32.444+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:32.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:32 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:32.444+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:32.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:32 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:32.444+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:32.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:32 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:32.444+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:32.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:32 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:32.444+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:33.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:33.348+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:33.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:33.348+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:33.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:33.348+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:33.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:33.348+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:33.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:33.145+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:33.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:33.145+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:33.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:33.145+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:33.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:33 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:33.145+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:33.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:33.403+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:33.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:33.403+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:33.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:33.403+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:33.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:33.403+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:33.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:33.403+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:33.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:33.407+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:33.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:33.407+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:33.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:33.407+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:33.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:33.407+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:33.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:33 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:33.407+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:34.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:34.194+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:34.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:34.194+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:34.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:34.194+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:34.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:34.194+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:34.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:34.356+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:34.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:34.356+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:34.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:34.356+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:34.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:34 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:34.356+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:34.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:34 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:34.415+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:34.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:34 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:34.415+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:34.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:34 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:34.415+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:34.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:34 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:34.415+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:34.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:34 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:34.415+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:34.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:34.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:34.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:34.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:34.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:34.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:34 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:34.400+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:35.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:35.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:35.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:35.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:35.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:35.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:35.373+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:35.196+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:35.196+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:35.196+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:35.196+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:35.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:35 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:35.196+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:11.973219+0000 front 2023-08-11T14:24:11.973187+0000 (oldest deadline 2023-08-11T14:24:34.873251+0000) 2023-08-11T14:24:35.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:35.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:35.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:35.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:35.793 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:35.793 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:35 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:35.428+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:35.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:35.451+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:35.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:35.451+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:35.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:35.451+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:35.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:35.451+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:35.930 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:35 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:35.451+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:36.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:36.211+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.265679+0000 front 2023-08-11T14:23:31.265905+0000 (oldest deadline 2023-08-11T14:23:54.066243+0000) 2023-08-11T14:24:36.544 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:36.211+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.567447+0000 front 2023-08-11T14:23:37.567541+0000 (oldest deadline 2023-08-11T14:24:02.267526+0000) 2023-08-11T14:24:36.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:36.211+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:48.069174+0000 front 2023-08-11T14:23:48.069301+0000 (oldest deadline 2023-08-11T14:24:13.969435+0000) 2023-08-11T14:24:36.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:36.211+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.970203+0000 front 2023-08-11T14:23:53.970278+0000 (oldest deadline 2023-08-11T14:24:17.370790+0000) 2023-08-11T14:24:36.545 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8[147511]: 2023-08-11T14:24:36.211+0000 7f063aaac700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:11.973219+0000 front 2023-08-11T14:24:11.973187+0000 (oldest deadline 2023-08-11T14:24:34.873251+0000) 2023-08-11T14:24:36.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:36.359+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:36.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:36.359+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:36.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:36.359+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:36.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:36.359+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:36.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:36 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:36.359+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:36.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.422+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:36.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.422+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:36.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.422+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:36.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.422+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:36.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.423+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:36.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:36 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:36.423+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:36.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:36 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:36.457+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:36.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:36 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:36.457+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:36.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:36 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:36.457+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:36.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:36 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:36.457+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:36.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:36 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:36.457+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:37.553 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:37.383+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:37.553 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:37.383+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:37.553 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:37.383+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:37.554 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:37.383+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:37.554 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:37.383+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:37.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.410+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:37.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.411+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:37.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.411+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:37.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.411+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:37.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.411+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:37.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:37 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:37.411+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:37.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:37.928 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:37.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:37.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:37.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:37.929 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:37 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:37.443+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:38.044 INFO:journalctl@ceph.osd.8.smithi187.stdout:Aug 11 14:24:37 smithi187 bash[164479]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-8 2023-08-11T14:24:38.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:38.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:38.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:38.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:38.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:38.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:38 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:38.413+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:38.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.425+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:38.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.425+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:38.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.425+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:38.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.425+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:38.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.426+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:38.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:38.426+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:38.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:38 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:38.412+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:38.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:38 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:38.412+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:38.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:38 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:38.412+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:38.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:38 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:38.412+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:38.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:38 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:38.412+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:38.833 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.8.service' 2023-08-11T14:24:38.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:38.874 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-08-11T14:24:38.875 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-08-11T14:24:38.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.9 2023-08-11T14:24:39.291 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:38 smithi003 systemd[1]: Stopping Ceph osd.9 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:39.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:39.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:39.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:39.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:39.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:39.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:39 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:39.409+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:39.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.392+0000 7fdc032fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:39.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.392+0000 7fdc032fd700 -1 osd.9 95 *** Got signal Terminated *** 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.392+0000 7fdc032fd700 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:39.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:39 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:39.450+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:39.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:39 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:39.372+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:39.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:39 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:39.372+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:39.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:39 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:39.372+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:39.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:39 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:39.372+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:39.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:39 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:39.372+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:40.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:40.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:40.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:40.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:40.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:40.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:40 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:40.411+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:40.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:40.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:40.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:40.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:40.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:40.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:40 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:40.446+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:40.793 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:40 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:40.411+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:40.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:40 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:40.411+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:40.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:40 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:40.411+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:40.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:40 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:40.411+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:40.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:40 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:40.411+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:41.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:41.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:41.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:41.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:41.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:41.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:41 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:41.396+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:41.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:41.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:41.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:41.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:41.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:41.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:41 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:41.434+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:41.793 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:41.365+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:41.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:41.365+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:41.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:41.365+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:41.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:41.365+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:41.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:41.365+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:42.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:42.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:42.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:42.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:42.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:42.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:42.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:42 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:42.357+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:42.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:42.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:42.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:42.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:42.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:42.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:42 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:42.454+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:42.793 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:42 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:42.351+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:42.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:42 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:42.351+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:42.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:42 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:42.351+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:42.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:42 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:42.351+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:42.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:42 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:42.351+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:43.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:43.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:43.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:43.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:43.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:43.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:43.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:43 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:43.326+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:43.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.492036+0000 front 2023-08-11T14:23:31.492186+0000 (oldest deadline 2023-08-11T14:23:57.392304+0000) 2023-08-11T14:24:43.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:37.392871+0000 front 2023-08-11T14:23:37.393218+0000 (oldest deadline 2023-08-11T14:24:01.393851+0000) 2023-08-11T14:24:43.791 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.996387+0000 front 2023-08-11T14:23:45.996226+0000 (oldest deadline 2023-08-11T14:24:08.896042+0000) 2023-08-11T14:24:43.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:56.398482+0000 front 2023-08-11T14:23:56.398440+0000 (oldest deadline 2023-08-11T14:24:22.298527+0000) 2023-08-11T14:24:43.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.299312+0000 front 2023-08-11T14:24:02.299323+0000 (oldest deadline 2023-08-11T14:24:28.198930+0000) 2023-08-11T14:24:43.792 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:43 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9[126543]: 2023-08-11T14:24:43.431+0000 7fdbfd0ee700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:08.199733+0000 front 2023-08-11T14:24:08.199814+0000 (oldest deadline 2023-08-11T14:24:34.099420+0000) 2023-08-11T14:24:43.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:43.334+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:43.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:43.334+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:43.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:43.334+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:43.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:43.334+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:43.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:43 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:43.334+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:44.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:44.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:44.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:44.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:44.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:44.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:44.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:44 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:44.295+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:44.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:44.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:44.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:44.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:44.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:44.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:44 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:44.331+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:45.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:45.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:45.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:45.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:45.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:45.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:45.682 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:45 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:45.305+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:45.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:45.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:45.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:45.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:45.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:45.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:45.292+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:46.041 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 11 14:24:45 smithi003 bash[161523]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-9 2023-08-11T14:24:46.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:46.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:46.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:46.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:46.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:46.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:46 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:46.251+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:46.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:46.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:46.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:46.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:46.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:46.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:46.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:46 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:46.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:47.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:47.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:47.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:47.544 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:47.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:47.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:47 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:47.281+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:47.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.9.service' 2023-08-11T14:24:47.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:47.678 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-08-11T14:24:47.679 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-08-11T14:24:47.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.10 2023-08-11T14:24:47.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:47.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:47.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:47.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:47.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:47.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:47.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:47.353+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:48.026 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:47 smithi103 systemd[1]: Stopping Ceph osd.10 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:48.307 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.025+0000 7fba16a5b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:48.307 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.025+0000 7fba16a5b700 -1 osd.10 95 *** Got signal Terminated *** 2023-08-11T14:24:48.307 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.025+0000 7fba16a5b700 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:48.308 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:48.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:48.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:48.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:48.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:48.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:48.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:48 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:48.306+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:48.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:48.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:48.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:48.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:48.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:48.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:48 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:48.308+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:49.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:49.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:49.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:49.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:49.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:49.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:49.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:49 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:49.340+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:49.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:49.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:49.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:49.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:49.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:49.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:49.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:50.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:50.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:50.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:50.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:50.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:50.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:50.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:50 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:50.312+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:50.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:50.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:50.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:50.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:50.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:50.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:50 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:50.341+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:51.678 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:51.679 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:51 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:51.283+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:51.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:51.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:51.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:51.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:51.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:51.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:51 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:51.318+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:52.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:27.597198+0000 front 2023-08-11T14:23:27.597325+0000 (oldest deadline 2023-08-11T14:23:52.798084+0000) 2023-08-11T14:24:52.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2023-08-11T14:23:40.201246+0000 front 2023-08-11T14:23:40.201274+0000 (oldest deadline 2023-08-11T14:24:05.501387+0000) 2023-08-11T14:24:52.680 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.502085+0000 front 2023-08-11T14:23:45.501981+0000 (oldest deadline 2023-08-11T14:24:11.402132+0000) 2023-08-11T14:24:52.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:51.402649+0000 front 2023-08-11T14:23:51.402609+0000 (oldest deadline 2023-08-11T14:24:17.302781+0000) 2023-08-11T14:24:52.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:01.403743+0000 front 2023-08-11T14:24:01.403828+0000 (oldest deadline 2023-08-11T14:24:26.703936+0000) 2023-08-11T14:24:52.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.204983+0000 front 2023-08-11T14:24:10.204943+0000 (oldest deadline 2023-08-11T14:24:37.205590+0000) 2023-08-11T14:24:52.681 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:52 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10[151836]: 2023-08-11T14:24:52.259+0000 7fba1204f700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:17.706673+0000 front 2023-08-11T14:24:17.706607+0000 (oldest deadline 2023-08-11T14:24:42.307044+0000) 2023-08-11T14:24:52.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:52.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:52.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:52.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:52.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:52.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:52 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:52.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:53.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:53.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:53.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:53.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:53.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:53.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:53 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:53.324+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:54.178 INFO:journalctl@ceph.osd.10.smithi103.stdout:Aug 11 14:24:53 smithi103 bash[162815]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-10 2023-08-11T14:24:54.545 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:54.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:54.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:54.546 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:54.547 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:54.547 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:54.548 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:54 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:54.287+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:55.330 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.10.service' 2023-08-11T14:24:55.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:24:55.372 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-08-11T14:24:55.372 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-08-11T14:24:55.372 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.11 2023-08-11T14:24:55.725 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:55.725 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:55.725 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.300+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 systemd[1]: Stopping Ceph osd.11 for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.723+0000 7f47367ea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:24:55.726 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.723+0000 7f47367ea700 -1 osd.11 95 *** Got signal Terminated *** 2023-08-11T14:24:55.727 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:55 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:55.723+0000 7f47367ea700 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-08-11T14:24:56.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:56.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:56.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:56.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:56.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:56.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:56.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:56 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:56.346+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:57.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:57.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:57.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:57.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:57.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:57.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:57.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:57 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:57.340+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:58.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:58.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:58.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:58.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:58.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:58.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:58.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:58 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:58.384+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:59.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:24:59.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:24:59.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:24:59.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:24:59.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:24:59.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:24:59.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:24:59.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:24:59 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:24:59.350+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2023-08-11T14:24:34.159447+0000 front 2023-08-11T14:24:34.159480+0000 (oldest deadline 2023-08-11T14:24:58.859439+0000) 2023-08-11T14:25:00.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2023-08-11T14:23:31.248035+0000 front 2023-08-11T14:23:31.247893+0000 (oldest deadline 2023-08-11T14:23:52.848607+0000) 2023-08-11T14:25:00.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-08-11T14:23:45.751119+0000 front 2023-08-11T14:23:45.751148+0000 (oldest deadline 2023-08-11T14:24:09.251280+0000) 2023-08-11T14:25:00.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2023-08-11T14:23:53.753453+0000 front 2023-08-11T14:23:53.753440+0000 (oldest deadline 2023-08-11T14:24:17.853505+0000) 2023-08-11T14:25:00.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2023-08-11T14:24:02.554784+0000 front 2023-08-11T14:24:02.554809+0000 (oldest deadline 2023-08-11T14:24:27.854978+0000) 2023-08-11T14:25:00.794 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-08-11T14:24:10.156033+0000 front 2023-08-11T14:24:10.156012+0000 (oldest deadline 2023-08-11T14:24:36.056238+0000) 2023-08-11T14:25:00.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2023-08-11T14:24:20.058031+0000 front 2023-08-11T14:24:20.058060+0000 (oldest deadline 2023-08-11T14:24:43.558102+0000) 2023-08-11T14:25:00.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2023-08-11T14:24:28.259119+0000 front 2023-08-11T14:24:28.259211+0000 (oldest deadline 2023-08-11T14:24:54.159079+0000) 2023-08-11T14:25:00.795 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:00 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11[152967]: 2023-08-11T14:25:00.339+0000 7f47305db700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2023-08-11T14:24:34.159447+0000 front 2023-08-11T14:24:34.159480+0000 (oldest deadline 2023-08-11T14:24:58.859439+0000) 2023-08-11T14:25:01.608 INFO:journalctl@ceph.osd.11.smithi187.stdout:Aug 11 14:25:01 smithi187 bash[164743]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-osd-11 2023-08-11T14:25:02.866 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@osd.11.service' 2023-08-11T14:25:02.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:02.908 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-08-11T14:25:02.909 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-08-11T14:25:02.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.a 2023-08-11T14:25:03.291 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:25:03 smithi003 systemd[1]: Stopping Ceph mds.a for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:03.791 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:25:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: 2023-08-11T14:25:03.445+0000 7fb3d584a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:03.791 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:25:03 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a[131967]: 2023-08-11T14:25:03.445+0000 7fb3d584a700 -1 mds.a *** got signal Terminated *** 2023-08-11T14:25:08.041 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 11 14:25:07 smithi003 bash[161805]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-a 2023-08-11T14:25:08.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.a.service' 2023-08-11T14:25:08.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:08.272 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-08-11T14:25:08.272 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-08-11T14:25:08.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.d 2023-08-11T14:25:08.541 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:25:08 smithi003 systemd[1]: Stopping Ceph mds.d for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:09.291 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:25:08 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-d[130972]: 2023-08-11T14:25:08.794+0000 7f8b9240c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:09.291 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:25:08 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-d[130972]: 2023-08-11T14:25:08.794+0000 7f8b9240c700 -1 mds.d *** got signal Terminated *** 2023-08-11T14:25:12.291 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 11 14:25:12 smithi003 bash[161952]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-d 2023-08-11T14:25:12.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.d.service' 2023-08-11T14:25:12.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:12.583 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-08-11T14:25:12.584 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-08-11T14:25:12.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.g 2023-08-11T14:25:12.791 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:25:12 smithi003 systemd[1]: Stopping Ceph mds.g for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:13.541 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:25:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: 2023-08-11T14:25:13.110+0000 7fa4290eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:13.541 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:25:13 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g[131580]: 2023-08-11T14:25:13.110+0000 7fa4290eb700 -1 mds.g *** got signal Terminated *** 2023-08-11T14:25:17.041 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 11 14:25:16 smithi003 bash[162100]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-g 2023-08-11T14:25:17.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.g.service' 2023-08-11T14:25:17.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:17.450 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-08-11T14:25:17.450 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-08-11T14:25:17.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.j 2023-08-11T14:25:17.792 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:17 smithi003 systemd[1]: Stopping Ceph mds.j for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:18.291 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-j[130586]: 2023-08-11T14:25:18.001+0000 7f6e03980700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:18.291 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:18 smithi003 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-j[130586]: 2023-08-11T14:25:18.001+0000 7f6e03980700 -1 mds.j *** got signal Terminated *** 2023-08-11T14:25:20.971 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:20 smithi003 bash[162244]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-j 2023-08-11T14:25:21.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.j.service' 2023-08-11T14:25:21.245 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:21 smithi003 systemd[1]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.j.service: Succeeded. 2023-08-11T14:25:21.245 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 11 14:25:21 smithi003 systemd[1]: Stopped Ceph mds.j for 69ee59aa-384e-11ee-9b37-001a4aab830c. 2023-08-11T14:25:21.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:21.270 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-08-11T14:25:21.270 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-08-11T14:25:21.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.b 2023-08-11T14:25:21.630 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:25:21 smithi103 systemd[1]: Stopping Ceph mds.b for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:21.928 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:25:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: 2023-08-11T14:25:21.628+0000 7f4127b94700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:21.928 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:25:21 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b[155603]: 2023-08-11T14:25:21.628+0000 7f4127b94700 -1 mds.b *** got signal Terminated *** 2023-08-11T14:25:24.678 INFO:journalctl@ceph.mds.b.smithi103.stdout:Aug 11 14:25:24 smithi103 bash[163095]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-b 2023-08-11T14:25:24.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.b.service' 2023-08-11T14:25:24.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:24.920 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-08-11T14:25:24.920 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-08-11T14:25:24.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.e 2023-08-11T14:25:25.178 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:25:24 smithi103 systemd[1]: Stopping Ceph mds.e for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:25.678 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:25:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: 2023-08-11T14:25:25.302+0000 7f3535f09700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:25.678 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:25:25 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e[154921]: 2023-08-11T14:25:25.302+0000 7f3535f09700 -1 mds.e *** got signal Terminated *** 2023-08-11T14:25:28.668 INFO:journalctl@ceph.mds.e.smithi103.stdout:Aug 11 14:25:28 smithi103 bash[163236]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-e 2023-08-11T14:25:28.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.e.service' 2023-08-11T14:25:28.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:28.889 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-08-11T14:25:28.889 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-08-11T14:25:28.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.h 2023-08-11T14:25:29.178 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:25:28 smithi103 systemd[1]: Stopping Ceph mds.h for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:29.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:25:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: 2023-08-11T14:25:29.260+0000 7fb796de6700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:29.678 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:25:29 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h[155263]: 2023-08-11T14:25:29.260+0000 7fb796de6700 -1 mds.h *** got signal Terminated *** 2023-08-11T14:25:32.871 INFO:journalctl@ceph.mds.h.smithi103.stdout:Aug 11 14:25:32 smithi103 bash[163377]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-h 2023-08-11T14:25:32.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.h.service' 2023-08-11T14:25:32.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:32.910 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-08-11T14:25:32.910 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-08-11T14:25:32.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.k 2023-08-11T14:25:33.178 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:25:32 smithi103 systemd[1]: Stopping Ceph mds.k for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:33.678 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:25:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: 2023-08-11T14:25:33.276+0000 7f135627c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:33.678 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:25:33 smithi103 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k[154567]: 2023-08-11T14:25:33.277+0000 7f135627c700 -1 mds.k *** got signal Terminated *** 2023-08-11T14:25:36.858 INFO:journalctl@ceph.mds.k.smithi103.stdout:Aug 11 14:25:36 smithi103 bash[163523]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-k 2023-08-11T14:25:36.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.k.service' 2023-08-11T14:25:36.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:36.900 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-08-11T14:25:36.900 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-08-11T14:25:36.900 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.c 2023-08-11T14:25:37.261 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:25:36 smithi187 systemd[1]: Stopping Ceph mds.c for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:37.544 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:25:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: 2023-08-11T14:25:37.259+0000 7fb712de5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:37.544 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:25:37 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c[156558]: 2023-08-11T14:25:37.259+0000 7fb712de5700 -1 mds.c *** got signal Terminated *** 2023-08-11T14:25:40.727 INFO:journalctl@ceph.mds.c.smithi187.stdout:Aug 11 14:25:40 smithi187 bash[165018]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-c 2023-08-11T14:25:40.915 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.c.service' 2023-08-11T14:25:40.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:40.953 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-08-11T14:25:40.953 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-08-11T14:25:40.953 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.f 2023-08-11T14:25:41.294 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:25:41 smithi187 systemd[1]: Stopping Ceph mds.f for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:41.794 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:25:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: 2023-08-11T14:25:41.333+0000 7f70d3bf9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:41.794 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:25:41 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f[156219]: 2023-08-11T14:25:41.333+0000 7f70d3bf9700 -1 mds.f *** got signal Terminated *** 2023-08-11T14:25:44.699 INFO:journalctl@ceph.mds.f.smithi187.stdout:Aug 11 14:25:44 smithi187 bash[165158]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-f 2023-08-11T14:25:44.882 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.f.service' 2023-08-11T14:25:44.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:44.917 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-08-11T14:25:44.917 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-08-11T14:25:44.917 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.i 2023-08-11T14:25:45.291 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:25:44 smithi187 systemd[1]: Stopping Ceph mds.i for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:45.543 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:25:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:25:45.290+0000 7fb643b92700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:45.544 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:25:45 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i[156901]: 2023-08-11T14:25:45.290+0000 7fb643b92700 -1 mds.i *** got signal Terminated *** 2023-08-11T14:25:48.933 INFO:journalctl@ceph.mds.i.smithi187.stdout:Aug 11 14:25:48 smithi187 bash[165297]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-i 2023-08-11T14:25:48.938 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.i.service' 2023-08-11T14:25:48.972 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:48.972 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-08-11T14:25:48.973 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-08-11T14:25:48.973 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.l 2023-08-11T14:25:49.294 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:25:49 smithi187 systemd[1]: Stopping Ceph mds.l for 69ee59aa-384e-11ee-9b37-001a4aab830c... 2023-08-11T14:25:49.794 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:25:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: 2023-08-11T14:25:49.346+0000 7ff931804700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-08-11T14:25:49.794 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:25:49 smithi187 ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l[155860]: 2023-08-11T14:25:49.346+0000 7ff931804700 -1 mds.l *** got signal Terminated *** 2023-08-11T14:25:52.794 INFO:journalctl@ceph.mds.l.smithi187.stdout:Aug 11 14:25:52 smithi187 bash[165448]: ceph-69ee59aa-384e-11ee-9b37-001a4aab830c-mds-l 2023-08-11T14:25:52.811 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-69ee59aa-384e-11ee-9b37-001a4aab830c@mds.l.service' 2023-08-11T14:25:52.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-08-11T14:25:52.845 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-08-11T14:25:52.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T14:25:53.242 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:25:56.821 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T14:25:57.210 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:25:58.798 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force --keep-logs 2023-08-11T14:25:59.100 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:26:00.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:26:00.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:26:00.803 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-08-11T14:26:00.834 INFO:tasks.cephadm:Archiving crash dumps... 2023-08-11T14:26:00.836 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi003/crash 2023-08-11T14:26:00.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T14:26:00.923 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T14:26:00.924 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2023-08-11T14:26:00.927 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi103/crash 2023-08-11T14:26:00.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T14:26:00.978 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T14:26:00.978 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2023-08-11T14:26:00.984 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi187/crash 2023-08-11T14:26:00.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash -- . 2023-08-11T14:26:01.038 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/crash: Cannot open: No such file or directory 2023-08-11T14:26:01.039 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2023-08-11T14:26:01.040 INFO:tasks.cephadm:Checking cluster log for badness... 2023-08-11T14:26:01.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-08-11T14:26:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:grep: /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph.log: No such file or directory 2023-08-11T14:26:01.131 INFO:tasks.cephadm:Compressing logs... 2023-08-11T14:26:01.132 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T14:26:01.136 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T14:26:01.139 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-11T14:26:01.181 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-08-11T14:26:01.181 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T14:26:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-08-11T14:26:01.187 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T14:26:01.189 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.5.log 2023-08-11T14:26:01.192 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.2.log 2023-08-11T14:26:01.192 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.11.log 2023-08-11T14:26:01.192 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.z.log 2023-08-11T14:26:01.193 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.8.log 2023-08-11T14:26:01.193 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.f.log 2023-08-11T14:26:01.193 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.c.log 2023-08-11T14:26:01.193 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.i.log 2023-08-11T14:26:01.194 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.c.log 2023-08-11T14:26:01.194 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.c.log: /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T14:26:01.194 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.l.log 2023-08-11T14:26:01.195 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log: /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.l.log: 93.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.z.log.gz 2023-08-11T14:26:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.10.log 2023-08-11T14:26:01.208 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.h.log 2023-08-11T14:26:01.209 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.e.log 2023-08-11T14:26:01.209 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.10.log: /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.4.log 2023-08-11T14:26:01.209 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.1.log 2023-08-11T14:26:01.209 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.k.log 2023-08-11T14:26:01.210 INFO:teuthology.orchestra.run.smithi187.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T14:26:01.211 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T14:26:01.211 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.7.log 2023-08-11T14:26:01.211 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.b.log 2023-08-11T14:26:01.211 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.b.log 2023-08-11T14:26:01.211 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.y.log 2023-08-11T14:26:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.b.log: /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.y.log: 93.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.y.log.gz 2023-08-11T14:26:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-08-11T14:26:01.228 INFO:teuthology.orchestra.run.smithi003.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-08-11T14:26:01.237 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-08-11T14:26:01.238 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.0.log 2023-08-11T14:26:01.239 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.g.log 2023-08-11T14:26:01.240 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.a.log 2023-08-11T14:26:01.241 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.6.log 2023-08-11T14:26:01.241 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.3.log 2023-08-11T14:26:01.242 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.d.log 2023-08-11T14:26:01.243 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.x.log 2023-08-11T14:26:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.9.log 2023-08-11T14:26:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log 2023-08-11T14:26:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.j.log 2023-08-11T14:26:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.a.log 2023-08-11T14:26:01.249 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.j.log: 94.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.d.log.gz 2023-08-11T14:26:01.256 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.a.log: 94.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.j.log.gz 2023-08-11T14:26:01.374 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-08-11T14:26:01.582 INFO:teuthology.orchestra.run.smithi187.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T14:26:01.698 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.l.log.gz 2023-08-11T14:26:01.828 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.a.log.gz 2023-08-11T14:26:01.996 INFO:teuthology.orchestra.run.smithi103.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T14:26:02.232 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T14:26:02.266 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-volume.log.gz 2023-08-11T14:26:02.282 INFO:teuthology.orchestra.run.smithi003.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-08-11T14:26:02.413 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mgr.x.log.gz 2023-08-11T14:26:02.919 INFO:teuthology.orchestra.run.smithi003.stderr: 93.9% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.g.log.gz 2023-08-11T14:26:03.096 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.c.log.gz 2023-08-11T14:26:03.262 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.k.log.gz 2023-08-11T14:26:03.357 INFO:teuthology.orchestra.run.smithi103.stderr: 92.1% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.b.log.gz 2023-08-11T14:26:03.505 INFO:teuthology.orchestra.run.smithi187.stderr: 93.4% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.c.log.gz 2023-08-11T14:26:03.791 INFO:teuthology.orchestra.run.smithi187.stderr: 94.6% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.f.log.gz 2023-08-11T14:26:04.003 INFO:teuthology.orchestra.run.smithi103.stderr: 94.0% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.h.log.gz 2023-08-11T14:26:05.068 INFO:teuthology.orchestra.run.smithi003.stderr: 90.4% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mon.a.log.gz 2023-08-11T14:26:05.587 INFO:teuthology.orchestra.run.smithi103.stderr: 93.4% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.e.log.gz 2023-08-11T14:26:10.030 INFO:teuthology.orchestra.run.smithi187.stderr: 94.1% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.i.log.gz 2023-08-11T14:26:33.526 INFO:teuthology.orchestra.run.smithi103.stderr: 94.0% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-mds.b.log.gz 2023-08-11T14:29:13.119 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.10.log.gz 2023-08-11T14:29:46.562 INFO:teuthology.orchestra.run.smithi187.stderr: 91.1% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.8.log.gz 2023-08-11T14:29:59.751 INFO:teuthology.orchestra.run.smithi187.stderr: 91.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.11.log.gz 2023-08-11T14:30:05.885 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.1.log.gz 2023-08-11T14:30:52.132 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.7.log.gz 2023-08-11T14:30:59.275 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.2.log.gz 2023-08-11T14:31:31.596 INFO:teuthology.orchestra.run.smithi103.stderr: 91.5% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.4.log.gz 2023-08-11T14:31:31.599 INFO:teuthology.orchestra.run.smithi103.stderr: 2023-08-11T14:31:31.599 INFO:teuthology.orchestra.run.smithi103.stderr:real 5m30.443s 2023-08-11T14:31:31.599 INFO:teuthology.orchestra.run.smithi103.stderr:user 7m41.437s 2023-08-11T14:31:31.599 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m17.305s 2023-08-11T14:31:36.025 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.0.log.gz 2023-08-11T14:31:56.038 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.3.log.gz 2023-08-11T14:32:05.062 INFO:teuthology.orchestra.run.smithi187.stderr: 91.2% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.5.log.gz 2023-08-11T14:32:05.065 INFO:teuthology.orchestra.run.smithi187.stderr: 2023-08-11T14:32:05.066 INFO:teuthology.orchestra.run.smithi187.stderr:real 6m3.911s 2023-08-11T14:32:05.066 INFO:teuthology.orchestra.run.smithi187.stderr:user 8m12.801s 2023-08-11T14:32:05.066 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m18.706s 2023-08-11T14:32:10.327 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.6.log.gz 2023-08-11T14:32:52.916 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/69ee59aa-384e-11ee-9b37-001a4aab830c/ceph-osd.9.log.gz 2023-08-11T14:32:52.919 INFO:teuthology.orchestra.run.smithi003.stderr: 2023-08-11T14:32:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:real 6m51.747s 2023-08-11T14:32:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:user 9m47.538s 2023-08-11T14:32:52.920 INFO:teuthology.orchestra.run.smithi003.stderr:sys 1m18.925s 2023-08-11T14:32:52.921 INFO:tasks.cephadm:Archiving logs... 2023-08-11T14:32:52.922 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi003/log 2023-08-11T14:32:52.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T14:34:11.851 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi103/log 2023-08-11T14:34:11.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T14:35:18.633 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi187/log 2023-08-11T14:35:18.634 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-11T14:36:29.862 INFO:tasks.cephadm:Removing cluster... 2023-08-11T14:36:29.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force 2023-08-11T14:36:30.558 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:36:33.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force 2023-08-11T14:36:34.009 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:36:35.043 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 69ee59aa-384e-11ee-9b37-001a4aab830c --force 2023-08-11T14:36:35.595 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 69ee59aa-384e-11ee-9b37-001a4aab830c 2023-08-11T14:36:36.667 INFO:tasks.cephadm:Removing cephadm ... 2023-08-11T14:36:36.668 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T14:36:36.717 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T14:36:36.757 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/cephadm 2023-08-11T14:36:36.785 INFO:tasks.cephadm:Teardown complete 2023-08-11T14:36:36.786 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-11T14:36:36.798 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-11T14:36:36.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T14:36:36.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T14:36:36.806 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-11T14:36:36.969 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-11T14:36:36.979 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-11T14:36:36.980 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T14:36:36.985 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T14:36:36.987 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-11T14:36:37.003 INFO:teuthology.orchestra.run.smithi187.stderr:bash: ntpq: command not found 2023-08-11T14:36:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:bash: ntpq: command not found 2023-08-11T14:36:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:bash: ntpq: command not found 2023-08-11T14:36:37.025 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T14:36:37.025 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2023-08-11T14:36:37.026 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +1152us[+1171us] +/- 75ms 2023-08-11T14:36:37.026 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1134us[-1115us] +/- 57ms 2023-08-11T14:36:37.026 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +491us[ +491us] +/- 75ms 2023-08-11T14:36:37.026 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T14:36:37.027 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T14:36:37.028 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2023-08-11T14:36:37.028 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +1139us[+1139us] +/- 75ms 2023-08-11T14:36:37.028 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1143us[-1140us] +/- 57ms 2023-08-11T14:36:37.028 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +502us[ +502us] +/- 75ms 2023-08-11T14:36:37.028 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T14:36:37.045 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-11T14:36:37.045 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2023-08-11T14:36:37.045 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 97 +1142us[+1137us] +/- 74ms 2023-08-11T14:36:37.045 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv02.front.sepia.ceph.com 3 7 377 40 -1147us[-1152us] +/- 57ms 2023-08-11T14:36:37.045 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 98 +488us[ +483us] +/- 74ms 2023-08-11T14:36:37.046 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-08-11T14:36:37.046 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-11T14:36:37.064 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-11T14:36:37.066 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-11T14:36:37.078 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T14:36:37.327 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T14:36:37.515 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-11T14:36:37.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T14:36:37.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:37.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T14:36:37.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:37.799 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-11T14:36:37.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:37.834 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-11T14:36:37.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-11T14:36:37.867 INFO:teuthology.task.internal:Duration was 4026.163303 seconds 2023-08-11T14:36:37.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-11T14:36:37.881 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-11T14:36:37.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T14:36:37.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T14:36:37.889 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-11T14:36:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T14:36:37.978 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T14:36:38.034 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-11T14:36:38.583 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-11T14:36:38.583 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2023-08-11T14:36:38.583 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T14:36:38.671 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2023-08-11T14:36:38.671 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T14:36:38.700 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2023-08-11T14:36:38.701 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-11T14:36:38.727 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-11T14:36:38.728 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T14:36:38.732 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T14:36:38.745 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-11T14:36:38.863 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-11T14:36:38.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T14:36:38.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T14:36:38.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-11T14:36:39.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-11T14:36:39.068 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-11T14:36:39.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T14:36:39.152 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T14:36:39.202 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-11T14:36:39.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-11T14:36:39.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.254 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.297 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2023-08-11T14:36:39.312 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2023-08-11T14:36:39.321 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2023-08-11T14:36:39.359 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:39.400 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:39.418 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-11T14:36:39.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-11T14:36:39.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-11T14:36:39.446 INFO:teuthology.task.internal:Transferring archived files... 2023-08-11T14:36:39.447 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi003 2023-08-11T14:36:39.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T14:36:39.535 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi103 2023-08-11T14:36:39.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T14:36:39.576 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898/remote/smithi187 2023-08-11T14:36:39.577 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-11T14:36:39.619 INFO:teuthology.task.internal:Removing archive directory... 2023-08-11T14:36:39.619 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T14:36:39.625 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T14:36:39.627 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-11T14:36:39.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-11T14:36:39.684 INFO:teuthology.task.internal:Not uploading archives. 2023-08-11T14:36:39.685 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-11T14:36:39.700 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-11T14:36:39.700 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T14:36:39.707 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T14:36:39.710 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-11T14:36:39.724 INFO:teuthology.orchestra.run.smithi103.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 14:36 /home/ubuntu/cephtest 2023-08-11T14:36:39.727 INFO:teuthology.orchestra.run.smithi187.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 14:36 /home/ubuntu/cephtest 2023-08-11T14:36:39.746 INFO:teuthology.orchestra.run.smithi003.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 11 14:36 /home/ubuntu/cephtest 2023-08-11T14:36:39.749 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-11T14:36:39.763 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-11T14:36:39.797 INFO:teuthology.nuke:Checking targets against current locks 2023-08-11T14:36:39.823 DEBUG:teuthology.nuke:shortname: smithi003 2023-08-11T14:36:39.823 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T14:36:39.845 DEBUG:teuthology.nuke:shortname: smithi103 2023-08-11T14:36:39.845 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T14:36:39.848 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.764735', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-11T14:36:39.871 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.766683', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-11T14:36:39.872 DEBUG:teuthology.nuke:shortname: smithi187 2023-08-11T14:36:39.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T14:36:39.890 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365898', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:18:42.768394', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-11T14:36:39.899 INFO:teuthology.orchestra.console.smithi003:Power off 2023-08-11T14:36:39.900 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T14:36:39.925 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-08-11T14:36:39.926 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:39.944 INFO:teuthology.orchestra.console.smithi103:Power off 2023-08-11T14:36:39.945 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T14:36:39.959 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-08-11T14:36:39.969 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off 2023-08-11T14:36:39.969 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:39.988 INFO:teuthology.orchestra.console.smithi187:Power off 2023-08-11T14:36:39.989 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T14:36:40.000 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2023-08-11T14:36:40.013 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2023-08-11T14:36:40.013 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:40.039 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-08-11T14:36:43.961 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:44.002 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:44.041 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:44.073 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-08-11T14:36:44.115 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2023-08-11T14:36:44.152 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-08-11T14:36:48.076 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:48.116 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:48.153 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:48.190 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-08-11T14:36:48.229 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2023-08-11T14:36:48.269 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-08-11T14:36:48.331 INFO:teuthology.orchestra.console.smithi103:Power off completed 2023-08-11T14:36:52.195 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:52.270 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T14:36:52.308 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-08-11T14:36:52.384 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-08-11T14:36:52.410 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-08-11T14:36:52.486 INFO:teuthology.orchestra.console.smithi187:Power off completed 2023-08-11T14:36:52.591 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a7fb1265323db972dd333f71b9a53e9479f62e37 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 4026.163302898407 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-08-11T14:36:52.592 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T14:36:52.683 INFO:teuthology.run:FAIL